Can golang benchmark give a custom output?

Question

I'm benchmarking code with different list sizes (lists of size S) Go benchmark shows ns/op but what I want is (ns/op)/S.

In other words, the output of go test -bench=. is:

BenchmarkMy10-4         100000000           15.7 ns/op
BenchmarkMy20-4         50000000            33.8 ns/op
BenchmarkMy30-4         30000000            43.8 ns/op
BenchmarkMy40-4         30000000            49.3 ns/op
BenchmarkMy50-4         30000000            56.6 ns/op
BenchmarkMy1000-4        2000000           686 ns/op
BenchmarkMy10000-4        200000          6685 ns/op
BenchmarkMy100000-4        20000         65425 ns/op

The "10" in "My10" represents a list of 10 items (S=10).

While it is useful to know the ns/op for different list sizes, I would also like to know the ns/op/S (time per item in the list).

Right now I'm pasting the results into a spreadsheet and doing the math there. However I'd like to have "go test" output this information for me.

My main_test.go file looks like:

import "testing"

var result int

func benchmarkMy(i int, b *testing.B) {
  var r int
  mylist := MakeList(i)
  b.ResetTimer()
  for n := 0; n < b.N; n++ {
    r = My(mylist)
  }
  result = r
}

func BenchmarkMy10(b *testing.B)         { benchmarkMy(10, b) }
func BenchmarkMy20(b *testing.B)         { benchmarkMy(20, b) }
func BenchmarkMy30(b *testing.B)         { benchmarkMy(30, b) }
func BenchmarkMy40(b *testing.B)         { benchmarkMy(40, b) }
func BenchmarkMy50(b *testing.B)         { benchmarkMy(50, b) }
func BenchmarkMy1000(b *testing.B)       { benchmarkMy(1000, b) }
func BenchmarkMy10000(b *testing.B)      { benchmarkMy(10000, b) }
func BenchmarkMy100000(b *testing.B)     { benchmarkMy(100000, b) }

It seems like the test.BenchmarkResult structure has the information I need, but I don't see how to use this structure.

No, stdandard Benchmark cannot do this simply because it is not generally useful. ns/op/S might be interesting for algorithms which are O(1) only. — Volker
– Volker, Commented Jun 26, 2016 at 14:52
@Volker why it wouldn't be useful to output custom results, i.e. track other values? What if someone wanted to measure compression ratio? — kravemir
– kravemir, Commented Jun 22, 2021 at 16:24
@kravemir Since Go 1.13 you can report custom metrics, but this is of limited use and your example of compression ration seems strange (unless we are talking about a randomized algorithm). — Volker
– Volker, Commented Jun 23, 2021 at 11:05

Aruna Herath · Accepted Answer · 2016-06-26 16:53:47Z

12

You can write a custom benchmark using Benchmark function from package testing. And get a BenchmarkResult instance you mention.

package main

import (
    "fmt"
    "testing"
)

func benchmarkMy(i int) {
    fn := func(b *testing.B) {
         // Code you want benchmarked
    }
    r := testing.Benchmark(fn)

    fmt.Printf("%d ns/op\n", int(r.T)/r.N)
    fmt.Printf("%d ns/op/i\n", int(r.T)/r.N/i)
}

func main() {
    benchmarkMy(10)
}

You'll have to put this in a different package and run with go run instead of go test.

Check an example in the playground.

edited Jun 26, 2016 at 16:53

answered Jun 26, 2016 at 14:26

Aruna Herath

6,5822 gold badges43 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Aruna Herath Over a year ago

I haven't done this before and couldn't find much supporting documents. To me this seems to be the way to do it.

francoposa · Accepted Answer · 2025-03-12 20:47:10Z

The accepted answer from Aruna is not the best way as of Go 1.13 (released Feb 2019).
Now the testing package itself includes B.ReportMetric.

The documentation is a bit lacking for actual good examples, but there is a nice concise example at this blog:

func BenchmarkSort(b *testing.B) {
    var cmps int64 // <--------
    data := make([]int, 1<<10)
    b.ResetTimer() // remove allocation
    for i := 0; i < b.N; i++ {
        for i := range data {
            data[i] = i ^ 0x2cc
        }
        b.StartTimer()
        sort.Slice(data, func(i, j int) bool {
            cmps++ // <--------
            return data[i] < data[j]
        })
        b.StopTimer()
    }
    b.ReportMetric(float64(cmps)/float64(b.N), "cmps/op") // <--------
}

Which would produce output like:

$ go test -bench=. -count=5
goos: darwin
goarch: amd64
pkg: github.com/commaok/blog/content/post
BenchmarkSort-8        18523         69874 ns/op         10091 cmps/op
BenchmarkSort-8        18654         64621 ns/op         10091 cmps/op
BenchmarkSort-8        18657         64589 ns/op         10091 cmps/op
BenchmarkSort-8        18774         64367 ns/op         10091 cmps/op
BenchmarkSort-8        18704         64408 ns/op         10091 cmps/op
PASS
ok      github.com/commaok/blog/content/post    12.062s

Collectives™ on Stack Overflow

Can golang benchmark give a custom output?

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related