cmd/go: go test -c is slower than go test

[onsi]

Go version

go version go1.24.1 darwin/arm64

Output of go env in your module/workspace:

AR='ar'
CC='cc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='c++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN='/Users/onsi/bin'
GOCACHE='/Users/onsi/Library/Caches/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/Users/onsi/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/rb/2gkngfn90ts_trzmrly1v9q00000gn/T/go-build785404135=/tmp/go-build -gno-record-gcc-switches -fno-common'
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMOD='/Users/onsi/code/browser/go.mod'
GOMODCACHE='/Users/onsi/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/onsi/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/homebrew/Cellar/go/1.24.1/libexec'
GOSUMDB='sum.golang.org'
GOTELEMETRY='on'
GOTELEMETRYDIR='/Users/onsi/Library/Application Support/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.24.1/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.24.1'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

I'm the author/maintainer of Ginkgo and have long struggled to get ginkgo's compile and launch performance to match go tests. For reasons related to ginkgo's architecture it's simpler for me to run go test -c to produce a binary file then run that file.

But I'm seeing that go test -c can be substantially slower than go test and I'd like to understand if this is a bug or if there's something I can do in how I invoke the go toolchain to improve performance.

What did you see happen?

As a simple reproducer you can clone https://github.com/gost-dom/browser (not my repo - but it exemplifies the problem well) and checkout sha 390898e.

What did you expect to see?

From the root directory of the repo I find:

code/browser » time go test -count=1
...
go test -count=1  1.21s user 1.20s system 261% cpu 0.922 total

vs

code/browser » time go test -c
go test -c  2.70s user 1.34s system 171% cpu 2.351 total

Am I holding it wrong?

[ianlancetaylor]

I don't know if this is the issue, but I'll note that one difference is that plain go test builds the test program without debug information, whereas go test -c includes debug information. You could try go test -c -gcflags=-dwarf=false to see if that makes a difference.

[seankhliao]

Is your filesystem just slow to write out a binary?
On a linux system, I get:

$ time go test -c
# github.com/gost-dom/v8go
In file included from deps/include/v8-isolate.h:22,
                 from deps/include/v8-snapshot.h:9,
                 from deps/include/v8-context.h:15,
                 from object_template.cc:4:
deps/include/v8-function-callback.h: In instantiation of ‘v8::Isolate* v8::PropertyCallbackInfo<F>::GetIsolate() const [with T = v8::Value]’:
object_template.cc:16:33:   required from here
   16 |   Isolate* iso = info.GetIsolate();
      |                  ~~~~~~~~~~~~~~~^~
deps/include/v8-function-callback.h:695:11: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
  695 |   return *reinterpret_cast<Isolate**>(&args_[kIsolateIndex]);
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
go test -c  0.58s user 0.16s system 512% cpu 0.145 total

$ time go test -count 1
# github.com/gost-dom/v8go
In file included from deps/include/v8-isolate.h:22,
                 from deps/include/v8-snapshot.h:9,
                 from deps/include/v8-context.h:15,
                 from object_template.cc:4:
deps/include/v8-function-callback.h: In instantiation of ‘v8::Isolate* v8::PropertyCallbackInfo<F>::GetIsolate() const [with T = v8::Value]’:
object_template.cc:16:33:   required from here
   16 |   Isolate* iso = info.GetIsolate();
      |                  ~~~~~~~~~~~~~~~^~
deps/include/v8-function-callback.h:695:11: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
  695 |   return *reinterpret_cast<Isolate**>(&args_[kIsolateIndex]);
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Running Suite: Browser Suite - /home/user/tmp/testrepo1099/browser
==================================================================
Random Seed: 1742399590

Will run 0 of 0 specs

Ran 0 of 0 Specs in 0.000 seconds
SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
ok  	github.com/gost-dom/browser	0.048s
go test -count 1  1.52s user 0.50s system 152% cpu 1.322 total

[onsi]

thanks @ianlancetaylor - i tried it but no dice:

code/browser » time go test -c -gcflags=-dwarf=false
go test -c -gcflags=-dwarf=false  2.69s user 1.12s system 135% cpu 2.804 total

Looking at builderTest it does look like the only build difference associated with -c (testC in the code) is to set OmitDebug which does twiddle the -dwarf flag.

And thanks @seankhliao - this well and truly might be some weird MacOS thing. That linux comparison is super helpful. I've noticed stuff like XProtect gets in the way of invoking binaries. I have that turned off in my terminal but something else may be going on as well.

Does go test not write a binary somewhere and then invoke it, though?

[gabyhelp]

Related Issues

• cmd/go: `go test` takes extra time to run even for packages that do not have any test files #17248 (closed)
• cmd/go: go run and go tool are slower than directly executing cached binary #71733
• cmd/go: -race adds significant overhead when running tests #61941 (closed)
• Benchmark results vastly different when Tests are skipped with -run=XXX #42696 (closed)
• testing: Benchmark, impossibly low ns/op #69417 (closed)
• go test is very slow on mac m1 #43370 (closed)
• cmd/go/internal/test: default pgo=auto results in surprising slow behaviour #61983
• cmd/go: build, generate, and test all take more than twice as long in go1.5 on darwin/amd64 #12211 (closed)
• testing: strange benchmark results with RunParallel and -benchtime #72079
• cmd/compile: build time increased in go 1.15 #40781 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[seankhliao]

On a mac, it's slower than linux, and much more variable. But executing the test is still slower.

$ time go test -c
go test -c  0.29s user 0.47s system 73% cpu 1.023 total

$ time go test -c
go test -c  0.29s user 0.50s system 170% cpu 0.465 total

$ time go test -c
go test -c  0.29s user 0.51s system 187% cpu 0.426 total

$ time go test -c
go test -c  0.26s user 0.44s system 249% cpu 0.280 total

$ time go test -c
go test -c  0.29s user 0.46s system 74% cpu 1.019 total

$ time go test -c
go test -c  0.29s user 0.48s system 166% cpu 0.467 total

$ time go test -c
go test -c  0.29s user 0.50s system 183% cpu 0.427 total

$ time go test -c
go test -c  0.27s user 0.48s system 272% cpu 0.276 total

$ time go test -c
go test -c  0.30s user 0.53s system 98% cpu 0.845 total

$ time go test -c
go test -c  0.29s user 0.51s system 170% cpu 0.471 total

$ time go test -c
go test -c  0.28s user 0.47s system 167% cpu 0.447 total

$ time go test -c
go test -c  0.27s user 0.45s system 254% cpu 0.284 total

$ time go test -c
go test -c  0.29s user 0.46s system 95% cpu 0.789 total

$ time go test -c
go test -c  0.29s user 0.52s system 170% cpu 0.480 total

$ time go test -count 1
Running Suite: Browser Suite - /Users/sean.liao/tmp/testrepo0077/browser
========================================================================
Random Seed: 1742400086

Will run 0 of 0 specs

Ran 0 of 0 Specs in 0.000 seconds
SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
ok  github.com/gost-dom/browser 0.822s
go test -count 1  1.06s user 0.71s system 84% cpu 2.087 total

$ time go test -count 1
Running Suite: Browser Suite - /Users/sean.liao/tmp/testrepo0077/browser
========================================================================
Random Seed: 1742400088

Will run 0 of 0 specs

Ran 0 of 0 Specs in 0.000 seconds
SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
ok  github.com/gost-dom/browser 0.714s
go test -count 1  1.11s user 0.73s system 121% cpu 1.521 total

$ time go test -count 1
Running Suite: Browser Suite - /Users/sean.liao/tmp/testrepo0077/browser
========================================================================
Random Seed: 1742400091

Will run 0 of 0 specs

Ran 0 of 0 Specs in 0.000 seconds
SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
ok  github.com/gost-dom/browser 0.678s
go test -count 1  1.13s user 0.73s system 90% cpu 2.061 total

[onsi]

@seankhliao you need to remove the binary between each invocation of go test -c. Otherwise something somewhere detects that the binary already exists and skips recompilation entirely:

code/browser » time go test -c
go test -c  0.30s user 0.90s system 539% cpu 0.222 total
code/browser » time go test -c
go test -c  0.29s user 0.89s system 519% cpu 0.226 total
code/browser » time go test -c
go test -c  0.30s user 0.94s system 547% cpu 0.226 total

code/browser » rm browser.test
code/browser » time go test -c && rm browser.test
go test -c  2.64s user 1.45s system 155% cpu 2.622 total
code/browser » time go test -c && rm browser.test
go test -c  2.66s user 1.29s system 169% cpu 2.325 total
code/browser » time go test -c && rm browser.test
go test -c  2.69s user 1.31s system 169% cpu 2.361 total
code/browser » time go test -c && rm browser.test
go test -c  2.68s user 1.34s system 167% cpu 2.400 total

note all those compile times are slower than go test (which, on my machine is also ~1.1s).

Do you see the same slow-down behavior on linux if you rm browser.test between runs?

[seankhliao]

Doesn't seem to be a discernable difference between the 2 runs:

$ time go test -c && rm browser.test
# github.com/gost-dom/v8go
In file included from deps/include/v8-isolate.h:22,
                 from deps/include/v8-snapshot.h:9,
                 from deps/include/v8-context.h:15,
                 from object_template.cc:4:
deps/include/v8-function-callback.h: In instantiation of ‘v8::Isolate* v8::PropertyCallbackInfo<F>::GetIsolate() const [with T = v8::Value]’:
object_template.cc:16:33:   required from here
   16 |   Isolate* iso = info.GetIsolate();
      |                  ~~~~~~~~~~~~~~~^~
deps/include/v8-function-callback.h:695:11: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
  695 |   return *reinterpret_cast<Isolate**>(&args_[kIsolateIndex]);
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
go test -c  1.50s user 0.51s system 157% cpu 1.271 total

$ time go test -c && rm browser.test
# github.com/gost-dom/v8go
In file included from deps/include/v8-isolate.h:22,
                 from deps/include/v8-snapshot.h:9,
                 from deps/include/v8-context.h:15,
                 from object_template.cc:4:
deps/include/v8-function-callback.h: In instantiation of ‘v8::Isolate* v8::PropertyCallbackInfo<F>::GetIsolate() const [with T = v8::Value]’:
object_template.cc:16:33:   required from here
   16 |   Isolate* iso = info.GetIsolate();
      |                  ~~~~~~~~~~~~~~~^~
deps/include/v8-function-callback.h:695:11: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
  695 |   return *reinterpret_cast<Isolate**>(&args_[kIsolateIndex]);
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
go test -c  1.50s user 0.49s system 157% cpu 1.268 total

$ time go test -c && rm browser.test
# github.com/gost-dom/v8go
In file included from deps/include/v8-isolate.h:22,
                 from deps/include/v8-snapshot.h:9,
                 from deps/include/v8-context.h:15,
                 from object_template.cc:4:
deps/include/v8-function-callback.h: In instantiation of ‘v8::Isolate* v8::PropertyCallbackInfo<F>::GetIsolate() const [with T = v8::Value]’:
object_template.cc:16:33:   required from here
   16 |   Isolate* iso = info.GetIsolate();
      |                  ~~~~~~~~~~~~~~~^~
deps/include/v8-function-callback.h:695:11: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
  695 |   return *reinterpret_cast<Isolate**>(&args_[kIsolateIndex]);
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
go test -c  1.46s user 0.49s system 155% cpu 1.254 total

$ time go test -count 1 && rm browser.test
# github.com/gost-dom/v8go
In file included from deps/include/v8-isolate.h:22,
                 from deps/include/v8-snapshot.h:9,
                 from deps/include/v8-context.h:15,
                 from object_template.cc:4:
deps/include/v8-function-callback.h: In instantiation of ‘v8::Isolate* v8::PropertyCallbackInfo<F>::GetIsolate() const [with T = v8::Value]’:
object_template.cc:16:33:   required from here
   16 |   Isolate* iso = info.GetIsolate();
      |                  ~~~~~~~~~~~~~~~^~
deps/include/v8-function-callback.h:695:11: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
  695 |   return *reinterpret_cast<Isolate**>(&args_[kIsolateIndex]);
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Running Suite: Browser Suite - /home/user/tmp/testrepo1099/browser
==================================================================
Random Seed: 1742401458

Will run 0 of 0 specs

Ran 0 of 0 Specs in 0.000 seconds
SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
ok  	github.com/gost-dom/browser	0.046s
go test -count 1  1.50s user 0.49s system 155% cpu 1.283 total
rm: cannot remove 'browser.test': No such file or directory

$ time go test -count 1 && rm browser.test
# github.com/gost-dom/v8go
In file included from deps/include/v8-isolate.h:22,
                 from deps/include/v8-snapshot.h:9,
                 from deps/include/v8-context.h:15,
                 from object_template.cc:4:
deps/include/v8-function-callback.h: In instantiation of ‘v8::Isolate* v8::PropertyCallbackInfo<F>::GetIsolate() const [with T = v8::Value]’:
object_template.cc:16:33:   required from here
   16 |   Isolate* iso = info.GetIsolate();
      |                  ~~~~~~~~~~~~~~~^~
deps/include/v8-function-callback.h:695:11: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
  695 |   return *reinterpret_cast<Isolate**>(&args_[kIsolateIndex]);
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Running Suite: Browser Suite - /home/user/tmp/testrepo1099/browser
==================================================================
Random Seed: 1742401460

Will run 0 of 0 specs

Ran 0 of 0 Specs in 0.000 seconds
SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
ok  	github.com/gost-dom/browser	0.043s
go test -count 1  1.56s user 0.47s system 158% cpu 1.280 total
rm: cannot remove 'browser.test': No such file or directory

$ time go test -count 1 && rm browser.test
# github.com/gost-dom/v8go
In file included from deps/include/v8-isolate.h:22,
                 from deps/include/v8-snapshot.h:9,
                 from deps/include/v8-context.h:15,
                 from object_template.cc:4:
deps/include/v8-function-callback.h: In instantiation of ‘v8::Isolate* v8::PropertyCallbackInfo<F>::GetIsolate() const [with T = v8::Value]’:
object_template.cc:16:33:   required from here
   16 |   Isolate* iso = info.GetIsolate();
      |                  ~~~~~~~~~~~~~~~^~
deps/include/v8-function-callback.h:695:11: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
  695 |   return *reinterpret_cast<Isolate**>(&args_[kIsolateIndex]);
      |           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Running Suite: Browser Suite - /home/user/tmp/testrepo1099/browser
==================================================================
Random Seed: 1742401463

Will run 0 of 0 specs

Ran 0 of 0 Specs in 0.000 seconds
SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS
ok  	github.com/gost-dom/browser	0.052s
go test -count 1  1.58s user 0.49s system 157% cpu 1.317 total
rm: cannot remove 'browser.test': No such file or directory

[onsi]

ok thanks - so that appears to confirm that this is a macos-specific issue... potentially a go-tool-chain-on-mac issue. but (perhaps more likely) it could be just-a-mac issue.

[onsi]

if anyone else has any thoughts/pointers/ideas/seen-this-before i'd appreciate the thoughts/help.

[randall77]

Looks like either -ldfags=-s or -ldflags=-w reduces the time for the go test -c build to below that of go test. Using -s gives a bit more improvement.
(Both of those flags are given to the linker during go test.)

[onsi]

brilliant! that was it - thanks @randall77, and everyone else who chimed in - you've helped me resolve a long-running performance gap in Ginkgo. ❤️