Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

process: migrate CI from Kokoro to LUCI #3533

Open
hyangah opened this issue Sep 11, 2024 · 21 comments
Open

process: migrate CI from Kokoro to LUCI #3533

hyangah opened this issue Sep 11, 2024 · 21 comments
Assignees
@h9jiang
Milestone
vscode-go/backlog

Comments

@hyangah
Copy link
Contributor

hyangah commented Sep 11, 2024

Move our primary CI to LUCI, the default CI & trybot infra in the Go project.
That will help us align our test workflow with other Go projects, simplify integration with Gerrit & Relui.

  • In addition to the go toolchain and git, vscode-go project requires extra tools in the builder image.
    • Node.js (npm)
    • xvfb (xvfb-run) for testing in linux
    • access to the network

Other tools can be built with go install and npm ci+npx on demand.

  • We will also need to figure out how to present the vscode-go integration test output.

  • Nice to have:
    Windows/Mac support
    Trybot

cc @golang/release @golang/tools-team
@gopherbot gopherbot added this to the Untriaged milestone Sep 11, 2024
@dmitshur dmitshur mentioned this issue Sep 11, 2024
Open
@dmitshur
Copy link

Adding some relevant notes here.

CIPD as used by LUCI builders already has an existing package providing Node.js, and it's used for testing the js/wasm port: https://chrome-infra-packages.appspot.com/p/infra/3pp/tools/nodejs.

For something like xvfb, it might work better to add it to the image (e.g., see here).

*-longtest builders have internet access by default (note that go test -short isn't expected to need internet, so tests that require internet are expected to skip when testing.Short() is true).

@findleyr findleyr modified the milestones: Untriaged, vscode-go/backlog Sep 26, 2024
@findleyr
Copy link
Member

@hyangah moved this to the backlog, since this doesn't seem naturally associated with any release. Please remilestone as appropriate.

@h9jiang h9jiang self-assigned this Jan 10, 2025
@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/641439 mentions this issue: main.star: introduce vscode-go to LUCI

@dmitshur dmitshur moved this to In Progress in Go Release Jan 10, 2025
gopherbot pushed a commit to golang/build that referenced this issue Jan 10, 2025
@h9jiang @gopherbot
main.star: introduce vscode-go to luci projects
a4d3005 
vscode-go test should only run against machines that
have docker installed matching the following dimentions:
"cipd_platform": "linux-amd64",
"machine_type": "n1-standard-16",
"os": "Ubuntu-20",
"docker_installed": "true",

For golang/vscode-go#3533

Change-Id: If49d5b55be8ea5dc59e5057c10f561d24d795ab3
Reviewed-on: https://go-review.googlesource.com/c/build/+/641439
Reviewed-by: Michael Knyszek <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Auto-Submit: Hongxiang Jiang <[email protected]>
@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/642415 mentions this issue: main.star: enable postsubmit for vscode-go

gopherbot pushed a commit to golang/build that referenced this issue Jan 13, 2025
@h9jiang
main.star: enable postsubmit for vscode-go
5110f3d 
Replace enabled type with "linux-amd64" to match pattern
os-arch.

For golang/vscode-go#3533

Change-Id: I83d708baec97e7384513a1a2254d1be31f667b60
Reviewed-on: https://go-review.googlesource.com/c/build/+/642415
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Michael Knyszek <[email protected]>
@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/642424 mentions this issue: main.star: explicitly disable presubmit and enable postsubmit

@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/642558 mentions this issue: main.star: disable CQ Group for vscode-go

gopherbot pushed a commit to golang/build that referenced this issue Jan 14, 2025
@h9jiang @gopherbot
main.star: disable CQ Group for vscode-go
246ae41 
Defining CQ Group (a set of refs to watch) for vscode-go
repo means LUCI will watch for vscode-go and triggers
LUCI test. LUCI finishes the test immedidately and reset
CQ tile to 0 preventing Kokoro from running.

For golang/vscode-go#3533

Change-Id: I571a8cb63010551edc178d79f2e9d7611a690109
Reviewed-on: https://go-review.googlesource.com/c/build/+/642558
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Michael Knyszek <[email protected]>
Auto-Submit: Hongxiang Jiang <[email protected]>
@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/642423 mentions this issue: docs/wiki.go: replace info.txt with wiki.go

gopherbot pushed a commit that referenced this issue Jan 14, 2025
@h9jiang
docs/wiki.go: replace info.txt with wiki.go
3fec532 
LUCI trigger go test ./... under every go module, a module
without a go file result in an error from go test.

For #3533

Change-Id: I6973e9c5cc904ff4a3ac4ce97a7bbc3d2b6e4eb0
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/642423
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
kokoro-CI: kokoro <[email protected]>
Commit-Queue: Hongxiang Jiang <[email protected]>
@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/642479 mentions this issue: extension/tools/release: skip validation from zip file

@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/642917 mentions this issue: build: migrate script trigger test to go trigger test

@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/642097 mentions this issue: DNR build: migrate all.bash ci script test with go test

@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/643855 mentions this issue: extension/tools/release: add env TEST_ALL to control test skipping

gopherbot pushed a commit that referenced this issue Jan 23, 2025
@h9jiang @gopherbot
extension/tools/release: add env var to control test skipping
7df3953 
Some of the go test require extra tools. VSCODE_GO_TEST_ALL is
used to control whether the test should be skipped or errored
if the required tool is missing.

For LUCI test, test is triggered by "go test ./..." meaning test will
skip if the tool is missing.
For docker test, test is triggered by
"VSCODE_GO_TEST_ALL=true go test ./..." meaning the test can not be
skipped.

Choose "TEST_ALL" over "CAN_SKIP" to avoid double negative.

For #3533

Change-Id: I7fdad5e872a377882aa274d27d88ae6ddffb613e
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/643855
Reviewed-by: Robert Findley <[email protected]>
Commit-Queue: Hongxiang Jiang <[email protected]>
Auto-Submit: Hongxiang Jiang <[email protected]>
kokoro-CI: kokoro <[email protected]>
gopherbot pushed a commit that referenced this issue Jan 27, 2025
@h9jiang @gopherbot
build: migrate script trigger test to go test trigger
2e07aa3 
Kokoro triggers the test through:
- sudo chown -R 1000:1000
- build docker image with all required tools
- mount vscode-go repo to docker image
- run npm test and go test in docker

This go test is slightly different than Kokoro:
- build docker image with the vscode-go copied and owner
modified to 1000:1000.
- run npm test and go test in docker without mounting.
- cleanup image.

LUCI test run https://ci.chromium.org/b/8724574170882251953

For #3533

Change-Id: If31e13d9b121fb0cfa56c65ec3c801989b9f0ebe
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/642917
Auto-Submit: Hongxiang Jiang <[email protected]>
kokoro-CI: kokoro <[email protected]>
Commit-Queue: Hongxiang Jiang <[email protected]>
Reviewed-by: Robert Findley <[email protected]>
gopherbot pushed a commit that referenced this issue Jan 27, 2025
@h9jiang @gopherbot
build: migrate "./all.bash ci" script test to go test
8b6e6f8 
"./build/all.bash ci" trigges go test, go generate, npm
build, npm lint and npm test.

Remove npm ansi coding through npm config set color false

Because logs are printed as text without any rendering in
LUCI, coloring is disabled for npm and mocha during
integration test.

LUCI test run https://ci.chromium.org/b/8724574189483344737

For #3533

Change-Id: I8cb3340db48ed91cfb06165dd227241ea5f5b0a4
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/642097
Reviewed-by: Robert Findley <[email protected]>
kokoro-CI: kokoro <[email protected]>
Commit-Queue: Hongxiang Jiang <[email protected]>
Auto-Submit: Hongxiang Jiang <[email protected]>
gopherbot pushed a commit that referenced this issue Jan 27, 2025
@h9jiang
extension/tools/release: skip validation for dir in zip
771091a 
TestBuildVSCGO failure causing LUCI post submit failure.

For #3533

Change-Id: I11fe1154293ff91b7e06897f7f572f24a9ccc5eb
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/642479
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
kokoro-CI: kokoro <[email protected]>
Commit-Queue: Hongxiang Jiang <[email protected]>
@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/644815 mentions this issue: main.star: enable presubmit for vscode-go

gopherbot pushed a commit to golang/build that referenced this issue Jan 28, 2025
@h9jiang
main.star: enable presubmit for vscode-go
0f04c5b 
All the test is now passing in LUCI dashboard.
https://ci.chromium.org/ui/p/golang/builders/ci/x_vscode-go-gotip-linux-amd64_docker
https://ci.chromium.org/ui/p/golang/builders/ci/x_vscode-go-go1.23-linux-amd64_docker
https://ci.chromium.org/ui/p/golang/builders/ci/x_vscode-go-go1.22-linux-amd64_docker

The go1.22 is skipped because vscode-go require 1.23.

For golang/vscode-go#3533

Change-Id: Ia9c9f10bfc440153ea8c70dbefb2f4adce4cc71f
Reviewed-on: https://go-review.googlesource.com/c/build/+/644815
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Michael Knyszek <[email protected]>
@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/645355 mentions this issue: main.star: run vscode-go presubmit against all go versions

gopherbot pushed a commit to golang/build that referenced this issue Jan 29, 2025
@h9jiang @gopherbot
main.star: run vscode-go presubmit against all go versions
7cf65fd 
Right now, only gotip presubmit is triggered in vscode-go presubmit.

For golang/vscode-go#3533

Change-Id: I91baea2d3fd978f3c46eeeb972279fc8b6af0bd4
Reviewed-on: https://go-review.googlesource.com/c/build/+/645355
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Michael Knyszek <[email protected]>
Auto-Submit: Hongxiang Jiang <[email protected]>
@h9jiang
Copy link
Member

h9jiang commented Jan 29, 2025

In CL 645116, I saw vscode-go test is triggered against go 1.22 1.23 1.24 and gotip.

The migration is done. I will mark this as closed.

@h9jiang h9jiang closed this as completed Jan 29, 2025
@github-project-automation github-project-automation bot moved this from In Progress to Done in Go Release Jan 29, 2025
@findleyr
Copy link
Member

Great!

Just so I understand, there are quite a few tests in the associated build that are skipped. Is that intended?
https://ci.chromium.org/ui/p/golang/builders/try/x_vscode-go-go1.22-linux-amd64_docker/b8724368980837991217/overview

@findleyr
Copy link
Member

@h9jiang can you please also update docs/contributing.md in the vscode-go repo to remove references to Kokoro, and describe LUCI?

@h9jiang
Copy link
Member

h9jiang commented Jan 30, 2025

Is that intended?

Yes. I believe so. Because vscode-go set go 1.23 in go.mod file so test with version that is lower than 1.23 is skipped. (Otherwise, it will use toochain to auto-upgrade which is the same test as 1.23.)

can you please also update docs/contributing.md in the vscode-go repo to remove references to Kokoro, and describe LUCI?

Thanks for the reminder, let me re-open this issue and draft the CL for it.

@h9jiang h9jiang reopened this Jan 30, 2025
@h9jiang
Copy link
Member

h9jiang commented Jan 30, 2025

Just realized there are some extra work needed in vscode-go refs/meta/config branch to add LUCI Result +1 as submission requirement.

I will create another CL for this. Sorry for closing this issue early.

@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/645635 mentions this issue: project.config: remove overwriting TryBots-Pass

@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/646157 mentions this issue: docs: replace kokoro with LUCI in contribution guide

gopherbot pushed a commit that referenced this issue Feb 3, 2025
@h9jiang
docs: replace kokoro with LUCI in contribution guide
831ca25 
Add Kokoro to LUCI migration in CHANGELOG.

For #3533

Change-Id: Ib121f3e9d8b9cb8e79016a5848596fc49d883545
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/646157
LUCI-TryBot-Result: Go LUCI <[email protected]>
kokoro-CI: kokoro <[email protected]>
Reviewed-by: Robert Findley <[email protected]>
@h9jiang h9jiang mentioned this issue Feb 4, 2025
Open
@dmitshur dmitshur moved this from Done to In Progress in Go Release Feb 12, 2025
@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/649635 mentions this issue: build: surface error when noticing test failure

gopherbot pushed a commit that referenced this issue Feb 14, 2025
@h9jiang @gopherbot
build: surface error when noticing test failure
726029b 
'set -e' means exit immediately on error.

Previously, the presubmit from integration test may experience
false positive because the exit code from npm test is not being
captured.

This false positive is noticed in CL 649258 where kokoro test
failed but LUCI test passed.

This CL should fail due to go1.24 vet behavior change but should
pass if rebased CL 649475.

For #3533

Change-Id: Iee7d5d02c178b8655a81b088f914d65237f79834
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/649635
Auto-Submit: Hongxiang Jiang <[email protected]>
Reviewed-by: Robert Findley <[email protected]>
kokoro-CI: kokoro <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@h9jiang h9jiang

Labels
None yet
Projects
Go Release
Status: In Progress
Milestone
vscode-go/backlog
Development

No branches or pull requests
5 participants
@dmitshur @hyangah @gopherbot @findleyr @h9jiang