Conversation
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
sozercan
requested review from
ashnamehrotra,
jeremyrickard,
omercnet and
robert-cronin
as code owners
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1639 +/- ##
==========================================
+ Coverage 44.23% 44.66% +0.42%
==========================================
Files 58 58
Lines 10312 10366 +54
==========================================
+ Hits 4562 4630 +68
+ Misses 5435 5415 -20
- Partials 315 321 +6
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR optimizes several non-BuildKit hot paths that were not covered by #1638: VEX document generation/output, the copa generate patch-layer tar hardlink rewrite, progress forwarding across multiplexed builds, and TUI progress bookkeeping. It adds focused microbenchmarks alongside the optimizations, and the changes are largely behavior-preserving refactors that trade repeated work/allocations for indexed lookups and cached values.
Changes:
- VEX: split
CreateVEXDocumentinto an internalcreateVEXDocumentreturning*vex.VEX, stream JSON directly to the output file, and replace O(n) statement/subcomponent scans with indexed maps (statementByVulnerability,seenSubcomponents) plus a precomputed PURL qualifier suffix. - Generate: track actual hardlink targets so the second tar pass only buffers needed file contents, and sort hardlink names for deterministic build contexts.
- Progress/TUI: cache prefixed digest remaps in
ForwardProgressWithPrefix, and scope progress tasks per-vertex (vertexState) so completed vertices drop their tasks without prefix-scanning.
Show a summary per file
| File | Description |
|---|---|
| pkg/vex/vex.go | Streams VEX JSON directly to the output file instead of buffering the full string |
| pkg/vex/openvex.go | Indexed grouping/dedup and precomputed PURL prefix/qualifier suffix; adds createVEXDocument, vexSubcomponentKey, osPackageQualifiers |
| pkg/vex/openvex_benchmark_test.go | New benchmarks for VEX creation and file output |
| pkg/tui/display.go | Per-vertex task map (vertexState) replaces global prefix-keyed task cleanup |
| pkg/tui/display_benchmark_test.go | New benchmark for progress task bookkeeping |
| pkg/generate/generate.go | Caches only real hardlink targets and writes hardlinks in sorted order |
| pkg/generate/generate_benchmark_test.go | New benchmark for sparse-hardlink tar rewrite |
| pkg/common/progress.go | Caches digest remaps; removes redundant slice pre-assignment |
| pkg/common/progress_test.go | New unit tests for remap correctness, non-mutation, nil-skip, cancellation |
| pkg/common/progress_benchmark_test.go | New benchmark for repeated-digest forwarding |
Review details
- Files reviewed: 10/10 changed files
- Comments generated: 1
- Review effort level: Medium
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a1633a8c61
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Signed-off-by: Sertac Ozercan <sozercan@gmail.com> # Conflicts: # test/e2e/nodejs/nodejs_test.go
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 005328b032
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 59fd46619f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Summary
Impact
Representative benchmark improvements from this pass:
Live end-to-end
copa generatecomparison on a warmed Alpine report flow showed a modest improvement because BuildKit/package-manager work dominates that scenario:This intentionally avoids the performance areas already covered by #1638.
Validation
go run mvdan.cc/gofumpt@v0.9.2 -w ...git diff --checkgo test ./pkg/... -count=1make buildgo test ./test/e2e/generate -run '^TestGenerateWithVEXOutput$' -count=1 -timeout 30m -args -copa $(pwd)/dist/darwin_arm64/release/copa -addr docker://pkg/vex,pkg/generate,pkg/common, andpkg/tuigolangci-lint run --new-from-rev=HEAD ./pkg/vex ./pkg/generate ./pkg/common ./pkg/tui→0 issuespython3 /Users/sozercan/.codex/skills/autoreview/scripts/autoreview --mode localautoreview clean: no accepted/actionable findings reportedoverall: patch is correct (0.86)Notes
make lintwas run and still reports pre-existing repository-wide lint issues unrelated to this change set (goconst, existinggosec,govet,gocritic, andstaticcheck). New lint findings from this change were fixed./tmp/perf3-copa.md.