ci: Windows build-once + Linux PR worker isolation#13615
Draft
midleman wants to merge 15 commits into
Draft
Conversation
|
E2E Tests 🚀 |
midleman
changed the title
midleman
force-pushed
the
mi/cache-e2e-test-deps
branch
from
6fb1617 to
2d4bf4d
Compare
midleman
changed the title
…EMFILE errors Retries EMFILE (too many open files) errors during gulp vscode-win32-x64 packaging. Same fix VS Code uses in their build tooling. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fromLocalWebpack was constructing File objects for all extension files at once and passing them via es.readArray(), which opened thousands of ReadStreams simultaneously. On Windows, this hit the OS EMFILE (too many open files) limit when packaging positron-python's 11,784 python_files. Switch to createSequentialFileStream (already used by fromLocalNormal) which opens files one at a time via a pump queue, matching the pattern that fromLocalNormal already relied on. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The EMFILE fix in build/lib/extensions.ts (sequential file opening via createSequentialFileStream) validated successfully on Windows CI. The spike workflow and the original graceful-fs patch are no longer needed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add test-e2e-windows-build.yml that compiles Positron once and uploads the full workspace (node_modules + compiled output) as a tar artifact. Strip the build steps from test-e2e-windows-run.yml so each shard just downloads and extracts that artifact instead of compiling independently. Update test-merge.yml (3 shards), test-full-suite.yml (4 shards), and test-pull-request.yml to call the build workflow first, then pass the artifact to the run shards. Also fix a pre-existing bug: test-merge.yml's slack-notify-win-extensions was checking needs.e2e-windows-electron.outputs.extensions_failed but the job is named e2e-windows. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each Playwright worker now gets its own Xvfb display (:10 + parallelIndex) so workers don't race on a shared X server. The setup-xvfb action accepts a workers input and starts that many displays at :10..:(10+N-1). Linux PR e2e-electron workers bumped from 2 to 3, expected to drop the job from ~31m to ~25m on cold cache / ~19m on warm cache. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 workers saved ~2m off Linux PR e2e. Trying 4 to see if there's another 1-2m available before scaling hits a ceiling. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The spike confirmed zstd is available on windows-latest-8x runners (191ms compress vs gzip's 1228ms, 320ms decompress vs 416ms). The production change landed in test-e2e-windows-build.yml; this validator is no longer needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
midleman
force-pushed
the
mi/cache-e2e-test-deps
branch
from
2d4bf4d to
a1099cb
Compare
midleman
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CI speed-ups for Windows e2e and Linux PR shards.
fromLocalWebpacktest/e2enode_modules per shard:10+ parallelIndex), workers 2 to 4QA Notes
Validated on this branch at workers=4. Cache-pressure fixes split out to #13669.