fix(runner): wire --timeout, default output dir, reliable artifacts#1377
Merged
Conversation
--timeout set the Playwright context default action timeout but not the @qawolf/flows expect-wrapper timeout, which stayed pinned at its hardcoded 30s default. Assertion failures (the common case) ignored the flag. Thread the timeout through initFlowRuntime into configureFlowRuntime as webExpectAttributes.defaultExpectTimeoutMs, and reword the flag help to reflect its per-action/assertion semantics. Non-web flows remain unwired (WIZ-10830). Fixes WIZ-10821
The readback test observes a process-global on @qawolf/flows and initFlowRuntime memoizes per flow directory. Sibling runner tests share that directory, so a prior cache entry could skip the reconfigure and leave a stale timeout — failing only under cross-file test ordering in CI. Reset the cache before the assertion to force a fresh configure.
Clears the @qawolf/flows expect-timeout config the readback test sets, so it cannot leak into later tests in this file (CodeRabbit nitpick).
Close contexts fully before browsers in createLaunch cleanup. Playwright flushes HAR/video/trace during context.close(), and the previous concurrent browser.close() could terminate the connection mid-flush, silently dropping artifacts (reproduced as flaky HAR loss). Wire trace recording through the runner: the parsed --trace flag was dropped in buildRunOptions, omitted from RunWebFlowOptions, and tracing.start/stop was never called. Add traceFlowPath/initTrace/ maybeCleanupTrace symmetric to HAR, start tracing after context creation, and stop to the trace path during cleanup. Respect on/off/retain-on-failure. Fixes WIZ-10823
Correct the cleanup comment: context.close() flushes HAR/video, while the trace zip is written by the preceding tracing.stop(). Note that a SIGINT-interrupted run preserves HAR/video but not the trace, since the signal handlers only close the context. Add a TODO WIZ-10839 for the multi-launch last-wins artifact path: multiple launch() calls in one flow stop every context to the same trace path, so only one zip survives (mirrors existing HAR behavior). Surfaced during adversarial review of WIZ-10823.
createLaunch registered context.close() and browser.close() as separate signal cleanups, but SignalRegistry runs cleanups concurrently — so a SIGINT could race browser.close() against the context's HAR/video flush, the same way the old concurrent cleanup did. Register a single ordered teardown (stop tracing, close contexts, then browsers) used by both normal cleanup and signal shutdown, guarded against double-run. Surfaced by CodeRabbit on PR #178.
When a passing flow's HAR/trace file cannot be deleted under retain-on-failure, the error was swallowed silently, leaving a stale artifact the user never heard about. Log a warning with the path via the runner logger so it can be removed manually. Cleanup failure still never fails a passing flow. Addresses review feedback from @michael-pr on PR #178.
|
Note Reviews pausedUse the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThree coordinated patches are applied to Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
Contributor
Author
|
CodeRabbit (@coderabbitai) review |
✅ Action performedReview finished.
|
Contributor
Author
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/domains/runner/runWebFlowUtils.ts`:
- Around line 59-69: The `closeAll` function uses a `closed` boolean flag to
prevent re-entry but does not expose the in-flight teardown promise, causing
concurrent callers to return early without waiting for cleanup to complete. To
fix this, declare a persistent variable outside the function to store the
pending teardown promise, then modify `closeAll` to check if a teardown is
already in progress and return that existing promise instead of returning
immediately. When initiating a new teardown, assign the entire promise chain
containing the tracing stop and context/browser closures to that variable before
awaiting it, ensuring all concurrent callers properly wait for the same teardown
operation to complete.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: eca08d45-8f16-47b3-97b6-bb1b7f390207
⛔ Files ignored due to path filters (1)
src/commands/__snapshots__/help.test.ts.snapis excluded by!**/*.snap
📒 Files selected for processing (26)
.changeset/gitignored-default-output-dir.md.changeset/har-trace-artifacts.md.changeset/timeout-web-flow-expect.mdsrc/commands/flows/run.register.tssrc/core/messages/runner.tssrc/core/paths.test.tssrc/core/paths.tssrc/domains/config/loadConfig.test.tssrc/domains/config/schema.tssrc/domains/init/templates.tssrc/domains/runner/dispatchViaSubprocess.test.tssrc/domains/runner/executeWorkerFlow.test.tssrc/domains/runner/initFlowRuntime.test.tssrc/domains/runner/initFlowRuntime.tssrc/domains/runner/run.manifestStamp.test.tssrc/domains/runner/runHelpers.test.tssrc/domains/runner/runHelpers.tssrc/domains/runner/runWebFlow.fixtures.tssrc/domains/runner/runWebFlow.test.tssrc/domains/runner/runWebFlow.trace.test.tssrc/domains/runner/runWebFlow.tssrc/domains/runner/runWebFlowUtils.test.tssrc/domains/runner/runWebFlowUtils.tssrc/domains/runner/web/contextSetup.test.tssrc/domains/runner/web/contextSetup.trace.test.tssrc/domains/runner/web/contextSetup.ts
✅ Action performedReviews paused. |
Michael Price (michael-pr)
force-pushed
the
replay-fixes-into-cli
branch
from
ee03192 to
70ac1e3
Compare
Mateus Zitelli (MateusZitelli)
approved these changes
Jun 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relates to cli-private #175, #176, #177, #178
Overview of Changes
Replays four reviewed and merged internal PRs (#175–#178) into the public
@qawolf/clirepository, preserving smonn's (Simon Ingeson) original authorship across all 12 commits. The changes close three gaps:--timeoutwas silently dropped before reaching the Playwrightexpectcall inside web flows; the default run output directory was not gitignored, risking accidental artifact commits; and HAR/trace artifacts were lost on early exits because browsers were closed before their contexts, with flush order also not preserved during signal shutdown.Testing
All three fixes ship with colocated tests added in the same commits (
runWebFlow.trace.test.ts,contextSetup.trace.test.ts,paths.test.ts, and updates torunWebFlow.test.ts). These commits are a verbatim replay of PRs already tested and merged in the internal repository — no additional manual verification steps are required beyond CI.Checklist