Add Claude-driven manual test workflow for PRs - #1160
Open
sugh01 wants to merge 1 commit into
Open
Conversation
Runs an exploratory manual test on every PR: boots the local ENS stack, drives the real app in a browser via the Playwright MCP server, and posts a report as a PR comment. An MCP-driven browser has none of the Playwright fixtures the e2e suite relies on, so init-page.cjs restores the two that matter via MCP's --init-page hook: it injects the same mock wallet the e2e suite uses (@ensdomains/headless-web3-provider, every request kind pre-authorized so nothing needs manual approval) and installs the page clock at the anvil block timestamp, since local chain time is not real time and every name would otherwise look expired. It also exposes a control server on :8546 for switching accounts and travelling in time from the shell. Supporting pieces: - wait-for-stack.mjs gates on contracts actually having code on chain and the subgraph reaching the chain head. A leftover .env.local is not proof the deploy finished. - smoke.mjs boots a real MCP server, loads the app and asserts the wallet, clock and control server work, before any Claude usage is spent. It spawns from the same MCP config the session gets, so the two cannot drift. - pnpm seed seeds fixtures through the existing makeName helper, without launching a browser. - The run book always exercises register and extend, so a PR with no user-facing change still gets meaningful coverage. Long-running processes are started with setsid: each step is its own shell session, and a merely backgrounded child is torn down with it.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| - name: Run manual test session | ||
| id: claude | ||
| if: ${{ !env.ACT || env.CLAUDE_IN_ACT == 'true' }} | ||
| uses: anthropics/claude-code-action@v1 |
|
|
||
| - name: Install Playwright browsers | ||
| run: | | ||
| pnpm exec playwright install --with-deps chromium |
| run: | | ||
| pnpm exec playwright install --with-deps chromium | ||
| # the MCP server ships its own playwright, which needs its own browser build | ||
| npx -y @playwright/mcp@latest install-browser chrome-for-testing |
Deploying ens-app-v3 with
|
| Latest commit: |
6d87a2b
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://fa437ab2.ens-app-v3.pages.dev |
| Branch Preview URL: | https://feat-claude-manual-test-work.ens-app-v3.pages.dev |
Claude manual test report — PR #11601. Verdict✅ looks good — Phase 0 (register, extend) both PASS, and the PR's own surface (the manual-test 2. What I understood the PR to do
3. Test planSince the diff is 100% harness/CI code with no
4. Results table
5. Issues foundNone — no functional defects in the harness or the app. Two pieces of console noise observed, both pre-existing/unrelated to this diff (verified no
6. Not covered
Screenshots (workflow artifacts)
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Runs an exploratory manual test on every PR: boots the local ENS stack, drives the real app in a browser via the Playwright MCP server, and posts a report as a PR comment.
An MCP-driven browser has none of the Playwright fixtures the e2e suite relies on, so init-page.cjs restores the two that matter via MCP's --init-page hook: it injects the same mock wallet the e2e suite uses (@ensdomains/headless-web3-provider, every request kind pre-authorized so nothing needs manual approval) and installs the page clock at the anvil block timestamp, since local chain time is not real time and every name would otherwise look expired. It also exposes a control server on :8546 for switching accounts and travelling in time from the shell.
Supporting pieces:
Long-running processes are started with setsid: each step is its own shell session, and a merely backgrounded child is torn down with it.