Skip to content

Address Copilot review comments; add native uv build backend#18

Merged
statik merged 4 commits intoclaude/mcp-evals-framework-BNUuzfrom
copilot/address-copilot-review-comments
Apr 9, 2026
Merged

Address Copilot review comments; add native uv build backend#18
statik merged 4 commits intoclaude/mcp-evals-framework-BNUuzfrom
copilot/address-copilot-review-comments

Conversation

Copy link
Copy Markdown

Copilot AI commented Mar 23, 2026

Nine Copilot review comments on the three-layer eval framework (PR #17) — covering missing build backends, incorrect CI conditions, connection leaks, and missing graceful-skip logic.

Build system

Both evals/inspect/pyproject.toml and evals/connect/pyproject.toml lacked a [build-system] table, causing pip install -e . to fail in CI. Added uv's native build backend:

[build-system]
requires = ["uv_build>=0.6.0,<0.11.0"]
build-backend = "uv_build"

uv_build uses a src layout and requires a Python package module, so minimal src/mcp_evals/__init__.py and src/connect_deploy_tests/__init__.py stubs were added to satisfy that requirement.

mcp_eval_tasks.py

  • list_indicators_quality and adoption_agent_eval: both now return a trivial no-op/skip Task when ANTHROPIC_API_KEY is absent, matching the stated "skip gracefully" design.
  • --cache-dir: subprocess now passes --cache-dir <tmpdir> on the CLI rather than MCP_CACHE_DIR (which the Go binary does not read), isolating cache state per eval run.

conftest.py

http_client fixture changed from return to yield with try/finally → client.close() to avoid leaking connections across the test session.

Workflow conditions

  • pr-connect-deploy.yml: job if: now also checks secrets.CONNECT_LICENSE != '' so the job actually skips when the secret is absent.
  • pr-evals.yml: both LLM step conditions changed from env.ANTHROPIC_API_KEY != '' (always false — env vars aren't set at evaluation time) to secrets.ANTHROPIC_API_KEY != ''.
Original prompt

read the copilot review comments and address them. For the python build backend I want to use uv.


⚡ Quickly spin up Copilot coding agent tasks from anywhere on your macOS or Windows machine with Raycast.

Copilot AI changed the title [WIP] Address Copilot review comments and set Python build backend to uv Address Copilot review comments; add uv/hatchling build backend Mar 23, 2026
Copilot AI requested a review from statik March 23, 2026 17:11
@statik
Copy link
Copy Markdown
Collaborator

statik commented Mar 23, 2026

@copilot i don't want to use hatchling, i want to use the native uv build backend

Copy link
Copy Markdown
Author

Copilot AI commented Mar 23, 2026

@copilot i don't want to use hatchling, i want to use the native uv build backend

Done in 790e2e3. Replaced hatchling with uv_build (uv's native build backend):

[build-system]
requires = ["uv_build>=0.6.0,<0.11.0"]
build-backend = "uv_build"

uv_build requires a src-layout Python package, so I also added minimal src/mcp_evals/__init__.py and src/connect_deploy_tests/__init__.py stubs. Both pip install -e . calls succeed with the new backend.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • docs.astral.sh
    • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@statik statik changed the title Address Copilot review comments; add uv/hatchling build backend fix: copilot review comments; add uv build backend Mar 23, 2026
@statik statik marked this pull request as ready for review March 23, 2026 17:22
Copilot AI changed the title fix: copilot review comments; add uv build backend Address Copilot review comments; add native uv build backend Mar 23, 2026
@statik
Copy link
Copy Markdown
Collaborator

statik commented Mar 31, 2026

@copilot fix the Go tests failure

Copy link
Copy Markdown
Author

Copilot AI commented Mar 31, 2026

@copilot fix the Go tests failure

Fixed in e5ab385. makeAdoptionSummaryHandler was returning a successful (empty) result when no cached data existed for the requested org. Added a check after iterating the cache — if totalRepos == 0, the handler now returns a tool error ("no cached scan data for org %s — run scan_org first"), matching the pattern used by makeRepoSummaryHandler. All Go tests pass locally.

@statik statik merged commit 174f497 into claude/mcp-evals-framework-BNUuz Apr 9, 2026
2 of 3 checks passed
@statik statik deleted the copilot/address-copilot-review-comments branch April 9, 2026 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants