test(council): dogfood + harden triage/pre-fanout/eval/attachment gates - #8
Merged
Conversation
Pin task-aware attachment resolution for representative frontend, fleet, generic edit, diagnostic, tie-break, over-cap, negative-trigger, and unknown-kind fallback profiles using the existing resolver and routing-card model.
Add table-driven triage dogfood coverage for realistic direct, parallel-program, and tightly-coupled sequential work profiles, and fix any classifier or gate-payload misclassification found.
Ensure task JSON validation enforces the resolved attachment contract that downstream workers rely on: non-empty MCP profile string and activeSkills bounded to the resolver cap.
Prove the pre-fanout static gate blocks real fanout/fleet execution paths, including the P0 same-wave overlap hazard, and surface actionable violation details through CLI errors instead of only a collapsed message.
Run the eval workflow and CLI against a realistic filesystem run directory containing events.jsonl, worker results, task metadata, verify outcomes, no-op, out-of-bounds, retry, and lucky-pass cases; close scoring gaps exposed by that fixture.
Extend Python/render/doctor validation so the merged triage/eval CLI surface and runtime attachment profile contract cannot drift silently from checked-in sources.
Close the WS-3 runtime seam by allowing supervise inputs and CLI parsing to pass a resolved MCP profile into WorkerSupervisorStartRequest so AGENT_MCP_PROFILE reaches worker processes.
Add composition-level coverage that projects app-produced worker lifecycle events into append-only traces and feeds those traces through the one-attempt repair-loop decision path, including terminal failure after the bounded repair is consumed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dogfoods and hardens the WS-3/WS-4 gates that landed in #6, via an 8-task all-codex fanout (100% line/stmt/func coverage, 456 tests).
council eval --runscored against realistic run-dir fixtures (retries, out-of-bounds edits, no-op, lucky-pass).supervise.All gates green: tsc, eslint (0 warn), dependency-cruiser, vitest 100% coverage, render --check, manifest validate, self-test.