Add schema-driven envelope validator with --strict mode

- schemas/qa-output.schema.json: source of truth for the machine block
  every agent emits (envelope + per-agent $defs.agent_extensions).
- scripts/validate-qa-output.mjs: envelope lint in default mode, per-agent
  extension lint under --strict. Zero deps, Node 18+. Derives constants
  (valid agents, verdicts, summary maxLength) from the schema so there's
  no hand-rolled duplicate to drift.
- scripts/validate-agents.mjs: refactored to the same {label, errors}
  reporting shape as the qa-output validator. Killed module-level state.
- tests/_invalid/: negative fixtures proving both modes reject what they
  should (envelope-violation fixture + strict-extension-only fixture).
- Golden fixtures (functional-reviewer, test-scenario-designer) now carry
  the machine block the validator checks against.
- AGENTS.md: new ยง0 mandatory skill-check preamble with the agent trigger
  map; line 4 cross-platform claim corrected to "Primary: Claude Code".
- .github/workflows/lint.yml: --all --strict, plus negative-fixture steps
  proving strict catches what envelope-only misses.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: Anass Rach

.github/workflows/lint.yml

@@ -21,3 +21,15 @@ jobs:
 
       - name: Validate agent structure
         run: node scripts/validate-agents.mjs
+
+      - name: Validate qa-output machine blocks (golden fixtures, strict)
+        run: node scripts/validate-qa-output.mjs --all --strict
+
+      - name: Validate envelope-violation fixture is rejected
+        run: node scripts/validate-qa-output.mjs --expect-fail tests/_invalid/expected-output.md
+
+      - name: Validate strict-extension fixture passes envelope-only
+        run: node scripts/validate-qa-output.mjs tests/_invalid/strict-extension.md
+
+      - name: Validate strict-extension fixture is rejected by --strict
+        run: node scripts/validate-qa-output.mjs --expect-fail --strict tests/_invalid/strict-extension.md

AGENTS.md

@@ -1,7 +1,26 @@
 # AGENTS.md — QA Orchestra Behavioral Instructions
 
 This file defines how AI agents must behave in this project.
-It applies to Claude Code, Copilot, Cursor, and any agent reading this workspace.
+
+**Primary support: Claude Code.** Best-effort: any agent runtime that honors AGENTS.md and reads Markdown instructions. Tool names in agent frontmatter (Agent, Bash, etc.) are Claude Code conventions; other runtimes may need a tool-mapping layer before the agents work correctly.
+
+---
+
+## 0. Mandatory skill check (read this first)
+
+Before responding to any QA-adjacent request, state which `qa-orchestra` agent applies. If one does, invoke it. If none does, say so explicitly and only then proceed with inline reasoning. Do not silently do QA-style work (gap analysis, scenario design, test selection, bug writing) without first naming the agent that owns that work — this is how specialist agents get forgotten and the discipline drifts.
+
+Quick map:
+- *"Does this diff implement the AC?"* → `functional-reviewer`
+- *"What scenarios do I need?"* → `test-scenario-designer`
+- *"Which tests will this change break?"* → `smart-test-selector`
+- *"File this as a bug."* → `bug-reporter`
+- *"Check the app in a browser."* → `browser-validator`
+- *"Set up the PR branch locally."* → `environment-manager`
+- *"Write Playwright/Cypress tests for these scenarios."* → `automation-writer`
+- *"Walk me through manual test execution."* → `manual-validator`
+- *"Analyze this release across repos."* → `release-analyzer`
+- *"Coordinate the full QA pipeline."* → `orchestrator`
 
 ---
 
@@ -30,7 +49,8 @@ If the user says the app is already running from the feature branch, skip the en
 ## 3. Output discipline
 
 - **Always save output to the correct file in `qa-output/`** (see CLAUDE.md agent map).
-- Output format is structured Markdown, copy-pasteable into GitHub Issues / Jira / Linear.
+- Output format is **a ```json qa-orchestra``` machine block at the top**, then structured Markdown below — see each agent's `## Output format` section for the required fields. Schema at `schemas/qa-output.schema.json`, validator at `scripts/validate-qa-output.mjs`.
+- The Markdown below the block stays copy-pasteable into GitHub Issues / Jira / Linear. The machine block and the prose must not contradict each other.
 - Never truncate output. If content is long, split into sections but deliver completely.
 - If you cannot produce a complete output (missing input), **stop and ask** — do not fabricate.
 - Screenshots from browser validation go to `qa-output/screenshots/`.
@@ -53,6 +73,8 @@ manual-validator    → reads test-scenarios.md
 
 Never redo upstream work. Trust the file.
 
+**When reading an upstream file, parse its ```json qa-orchestra``` block — not the prose.** The prose is for humans; LLM-on-LLM prose parsing is brittle. If the block is missing or fails to parse, stop and report rather than falling back to regex.
+
 ---
 
 ## 5. Parallelism

schemas/qa-output.schema.json

@@ -0,0 +1,196 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://qa-orchestra.com/schemas/qa-output.schema.json",
+  "title": "QA Orchestra machine block",
+  "description": "Schema for the ```json qa-orchestra fenced block that every agent must emit at the top of its qa-output/*.md report. Downstream agents read this block as source of truth; the prose below it is for humans. Kept deliberately shallow — strings for file references, not nested objects.",
+  "type": "object",
+  "required": ["agent", "version", "verdict", "summary"],
+  "additionalProperties": true,
+  "properties": {
+    "agent": {
+      "type": "string",
+      "enum": [
+        "orchestrator",
+        "environment-manager",
+        "functional-reviewer",
+        "test-scenario-designer",
+        "browser-validator",
+        "automation-writer",
+        "bug-reporter",
+        "manual-validator",
+        "release-analyzer",
+        "smart-test-selector"
+      ]
+    },
+    "version": { "const": 1 },
+    "timestamp": {
+      "type": "string",
+      "description": "ISO-8601 UTC timestamp. Optional — omit if the agent cannot observe wall-clock reliably."
+    },
+    "inputs": {
+      "type": "array",
+      "description": "Provenance — what the agent read. Each entry is a short {kind, ref} pair.",
+      "items": {
+        "type": "object",
+        "required": ["kind", "ref"],
+        "properties": {
+          "kind": { "type": "string" },
+          "ref": { "type": "string" }
+        }
+      }
+    },
+    "verdict": {
+      "type": "string",
+      "enum": ["pass", "pass_with_conditions", "fail", "not_applicable", "blocked"]
+    },
+    "summary": {
+      "type": "string",
+      "maxLength": 280,
+      "description": "One-line human-readable summary. Must be consistent with the prose verdict below."
+    },
+    "next_actions": {
+      "type": "array",
+      "items": { "type": "string" }
+    }
+  },
+  "$defs": {
+    "agent_extensions": {
+      "description": "Per-agent fields. Each agent SHOULD include its own extension, but unknown fields are allowed (additionalProperties: true) so agents can evolve without breaking the validator.",
+      "functional-reviewer": {
+        "risk_score": { "type": "integer", "minimum": 1, "maximum": 10 },
+        "ac_compliance": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": ["id", "status"],
+            "properties": {
+              "id": { "type": "string" },
+              "status": { "enum": ["covered", "partial", "missing", "ambiguous"] },
+              "file_ref": { "type": ["string", "null"] }
+            }
+          }
+        },
+        "gaps": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "properties": {
+              "ac": { "type": "string" },
+              "desc": { "type": "string" }
+            }
+          }
+        },
+        "regression_risk": { "enum": ["low", "medium", "high"] }
+      },
+      "test-scenario-designer": {
+        "scenarios": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": ["id", "type"],
+            "properties": {
+              "id": { "type": "string" },
+              "type": { "enum": ["happy", "negative", "boundary", "edge"] },
+              "priority": { "enum": ["must_test", "should_test", "nice_to_have"] },
+              "title": { "type": "string" }
+            }
+          }
+        }
+      },
+      "browser-validator": {
+        "scenarios": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": ["id", "status"],
+            "properties": {
+              "id": { "type": "string" },
+              "status": { "enum": ["pass", "fail", "skipped", "blocked"] },
+              "evidence_path": { "type": ["string", "null"] }
+            }
+          }
+        },
+        "screenshots": { "type": "array", "items": { "type": "string" } }
+      },
+      "bug-reporter": {
+        "bugs": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": ["id", "severity"],
+            "properties": {
+              "id": { "type": "string" },
+              "severity": { "enum": ["critical", "major", "minor", "trivial"] },
+              "ac_ref": { "type": ["string", "null"] },
+              "file_ref": { "type": ["string", "null"] },
+              "title": { "type": "string" }
+            }
+          }
+        }
+      },
+      "smart-test-selector": {
+        "affected_tests": { "type": "array", "items": { "type": "string" } },
+        "coverage_gaps": { "type": "array", "items": { "type": "string" } }
+      },
+      "environment-manager": {
+        "ready": { "type": "boolean" },
+        "services": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": ["name", "status"],
+            "properties": {
+              "name": { "type": "string" },
+              "url": { "type": ["string", "null"] },
+              "status": { "enum": ["up", "down", "degraded", "unknown"] }
+            }
+          }
+        }
+      },
+      "automation-writer": {
+        "generated_files": { "type": "array", "items": { "type": "string" } }
+      },
+      "manual-validator": {
+        "scenarios": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": ["id", "status"],
+            "properties": {
+              "id": { "type": "string" },
+              "status": { "enum": ["pass", "fail", "skipped"] },
+              "tester_note": { "type": ["string", "null"] }
+            }
+          }
+        }
+      },
+      "release-analyzer": {
+        "repos": { "type": "array", "items": { "type": "string" } },
+        "deployment_risks": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "properties": {
+              "risk": { "type": "string" },
+              "severity": { "enum": ["low", "medium", "high"] }
+            }
+          }
+        }
+      },
+      "orchestrator": {
+        "steps": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": ["agent", "status"],
+            "properties": {
+              "agent": { "type": "string" },
+              "status": { "enum": ["pending", "running", "done", "skipped", "failed"] },
+              "output_ref": { "type": ["string", "null"] }
+            }
+          }
+        }
+      }
+    }
+  }
+}