You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
refactor(do): remove fable from the model selection policy (#896)
Owner directive: "our document for what agent to use should remove fable
entirely" — removed, not demoted.
- skills/meta/do/SKILL.md: drop the Fable-5 variant row, the manual-pick
set and ordering, the cross-provider escalation target, and the
felt-quality clause comparing fable to sol; reword the sentences that
depended on them.
- scripts/build-dispatch.py: drop fable from VALID_MODELS,
ANTHROPIC_MODELS, and the manual-override gate (now sonnet alone).
- hooks/routing-decision-recorder.py: drop fable from the valid-model set.
- scripts/validate_component_models.py: ALLOWED_MODELS is {sonnet, opus}.
- docs/PHILOSOPHY.md, agent-frontmatter-template.md,
component-audit-checklists.md: same removal in live instructions.
- Tests updated to match, including SUPPLIED_CLAUDE_POINTS.
Opus-4.8 and Sonnet-5 prior-measurement rows are kept as manual-only:
they are the Anthropic lane's only DeepSWE data (Opus 5 is unbenchmarked)
and cover cost, latency, context-window, and fan-out constraints.
Dated historical records keep their fable references
(docs/what-didnt-work.md, scripts/routing-ab-results/) — editing them
would falsify experiment records; neither is a live routing instruction.
Owner model-selection policy (ADR `model-selection-policy`; operational table in `skills/meta/do/SKILL.md`, Model Selection):
346
346
347
-
**Harness-native routing:** each harness defaults to its own provider's model lane. Cross-provider dispatch is manual-only, never automatic. Start low, escalate on miss — high tiers cost 3-6x per Pass@1 point where measured. Plan budget ($200/month per provider) makes cost a first-class constraint. Three decision axes: the current session model (the harness runs Opus 5, the owner-directed Anthropic-lane default at every task class), DeepSWE Pass@1 (agentic completion) where a model has been measured, and owner-observed felt quality (fable > sol noticeable, opus > gpt-5.5 marginal); ties resolve in favor of felt quality.
347
+
**Harness-native routing:** each harness defaults to its own provider's model lane. Cross-provider dispatch is manual-only, never automatic. Start low, escalate on miss — high tiers cost 3-6x per Pass@1 point where measured. Plan budget ($200/month per provider) makes cost a first-class constraint. Three decision axes: the current session model (the harness runs Opus 5, the owner-directed Anthropic-lane default at every task class), DeepSWE Pass@1 (agentic completion) where a model has been measured, and owner-observed felt quality (opus > gpt-5.5 marginal); ties resolve in favor of felt quality.
348
348
349
349
| Task class | Anthropic lane (Claude Code) | OpenAI lane (Codex CLI) |
The `/do` table is canonical and records the full DeepSWE Pass@1 / cost / tokens / steps data, including the prior Fable-5 / Opus-4.8 / Sonnet-5 measurements kept for manual picks. Opus 5 has no DeepSWE run yet, so its pts/USD is uncomputable and the selection rests on the session-model and owner-directive axes; effort still follows start-low-escalate-on-miss. Max-power requires `manual_model_override=true` in both lanes. Fable/sonnet are manual-only. Legacy `gpt-5.5` and non-default GPT-5.6 points are manual-only. Haiku is retired (routing was Haiku pre-#777; self-route since — `scripts/routing-ab-results/self-route-v1/VERDICT.md`). Defaults, not limits: escalate when cheaper output misses the bar; for anything that ships, intelligence > taste > cost, with cost a tie-breaker only. Fan-out uses the lane's low-risk point; one synthesis agent may run one tier higher.
357
+
The `/do` table is canonical and records the full DeepSWE Pass@1 / cost / tokens / steps data, including the prior Opus-4.8 / Sonnet-5 measurements kept for manual picks. Opus 5 has no DeepSWE run yet, so its pts/USD is uncomputable and the selection rests on the session-model and owner-directive axes; effort still follows start-low-escalate-on-miss. Max-power requires `manual_model_override=true` in both lanes. Sonnet is manual-only. Legacy `gpt-5.5` and non-default GPT-5.6 points are manual-only. Haiku is retired (routing was Haiku pre-#777; self-route since — `scripts/routing-ab-results/self-route-v1/VERDICT.md`). Defaults, not limits: escalate when cheaper output misses the bar; for anything that ships, intelligence > taste > cost, with cost a tie-breaker only. Fan-out uses the lane's low-risk point; one synthesis agent may run one tier higher.
358
358
359
359
**Coordinator model.** The main-thread coordinator routes and evaluates, never executes — its cost is input-dominated and Pass@1 measures execution it never does. Anthropic harness → opus (Opus 5, the session model); OpenAI harness → gpt-5.6-terra/high; downgrade the anthropic coordinator to sonnet only as a deliberate plan-limit measure, set via harness config (`/model`), not per-turn. Full rule: `skills/meta/do/SKILL.md`, Model Selection.
0 commit comments