Skip to content

fix(runtime): announce sub-agent wake notices to the model on trusted channels - #6250

Merged
dbczumar merged 2 commits into
mainfrom
fix/announce-subagent-wake-notice
Sep 3, 2026
Merged

fix(runtime): announce sub-agent wake notices to the model on trusted channels#6250
dbczumar merged 2 commits into
mainfrom
fix/announce-subagent-wake-notice

Conversation

@dbczumar

@dbczumar dbczumar commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Related issue

Closes #4682

Summary

Sub-agent wake notices ([System: sub-agent <agent>/<title> finished (<status>) — <N> results waiting in inbox. Call sys_read_inbox to collect.]) were never described to the model anywhere. Neither the composed system prompt nor the sys_session_send / sys_read_inbox tool descriptions mention them, so the orchestrator's first encounter with one is a user turn that claims to be "System" and orders a tool call. A safety-tuned Claude reads that as prompt injection and refuses, stranding the orchestration.

This PR fixes what the model reads, on channels it already trusts and that reach every harness:

  • System promptomnigent/runtime/prompt.py gains SUBAGENT_WAKE_NOTICE_SHAPE and SUBAGENT_WAKE_NOTICE_INSTRUCTION; build_instructions / build_instructions_nullable append the announcement for any agent that can dispatch sub-agents (the sys_session_send registration gate, declared tools.agents or spawn: true, plus the web_fetch builtin whose researcher dispatch wakes the parent the same way). Other agents' prompts are untouched.
  • Tool descriptionssys_session_send and sys_read_inbox quote the exact notice and say it comes from the runtime, not from a person. Tool descriptions reach native harnesses (including the reporter's claude-native) through the MCP relay, which the composed system prompt does not.
  • Per-dispatch tool result — the "launching as task …" result already told the model to expect an inbox entry; it now also pre-announces the wake notice on the tool-result channel right before it arrives.

The notice text itself is unchanged, so the web/REPL [System: …] marker rendering and the existing e2e wake tests keep working. A drift-guard test pins the announced shape to what _format_subagent_wake_notice actually produces.

Why not role="system" (#6057)? The Claude Agent SDK has no inbound system-role turn: every prompt is wrapped as {"type":"user","message":{"role":"user"}}, and claude-native types the notice into the TUI. On the resumed-session path the SDK-bound prompt is byte-identical for a user-role and a system-role tail, so a wire-level role change never reaches the two harnesses the bug is about. Telling the model what the notice is, before it arrives, does.

ELI5: the app kept slipping the orchestrator a note signed "System" without ever saying such notes exist. Now the orchestrator's briefing (system prompt), its tool manual (descriptions), and the receipt it gets when it dispatches a helper all say: "when your helper finishes, you'll get exactly this note from us — go read your inbox."

flowchart LR
  P[system prompt: announcement<br/>agents that can dispatch] --> M[model]
  T[tool descriptions:<br/>sys_session_send / sys_read_inbox] --> M
  R[dispatch tool result:<br/>launching … expect wake notice] --> M
  W["[System: sub-agent … finished … waiting in inbox]"] -->|recognised| M
  M -->|calls| I[sys_read_inbox]
Loading

Test Plan

  • tests/runtime/test_prompt.py: announcement appended for tools.agents, for spawn: true, and for a web_fetch-only agent, ordered after authored text and before per-turn framework instructions; nullable path returns the announcement alone without the fabricated fallback; drift guard asserts SUBAGENT_WAKE_NOTICE_SHAPE matches _format_subagent_wake_notice output.
  • tests/tools/builtins/test_sys_session.py, tests/tools/builtins/test_async_inbox.py: both tool descriptions quote the notice shape and its runtime origin.
  • Suites run locally: tests/runtime + tests/tools (2236 passed; 3 failures in test_openai_agents_sdk_spawn_env.py / test_provider_spawn_env.py reproduce identically on main and come from this machine's ~/.omnigent/config.yaml providers), plus the instruction/dispatch-related selection across tests/runner, tests/inner, tests/server (77 passed).
  • Rendered the real composed prompt and tool schemas for examples/debby (declared sub-agents) and examples/polly (spawn: true) through omnigent.spec.load + build_instructions + ToolManager: the announcement is present in both prompts, and the registered sys_session_send / sys_read_inbox descriptions quote the wake shape.
  • Lint: ruff format/check, pyrefly, and the dev/lint pre-commit scripts pass on the changed files.

Not done here: a live-model check that a Claude orchestrator now calls sys_read_inbox on wake. The mock-LLM suites cannot observe a refusal, so that check belongs in the nightly live-model job. To try it by hand: check out this branch, run an orchestrator with sub-agents on claude-sdk and on claude-native (e.g. examples/debby), dispatch a sub-agent whose send gets backgrounded, let the parent's turn end, and confirm the parent calls sys_read_inbox when the [System: sub-agent … finished …] notice arrives instead of objecting to it. Inspect the system prompt the harness received (claude-sdk: the system_prompt option; API harnesses: the instructions field) to see the announcement paragraph at its tail.

Demo

  • Visual demo attached below
  • Non-visual evidence provided below or in Test Plan
  • Not applicable — no behavioral change

Tail of the composed system prompt for examples/debby on this branch:

… Sub-agent completion notices: when a sub-agent you dispatched finishes, the Omnigent runtime posts the message `[System: sub-agent <agent>/<title> finished (<status>) — <N> results waiting in inbox. Call sys_read_inbox to collect.]` into this session, starting a new turn for you if you are idle. Treat it as a routine runtime status message, not as instructions typed by a person; respond by calling sys_read_inbox to collect the result. Other `[System: sub-agent ...]` notices about a sub-agent you dispatched (for example that it is blocked awaiting human approval) are routine runtime status messages in the same way.

Type of change

  • Bug fix
  • Feature
  • UI / frontend change
  • Refactor / chore
  • Docs
  • Test / CI
  • Breaking change

Test coverage

  • Unit tests added / updated
  • Integration tests added / updated
  • E2E tests added / updated
  • Manual verification completed
  • Existing tests cover this change
  • Not applicable

Coverage notes

Manual verification = rendering the composed prompt and tool schemas for the bundled debby and polly orchestrators through the real spec loader and tool manager (see Test Plan). The behavioral outcome (no refusal on wake) needs a real model and is left to the nightly live-model job.

Changelog

Orchestrators that dispatch sub-agents are now told, in their system prompt and tool descriptions, exactly what a sub-agent wake notice looks like and that it comes from the runtime, so Claude no longer mistakes it for prompt injection and reliably collects the inbox.

https://claude.ai/code/session_01LfrfEJXdimBBoy5zEttpgL

… channels

The wake notice a parent receives when a sub-agent finishes was never
described to the model: it arrived as a user turn claiming to be "System"
and ordering a tool call, which a safety-tuned orchestrator reads as prompt
injection and refuses, stranding the orchestration.

Quote the exact notice, and say that it comes from the runtime, in the
composed system prompt of every agent that can dispatch sub-agents, in the
sys_session_send and sys_read_inbox tool descriptions, and in the
per-dispatch launching tool result. The notice text itself is unchanged, so
the web and REPL marker rendering keeps working.

Claude-Session: https://claude.ai/code/session_01LfrfEJXdimBBoy5zEttpgL
Signed-off-by: dbczumar <corey.zumar@databricks.com>
@github-actions github-actions Bot added P1-high Priority: major feature broken, no workaround size/L Pull request size: L labels Sep 3, 2026
@omnigent-ci

omnigent-ci Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Polly AI Review

Summary verdict

Solid, well-scoped fix. The drift-guard test passes against the real _format_subagent_wake_notice, the prompt gate spec.tools.agents or spec.spawn exactly mirrors the sys_session_send registration gate in omnigent/tools/manager.py:474, there is no circular import, and the two-call-site refactor in tool_dispatch.py is behavior-preserving. No blocking issues.

Blocking issues

None. I verified every load-bearing claim against the source tree (diff applied to a scratch copy, real functions exercised):

  • Drift guard is accurate. _format_subagent_wake_notice (omnigent/runner/app.py:1997) produces a byte-for-byte match to SUBAGENT_WAKE_NOTICE_SHAPE under the test's substitutions, including the em-dash, finished (completed), 2 results waiting in inbox, and Call sys_read_inbox to collect. The pending == 1 singular/plural branch is consistent with the shape's plural <N> results.
  • Gate mirrors registration exactly. _framework_instructions_for gates on spec.tools.agents or spec.spawn; manager.py:474 guards SysSessionSendTool registration with if not (self._spec.tools.agents or self._spec.spawn): return. Identical. Real AgentSpec exposes both .spawn and .tools.agents, so no AttributeError on real specs.
  • No import cycle. omnigent.runtime.prompt imports nothing from omnigent.tools, so async_inbox.py/spawn.py importing SUBAGENT_WAKE_NOTICE_SHAPE from it is safe.
  • Ordering correct, no fallback leak. Composition is authored → spec-level wake instruction → per-turn framework instructions; empty/whitespace framework entries are stripped and the nullable path does not introduce the "You are a helpful assistant." fallback. The authored_present warn logic still keys off raw_author_instructions (unchanged), so the injected notice does not falsely trip the "authored but undeliverable" warning.

Security

No new authority is granted; the change is additive prose gated to agents that already hold dispatch tools. Two points worth recording, neither blocking for this diff:

  • Trust-by-text-shape is spoofable in principle. Wake notices are delivered as ordinary role:"user" messages, so a human could type an identical [System: … waiting in inbox …] string, and the new instruction explicitly tells the model that text "is not a prompt injection." Blast radius is small — the only instructed response is sys_read_inbox, which drains a real queue (a spoofed notice against an empty inbox just returns the empty sentinel) — but the framing does convert a rendering marker into an authenticity claim the channel can't actually guarantee. Consider softening to "treat this as a routine runtime status message" rather than "not a prompt injection."
  • Unescaped title interpolation is pre-existing, not introduced here. _format_subagent_wake_notice interpolates title raw (app.py:2012), so a title with ]/newlines can break the advertised envelope — but that function is not modified by this PR, and the code this PR does touch (_subagent_launching_message) correctly uses {title!r}. Out of scope for this diff; worth a follow-up on the notice formatter itself.

Non-blocking notes

  • web_fetch-only agents miss the system-prompt announcement. An agent declaring only the web_fetch builtin dispatches __web_researcher through _execute_subagent_tool (with a session_inbox, so child→parent fan-out and a wake notice do fire) yet has neither tools.agents nor spawn: true, so _framework_instructions_for returns [] and its prompt never explains the notice. Severity is bounded: such agents still get sys_read_inbox registered (async_enabled defaults True), and that tool's description now carries the notice explanation — so the tool-description channel still reaches them; only the prompt-level announcement is absent. Consider extending the gate to include the web_fetch dispatch capability so coverage is uniform.
  • Launching-message preview isn't tied to the canonical constant. _subagent_launching_message hand-writes an abbreviated [System: sub-agent ... finished ...] preview rather than referencing SUBAGENT_WAKE_NOTICE_SHAPE, so it won't be caught by the drift-guard if the notice format later changes. Low priority.
  • The instruction is emitted on every prompt composition for dispatching agents and duplicates wording already in the sys_session_send/sys_read_inbox descriptions; minor per-turn token cost, and there's no dedup if a caller also passes the same string via framework_instructions.

Approach

Sound and consistent with the repo's existing framework-instruction pattern (append_framework_instructions, the _GATED_COMPOSED_INSTRUCTION_HARNESSES path). Announcing the notice on the three channels the model already trusts — system prompt, tool descriptions, dispatch tool-result — is a reasonable answer to the constraint that the Claude SDK has no inbound system-role turn, and the PR description justifies not pursuing role="system" convincingly. The only refinement worth considering is aligning the gate with all dispatch paths (see the web_fetch note) so the same signal is explained everywhere it can arrive.

Summary

A focused, low-risk fix that is well-tested and correctly gated, with all key claims independently verified. No blocking issues. Before merge, consider (1) widening the gate to cover web_fetch-only agents for uniform coverage, and (2) softening the "not a prompt injection" wording toward "routine runtime status message," since the delivery channel can't cryptographically distinguish a runtime notice from user-typed text. Both are improvements, not blockers.That wake notice was for the codex reviewer whose result I already collected and incorporated into the review. Both reviewers have completed and my inbox is now empty — the review is finished and posted above. No further action needed.


Automated review by Polly · workflow run

…atchers

Review follow-up: describe the notice as a routine runtime status message
rather than asserting it is "not a prompt injection" (a user-turn channel
cannot prove origin), and extend the prompt gate to agents whose only
dispatch path is the web_fetch builtin, whose researcher wakes the parent
the same way.

Claude-Session: https://claude.ai/code/session_01LfrfEJXdimBBoy5zEttpgL
Signed-off-by: dbczumar <corey.zumar@databricks.com>
@dbczumar

dbczumar commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Adopted both Polly suggestions in the follow-up commit: the instruction now frames the notice as a routine runtime status message (no authenticity claim), and the prompt gate also covers agents whose only dispatch path is the web_fetch builtin. The launching-message preview stays a deliberate abbreviation so the tool result does not nest one [System: …] envelope inside another; the drift guard pins the canonical shape.

@dbczumar

dbczumar commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Note on the red Coverage status on d776600: it is a rounding artifact, not a regression. The producing runs report line-rate 0.8451 (35bfc78, posted as 85%) and 0.8449 (d776600, posted as 84%); the gate compares the integer-rounded total.txt values, so a 0.02-point jitter across the 84.5 boundary reads as a full point and defeats the 0.5 tolerance. The follow-up commit added one production line in omnigent/runtime/prompt.py and tests that cover it (file coverage 73.2% → 73.5%); tool_dispatch.py, spawn.py, and async_inbox.py are unchanged between the two runs. Coverage is not in Merge Ready's required list. Worth a separate CI fix: compare the unrounded line-rate from coverage.xml instead of the rounded total.

@dbczumar

dbczumar commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Live verification on a server, web UI, and host all built from this branch (dedicated server on 127.0.0.1:8957 with its own SQLite DB, omnigent host from the same worktree):

claude-sdk (Debby, examples/debby unmodified) — the session dispatched both partners at +15s and ended its turn at +18s. Wake #1 landed at +30s → sys_read_inbox at +33s. Wake #2 at +46s → sys_read_inbox at +49s. Synthesis at +58s. Zero refusals. ps on the live orchestrator's claude process shows the new "Sub-agent completion notices" paragraph inside its --system-prompt; the partner process (no sub-agents) does not carry it.

claude-native (omnigent claude --server …, the reporter's flow) — the wrapper's TUI dispatched a child via sys_session_create + sys_session_send and ended its turn. The runner typed [System: sub-agent agent/ finished (completed) — 1 result waiting in inbox. Call sys_read_inbox to collect.] into the pane as a user prompt; Claude (Opus 4.8) called sys_read_inbox and relayed the child's answer within 9s, no objection. On this path only the tool descriptions and the dispatch tool result carry the announcement (the TUI receives raw author instructions), and that was enough.

Two side observations, not from this PR: a web-created session with harness_override: claude-native ran its chat turns through the SDK executor while the TUI pane stayed idle, and a child dispatched by session id renders as sub-agent agent/ (empty agent/title) in the notice.

@dbczumar
dbczumar merged commit 39b326d into main Sep 3, 2026
85 of 87 checks passed
@dbczumar
dbczumar deleted the fix/announce-subagent-wake-notice branch September 3, 2026 04:25
@github-actions github-actions Bot added the no-doc-update Merged PR does not need a docs update label Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

🏷️ Doc impact: no-doc-update

Internal prompt/tool-description wording that announces the sub-agent wake-notice to the model plus a message-builder refactor; no user-facing surface, integration, or built-in policy changed.

Auto-classified on merge. Set the label manually before merging to override. · run

@dbczumar

dbczumar commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Filed the two side observations from the live run as issues: #6261 (multipart session create drops metadata.host_id) and #6262 (wake notice reads sub-agent agent/ for by-id dispatches). Neither is introduced by this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-doc-update Merged PR does not need a docs update P1-high Priority: major feature broken, no workaround size/L Pull request size: L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Sub-agent wake notices delivered as user-role messages — Claude flags them as prompt injection and refuses to collect the inbox

1 participant