Skip to content

feat(orchestration-v2): add native subagent and workflow observability - #4664

Open
shivamhwp wants to merge 57 commits into
t3code/codex-turn-mappingfrom
subagent-obs/05-thread-visibility
Open

feat(orchestration-v2): add native subagent and workflow observability#4664
shivamhwp wants to merge 57 commits into
t3code/codex-turn-mappingfrom
subagent-obs/05-thread-visibility

Conversation

@shivamhwp

@shivamhwp shivamhwp commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

This adds native subagent and workflow observability to Orchestrator V2. It is based directly on the V2 implementation in PR #2829; no V1 runtime or compatibility data source remains in this PR.

Why this PR exists

Orchestrator V2 has its own event model, projection, provider boundary, recovery path, and thread lifecycle. When V2 ships, provider-native child work needs V2-owned identities and activations so the Agents experience stays accurate through reuse, interruption, restart, nesting, and background completion.

Before and after

The comparison below is against the current t3code/codex-turn-mapping V2 base, not main/V1.

V2 base — before This PR — after
Every native child renders as a separate timeline row. Internal child threads also appear in Lineage. One compact CTA represents the spawn batch; live state, settled count, and token totals update in place. Internal provider child threads stay out of the sidebar and Lineage.
V2 base showing separate live subagent rows PR showing a compact live Agents CTA
Completed children remain separate cards in the conversation. The CTA opens one detailed Agents panel with stable rows for role, duration, progress/result, model, tokens, tools, status, and aggregate totals.
V2 base showing separate completed subagent rows PR showing the expanded Agents panel

Responsive web verification at 390×844:

Responsive Agents panel at 390 by 844

What changes

V2 domain and persistence. Reusable subagent identities and immutable per-activation records live in the V2 projection. Migration 050 persists activations, and schema defaults keep older V2 payloads readable without a backfill.

Provider-native projection. Claude, Codex, Cursor, OpenCode, and ACP/Grok emit V2 subagent and activation lifecycle updates, including role, usage, recent activity, workflow membership, activation counts, idle state, and provider-reported nesting.

Reuse and recovery. Reused agents bind to the run that reactivates them. Activation proof prevents stale post-interrupt traffic from attaching to a later run, and recovery retains the cumulative usage baseline.

V2-only client state. The Agents panel derives directly from OrchestrationV2Subagent[]. The V1 activity fold, dual-source fallback, and obsolete tests are removed. V2 timeline items collapse into one CTA per workflow or direct-spawn batch.

Internal thread visibility. One shared V2 predicate controls filtering on server, web, and mobile. Hidden child deep links resolve through the existing targeted projection RPC and redirect to their parent without adding child shells to the global stream. Unfiltered thread refs remain available for cleanup so hidden worktrees are not mistaken for orphans.

Explicitly not included

  • No V1 orchestration runtime, V1 activity fold, or compatibility data source.
  • No second Agents panel or alternate roster.
  • No new targeted subagent-stop command or backend-only provider-control path.
  • No replacement for background liveness or workflow script inspection already present in the V2 base.

Current stack

  • Base: 4c55679fdf6387808d815b69f486b6caadadad6f (t3code/codex-turn-mapping)
  • Head: 0d57bbdf838b07163c11a9b2da2914db75106fc5
  • 57 commits, 89 files, +7,406 / -2,434
  • Rebased onto the current V2 base; no merge conflicts remain locally

Verification

  • 171/171 changed UI and client-runtime tests pass
  • 179/179 focused Orchestrator V2 and provider-adapter tests pass
  • Targeted formatting, lint, diff whitespace, web typecheck, client-runtime typecheck, and contracts typecheck pass
  • Chrome: live and completed native-agent batches, Agents-panel open/close, keyboard focus ring and pointer affordance, responsive 390×844 layout, no horizontal overflow, no top-level child-thread leakage, and hidden-child deep-link redirect
  • Clean reload and redirect flows produce no browser console errors
  • Server typecheck still hits the same unrelated ProviderSessionManager.test.ts:836 error on the untouched V2 base

Provider limitation: if a provider reports a nested agent only inside its parent's final text and emits no lifecycle frame, T3 shows that summary but does not invent a nested agent row.

Built with Codex (GPT-5.6 Sol) through T3 Code.

Note

Add subagent activation tracking and observability to orchestration-v2

  • Introduces OrchestrationV2SubagentActivation as a first-class contract type with id, status, usage, timestamps, and ordinal; persisted in a new orchestration_v2_projection_subagent_activations table (migration 050) and returned as part of thread projections.
  • All provider adapters (Claude, Codex, Cursor, ACP, OpenCode) now emit subagent_activation.updated events, accumulate token usage per activation, track currentActivationId, and propagate terminal status on turn completion.
  • Internal subagent threads (lineage subagent or node-forked) are filtered from shell snapshots, sidebar, and thread-ref lists; deep links to hidden threads redirect to the parent thread on both web and mobile.
  • The messages timeline collapses per-subagent events into grouped CTA rows (AgentSpawnCtaRow) showing aggregated status buckets and token totals; the Agents panel now indents nested direct-spawn agents.
  • Run execution service gains identity-based routing for reusable subagents: a run can re-adopt a subagent after idle by proving activation-count progression, and activations keep runs open while live.
  • A new listAllThreadRefs WS RPC (requiring orchestration read scope) returns unfiltered thread refs including hidden subagents, used by worktree cleanup to avoid orphaning subagent worktrees.
  • Risk: projection schema version bumped from 2 to 3; the ./state/subagentRuntime client-runtime export is replaced by ./state/thread-subagents.

Macroscope summarized 0d57bbd.


Note

High Risk
Large cross-cutting changes to provider adapters, projection schema/persistence, and run attribution for reusable subagents; incorrect reuse or stale-event handling could mis-route events or strand background work.

Overview
Orchestration V2 gains per-activation subagent records (subagent_activation.updated, usage, currentActivationId, roles/kinds, workflow workers) projected from Claude, Codex, Cursor, and ACP adapters, with registry/rehydration from existingSubagents after idle or restart and race-safe resume/terminal handling (Codex idle vs completed, Claude commitClaudeSubagentRegistryEntry, nested Agent/Workflow parenting).

Clients treat internal subagent threads via isOrchestrationV2InternalSubagentThread: sidebar/list filtering, mobile deep-link redirect to parent plus a short unavailable delay, relationships banner drops stale subagent edges, and work log no longer links subagent rows as separate threads. Server adds listAllThreadRefs (orchestration read scope) so cleanup sees hidden child worktrees.

Tests/fixtures expand for activations, Codex reuse-after-idle replay, and list filtering (including node-forked children).

Reviewed by Cursor Bugbot for commit 0d57bbd. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: bde13937-a17e-478e-b7e2-db16c3d0c4b4

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch subagent-obs/05-thread-visibility

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Jul 27, 2026
@shivamhwp
shivamhwp force-pushed the subagent-obs/04-agents-panel branch from 940e8fb to c3a50f8 Compare July 27, 2026 20:20
@shivamhwp
shivamhwp force-pushed the subagent-obs/05-thread-visibility branch from d2ef6c7 to adb1e9b Compare July 27, 2026 20:20
@shivamhwp
shivamhwp force-pushed the subagent-obs/04-agents-panel branch from c3a50f8 to 8c6a796 Compare July 27, 2026 20:52
@shivamhwp
shivamhwp force-pushed the subagent-obs/05-thread-visibility branch from adb1e9b to 39912b0 Compare July 27, 2026 20:52
Comment thread apps/web/src/components/chat/ThreadRelationshipsControl.tsx
@shivamhwp
shivamhwp force-pushed the subagent-obs/04-agents-panel branch from 8c6a796 to 103b937 Compare July 27, 2026 23:03
@shivamhwp
shivamhwp force-pushed the subagent-obs/05-thread-visibility branch from 39912b0 to 6bc3a8d Compare July 27, 2026 23:03
@shivamhwp
shivamhwp force-pushed the subagent-obs/04-agents-panel branch from 103b937 to 31adb36 Compare July 28, 2026 15:31
@shivamhwp
shivamhwp force-pushed the subagent-obs/05-thread-visibility branch from 6bc3a8d to 05fe44c Compare July 28, 2026 15:31
@shivamhwp
shivamhwp force-pushed the subagent-obs/04-agents-panel branch from 31adb36 to b270c47 Compare July 28, 2026 19:27
@shivamhwp
shivamhwp force-pushed the subagent-obs/05-thread-visibility branch from 05fe44c to 7458298 Compare July 28, 2026 19:27
@shivamhwp
shivamhwp force-pushed the subagent-obs/04-agents-panel branch from b270c47 to daf1a88 Compare July 28, 2026 19:46
@shivamhwp
shivamhwp force-pushed the subagent-obs/05-thread-visibility branch from 7458298 to 4daeac4 Compare July 28, 2026 19:46
@shivamhwp
shivamhwp force-pushed the subagent-obs/04-agents-panel branch from daf1a88 to 1dce464 Compare July 28, 2026 20:26
@shivamhwp
shivamhwp force-pushed the subagent-obs/05-thread-visibility branch from 4daeac4 to 095cab7 Compare July 28, 2026 20:26
@shivamhwp
shivamhwp force-pushed the subagent-obs/04-agents-panel branch from 1dce464 to 4c169ed Compare July 29, 2026 01:19
@shivamhwp
shivamhwp force-pushed the subagent-obs/05-thread-visibility branch from 095cab7 to 1d37a6f Compare July 29, 2026 01:19
@shivamhwp
shivamhwp force-pushed the subagent-obs/04-agents-panel branch from 4c169ed to 70d5111 Compare July 29, 2026 01:24
@shivamhwp
shivamhwp force-pushed the subagent-obs/05-thread-visibility branch from 1d37a6f to 6cf67a2 Compare July 29, 2026 01:25
@shivamhwp
shivamhwp force-pushed the subagent-obs/04-agents-panel branch from 70d5111 to c78d0ff Compare July 30, 2026 00:00
@shivamhwp
shivamhwp force-pushed the subagent-obs/05-thread-visibility branch from 6cf67a2 to 5e3c535 Compare July 30, 2026 00:00
@shivamhwp
shivamhwp force-pushed the subagent-obs/04-agents-panel branch from c78d0ff to a997806 Compare July 30, 2026 00:45
@shivamhwp
shivamhwp force-pushed the subagent-obs/05-thread-visibility branch from 5e3c535 to 2430736 Compare July 30, 2026 00:45
@shivamhwp
shivamhwp marked this pull request as ready for review July 30, 2026 20:04
Comment thread apps/web/src/state/entities.ts
shivamhwp and others added 29 commits August 17, 2026 18:41
The rebase both-added this file: the base's lineage-window tests and the
detach-control tests landed as competing versions. Keep both suites, and
teach the panel mocks about the base's row-ordering export.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ting

Pre-seeded reactivatable identities were treated as current-run
ownership, so an interrupted prior run's trailing traffic — delayed
non-terminal rows, activation updates, subagent items — was admitted and
misattributed: stale child threads got adopted, ingestion could be held
open indefinitely, and finalization could terminalize a prior run's
activation with this run's outcome.

Identity routing now requires proof that this run drives the agent. The
seed carries each identity's activation count at turn start; a genuine
re-activation always advances past that baseline, while trailing
traffic re-emits at it. Rows carrying this run's id, rows advancing the
baseline, and activations with a higher ordinal all prove it; until one
arrives, identity-only events are refused, and the lifecycle tracker
keys on the proven set rather than the seeded one.

Also folds in two review fixes: Claude cache-read tokens now count into
inputTokens (they are input the model consumed, and excluding them broke
reconciliation with the four-component total), and a running workflow
coordinator stays counted in the Agents panel while all its members are
settled — between phases the coordinator is the only live row, and
dropping it read "0 active" for a visibly running workflow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…turn-start test

Aliasing a module's layer erases the service namespace at the call site;
the boundary convention keeps the module name visible.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n activation

registerSubagentThreads emits a running subagent whose currentActivationId
stays null until the child turn opens an activation. The settled-noise gate
skipped every state without one, so a terminal state carried on the spawning
frame itself was dropped and the agent stuck at running. The gate now only
drops states for identities that are already settled; an active identity
without an activation is a just-registered agent whose state still counts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The base rebased onto main again, renumbering migrations (subagent
observability moves 045 to 046) and reworking three paths this series
extends: Claude's background-task lookup, ACP's terminal-status helper,
and turn-item routing for inherited background items. Each merge keeps
both intents — the base's newer mechanism with this branch's activation,
identity-proof, and completedAt semantics.

ProviderTurnStartService.test.ts was added on both sides as different
files; rebuilt as the base's suite plus this branch's rehydration tests.
Test mocks and fixtures gain the members the observability types now
require.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Worktree cleanup treats listAllThreadRefs as the complete set, but it
mapped only the live shells. A worktree held solely by an archived
thread was therefore classified orphaned and offered for removal — the
same data-loss shape this RPC exists to prevent, through a different gap.

Also drops the two toSorted calls in the subagent derivations: this
module is mobile-reachable and Hermes ships no ES2023 change-by-copy
array methods, so an Android import could crash. The regression deletes
toSorted from the prototype while exercising both derivations.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n the turn-start test

The rebase reconstruction of this file left each service module imported
twice — once as a namespace and once for its tag. At a service boundary
the namespace shape is the convention, so the duplicate named imports go
and their uses read through the module.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@shivamhwp
shivamhwp force-pushed the subagent-obs/05-thread-visibility branch from 5c045fd to 0d57bbd Compare August 17, 2026 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL 1,000+ changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant