Skip to content

fix(core): surface systemic authority refusal recurrence - #1034

Open
Lykhoyda wants to merge 5 commits into
mainfrom
implement-react-native-development-agent-features
Open

Lykhoyda wants to merge 5 commits into
mainfrom
implement-react-native-development-agent-features

Conversation

@Lykhoyda

@Lykhoyda Lykhoyda commented Sep 14, 2026

Copy link
Copy Markdown
Owner

Summary

  • Recognize all six authority refusal codes through bounded envelope decoding, preserve sanitized tool-level evidence, and stop successful diagnostic calls from being counted as authority recovery.
  • Add deterministic cross-tool refusal aggregation and JSON/text CLI reporting while retaining the existing family, new-pattern, and recurring collections.
  • Include unambiguous legacy evidence without rewriting historical records, regenerate both host runtimes, and add patch changesets for core and plugin delivery.
  • Redact UUID-shaped device identifiers (simulator UDIDs) from stored refusal symptoms and bump the redaction rules version so retained rows re-sanitize on their next rewrite.

Closes #981.

Behavior And Boundaries

Failed tool call
  -> bounded refusal decoding and sanitized recording
  -> existing tool-specific pattern
  -> systemic group by code / axis / cause / platform
       -> occurrences, tools, signatures, observation dates, provenance
       -> recovery not verified; current authority state unknown

Structured codes take precedence over prose, including unknown codes. Only the first MCP content item is eligible, with a 16 KiB JSON limit. Missing or conflicting metadata stays unknown; causes remain unknown until a fixed vocabulary is supported. Malformed refusal extensions cannot fall back to legacy interpretation.

Reporting covers retained local history, not exact time-window counts; --since affects only new-pattern selection. Rising counts with zero recorded recoveries do not prove a dead end or a currently blocked session. Session authority, admission, cleanup, runner sidecars, and the storage-signature algorithm remain unchanged. No live-device recovery proof is claimed.

Validation

Measured at head d4d7de43 (the UUID redaction fix on top of 3956fff9):

  • Core build and all 90 tests across the authority-refusal, evidence, systemic-trends, trends-CLI, instrumentation, and runner-diagnostics suites, including the new two-UDID redaction regression.
  • Core and both shipped CLIs agree across 24 JSON/text subprocess invocations; store bytes, mtime, and directory contents remain unchanged.
  • Both host runtimes regenerated through the canonical writer; check-dist-fresh, package-sync, TypeScript-only, formatting, lint, and git diff --check pass.
  • Required Build & Test CI passes at d4d7de43: 24 check runs succeeded and CodeQL reported neutral.
  • Independent QA at d4d7de43: reporting scope PASS (fix(core): surface systemic authority refusal recurrence #1034 (comment)); full iOS/Android acceptance FAILED on the Android first-run tutorial and saved-action replay journey (fix(core): surface systemic authority refusal recurrence #1034 (comment)), which is being addressed in the workspace fixture rather than in this reporting change.
  • Full local test suite was not re-run at d4d7de43; the exact-revision comparison below covers the eight historical cases only.
  • Final review.

Local Test Failures

Historical full-suite runs on earlier checkpoints (not re-run since, kept for the record):

Revision Tests Passed Failed
Earlier full implementation checkpoint (pre-rebase) 5901 5893 8
Unchanged Phase 2 checkpoint 08dfa90e 5889 5882 7

Both completed runs had zero cancelled or skipped tests; the serial leg passed separately on both. Seven failures matched: five action-engine migration assertions, one save-as-action overwrite-lock timeout, and one managed-Metro attestation assertion. The eighth, hard-reset shutdown proof, reproduced in 2 of 10 isolated runs on each revision at that time. The comparison checkpoint includes Phases 1 and 2; it is not pristine main.

Exact-revision comparison of those eight cases (2026-09-14): candidate d4d7de43 versus current main d76a9e2a (merge-base 20854792), same macOS host, Node v26.8.1, unsandboxed shell, disposable copies, the four test files byte-identical on both revisions. "Original" is the default macOS $TMPDIR under /var/folders (a symlink to /private/var); "normalized" is a realpath-clean temp root on the task-owned SSD.

Case Original $TMPDIR Normalized $TMPDIR Classification
Five action-engine-compat migration assertions Fail on both revisions (64/69) Pass on both (69/69) Environment: temp-root symlink versus realpathed action paths. Pre-existing.
save-as-action overwrite-lock timeout Fail on both (10/11) Pass on both (11/11) Environment: same temp-root symlink. Pre-existing.
Managed-Metro attestation (managed-sandbox-v1 expected, reported-v1 actual) Fail on both (14/15) Fail on both (14/15) Environment: Darwin preflight reports resolvedCommandAllowed:false, command exit 127, manifest utility unresolved. Pre-existing.
Hard-reset shutdown proof, 10 isolated runs each 0/10 failed on both not repeated Still unreproduced; the historical intermittent label stands.

None of the eight is introduced by this PR. No test or authority requirement was weakened, and the historical rows are not relabeled as current passes.

Merge Status

Head d4d7de43 is based on 20854792; main has since moved to d76a9e2a without touching the experience or trends paths. GitHub reports the PR as mergeable with the required Build & Test check green. Full acceptance remains open on the Android journey above, so no merge is requested until independent QA passes that row.

Greptile Summary

This PR adds bounded decoding and sanitized persistence for six authority-refusal codes, aggregates recorded and unambiguous legacy evidence into deterministic systemic trends, and exposes those trends through JSON and text CLI reporting.

  • Keeps successful diagnostic calls from being treated as verified authority recovery.
  • Groups retained observations by refusal code, axis, cause, and sanitized platform.
  • Preserves existing family, new-pattern, and recurring trend collections.
  • Regenerates both marketplace host runtimes and adds the required core/plugin changeset.

Confidence Score: 5/5

The reviewed changes appear safe to merge, with no concrete correctness, security, or repository-rule violations identified.

The decoder matches the established producer shapes, conflicting metadata intentionally fails closed to unknown, aggregation remains deterministic, and the reporting boundaries are explicitly represented and extensively tested.

Important Files Changed

Filename Overview
packages/rn-dev-agent-core/src/experience/authority-refusal.ts Introduces bounded structured and legacy refusal decoding, metadata validation, merging, and stable systemic-key generation.
packages/rn-dev-agent-core/src/experience/evidence.ts Records sanitized authority-refusal facts and prevents refusal observations from becoming immediate-recovery candidates.
packages/rn-dev-agent-core/src/experience/trends.ts Adds deterministic retained-history aggregation for recorded and eligible legacy authority refusals.
packages/rn-dev-agent-core/src/experience-trends.ts Extends text reporting and usage guidance with systemic-refusal output and explicit scope limitations.
seed-experience/common-failures.yaml Documents all six authority-refusal families and supported recovery guidance.
packages/rn-dev-agent-core/test/unit/experience-systemic-trends.test.ts Covers grouping identity, legacy compatibility, deterministic ordering, metadata validation, retention, and reporting semantics.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Failed or errored tool call] --> B[Bounded authority-envelope decoding]
    B -->|Recognized code| C[Sanitize symptom and metadata]
    B -->|Not recognized| D[Existing experience classification]
    C --> E[Persist code-specific tool pattern]
    E --> F[Aggregate retained records]
    G[Eligible legacy symptom] --> F
    F --> H[Group by code, axis, cause, and platform]
    H --> I[JSON report]
    H --> J[Text CLI report]
    I --> K[Recovery not verified; authority state unknown]
    J --> K
Loading

Reviews (1): Last reviewed commit: "feat(core): include unambiguous legacy r..." | Re-trigger Greptile

@Lykhoyda
Lykhoyda marked this pull request as ready for review September 14, 2026 08:28
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-14T09:49:56.090955Z d4d7de4 New commits
🔒 Security Review Completed 2026-09-14T08:40:41.546366Z a8a10ed Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cursor
cursor Bot force-pushed the implement-react-native-development-agent-features branch from a8a10ed to 3956fff Compare September 14, 2026 08:37
A recognized authority refusal persists the producer's error text after
sanitizeString, and METRO_ORIGIN_MISMATCH names the bound device by its
simulator UDID. The shared redaction rules did not cover UUIDs, so the
UDID landed in patterns.jsonl.

Add one UUID rule to REDACTION_RULES and bump REDACTION_RULES_VERSION so
retained rows re-sanitize on their next rewrite. Evidence pointers keep a
random id but drop the dashes so the rule does not collapse them.
Regenerate both host runtimes.
@Lykhoyda

Lykhoyda commented Sep 14, 2026

Copy link
Copy Markdown
Owner Author

FAILED — acceptance incomplete at d4d7de43667e9c00f53d25a608ecdf77c93bc581. The reporting checks exercised with genuine product-emitted refusals passed. The required live iOS journey did not establish usable UI/navigation, and the decoder boundary scenarios below remain NOT RUN. This is evidence for the captain, not a merge gate. A changed or rebased head needs fresh verification.

The brief's earlier conflict/no-CI/draft caveats were checked against GitHub: this head is OPEN, not draft, MERGEABLE, and CI exists. The description still records 8 unresolved earlier local test failures: 7 matching its comparison checkpoint and 1 intermittent hard-reset shutdown proof failure reproduced on both revisions. The checkpoint already contains implementation Phases 1 and 2; it is not pristine main. Those local results were not rerun or relabeled here.

Final GitHub recheck (2026-09-14 10:01 UTC): head unchanged; OPEN, draft=false, MERGEABLE. Check summary: 1 NEUTRAL, 24 SUCCESS.

Observed on the exact candidate, one isolated iOS session:

  • BUNDLE_HANDSHAKE_UNAVAILABLE from cdp_dev_settings(hideDevMenu), cdp_component_tree, and cdp_navigation_state became one systemic group: 3 occurrences, 3 tools, 3 signatures, exact dates, axis B, unknown cause/platform, and recorded provenance. The unknown platform reflects the event payload, despite the iOS session.
  • RUNNER_OWNERSHIP_MISMATCH from device_screenshot remained a separate group, axis R/platform iOS. No refusal envelopes or store records were fabricated or injected.
  • Successful status and inventory calls between refusals left all recognized records at zero recovery count, with recovery unverified and current authority state unknown.
  • A real outside-family code, APP_INSTALL_IDENTITY_CHANGED, stayed UNKNOWN and outside systemic aggregation.
  • Both shipped host CLIs agree in JSON and text. Fixed --since; only generated-at timestamps normalized; both diff -u results empty. JSON includes signatures/provenance. The systemic text section deliberately omits those fields; that is not a defect.
  • Reporting preserved the live store's 8,577 bytes, mtime 1789379682337610823 ns, SHA-256 0c50044cfbf12858ebb38875b786dfef4bcbf27d6ad24b37d307c05abec6a860, and directory contents. A preserved copy of genuine prior history also stayed unchanged and produced 12 legacy-derived groups / 942 retained occurrences; both hosts agreed and member counts/tools/dates matched.

Privacy: CLEAN for the fresh patterns.jsonl inspected. All seven records use redaction version 2. No UUID, known device identifier, known hostname, or local/home path was persisted there. A Metro refusal carrying a real simulator identifier did not occur, so the specific UDID-scrubbing case remains NOT RUN; the clean scan does not prove that missing case.

NOT RUN: fresh MCP cases for SESSION_AUTHORITY_REQUIRED, METRO_ORIGIN_MISMATCH, HANDOFF_NOT_AUTHORIZED, and NON_GIT_MANIFEST_REQUIRED; structured-code/prose disagreement; conflicting later MCP content items; the 16 KiB boundary; Android. The actual structured envelopes contained a single item of 225–377 bytes. Missing cases were not replaced with synthetic tests or inferred from source.

Why live acceptance stopped: first build hit the known simulator install race; the single prescribed stop-Metro/build retry installed and launched the Test App. Hermes then exposed one matching target, but the validator's final process probe returned APP_NOT_RUNNING. Its preserved output eventually contained the Test App entry after about 7.14 seconds, exceeding the 5-second probe limit. This supports a timed-out probe, not an established app crash. Workspace dev/validator/lib/probe.sh:131–133 maps probe errors to an empty process list and then to that misleading code; distinguishing probe unavailability from app absence is actionable follow-up. No third build or alternate native control path was used.

Workspace/Test App base: 0715df24e4b18f027d21158c26176785a7399533, detached. The supplied thin stdio client relayed public candidate MCP calls, with JSON built using jq. simctl output was captured on the boot volume first. The validator's menu-dismissal step was delegated to the required public hideDevMenu request, which refused. The diagnostic image below renders captured results; no app screenshot or navigation proof is claimed.

Cleanup proven: owned runner closed, Metro stopped, integration restored, session released, transport stopped, and final validator cleanup reports empty authority plus absent owned simulator/listener/integration residue. Foreign devices were untouched. The subsystem-removal proposal is outside this verdict.

Diagnostic summary of real refusal and reporting evidence

@Lykhoyda

Lykhoyda commented Sep 14, 2026

Copy link
Copy Markdown
Owner Author

PASS — tested refusal recognition/reporting at d4d7de43667e9c00f53d25a608ecdf77c93bc581. Four priority codes were emitted by the real candidate on owned iOS/Android targets. This is scoped QA evidence, not a merge gate; any changed or rebased head needs fresh verification.

GitHub currently reports OPEN, not draft, MERGEABLE, with 24 successful checks and one neutral check. The earlier conflict/no-CI caveats no longer match that state. The description’s eight earlier local failures remain documented: seven matching its comparison checkpoint and one flaky hard-reset shutdown proof. That checkpoint includes implementation Phases 1 and 2; these failures were not rerun or relabeled here.

The repaired workspace base was 8c0187e4622694708c397835fc3125dea8ec43cf. The candidate remained clean. A thin stdio client relayed public MCP tools only; no refusal envelopes or experience records were injected.

  • iOS: the validator reached readiness after its one prescribed install-race retry. Native UI, route/store reads and the saved action passed through to the QA result screen. Replay transport was cdp-js with per-step readback.
  • Android: the pinned maestro-runner 1.1.24 action failed at app-root after a first-run tutorial was observed. Its parked runner then produced real screenshot/snapshot refusals. Subsequent pinning timed out with no target; the final screenshot is the dev-client picker, not successful app navigation.
Code Live evidence
BUNDLE_HANDSHAKE_UNAVAILABLE Before pinning and after Android replay/relaunch; multiple CDP/session tools
RUNNER_OWNERSHIP_MISMATCH Android screenshot and snapshot after runner parking, with successful diagnostics between
METRO_ORIGIN_MISMATCH Owned iOS app terminated through public MCP while managed Metro remained; origin gate refused the missing attachment
SESSION_AUTHORITY_REQUIRED Real refusal of integration restoration while Android managed Metro was active
HANDOFF_NOT_AUTHORIZED NOT RUN: no genuine donor/recipient handoff; no fake capability introduced
NON_GIT_MANIFEST_REQUIRED NOT RUN: the prescribed Test App is a Git checkout

The requested pre-bind SESSION_AUTHORITY_REQUIRED variant was not provoked: source auto-binding and earlier install/device gates emitted other codes. The foreign-Metro/UDID variant was also NOT RUN; no second Metro or foreign lane was used.

Fresh reporting checks passed: 17 recognized occurrences / six complete-key groups. The Android runner group spans two tools; the main bundle group spans four. Every key, count, tool set, member signature, date and recorded provenance matched the store. Diagnostics did not record recovery. Both shipped host CLIs agreed in JSON and text after normalizing generated-at timestamps only. JSON carries signatures/provenance; their omission from the systemic text section is intentional.

Reporting left 24,193 bytes, mtime 1789384740834568538 ns, hash and directory contents unchanged. The earlier exact-head legacy result is carried forward as instructed: 12 legacy-derived groups / 942 occurrences, without rewriting history. Structured-code/prose precedence, first-content-item selection and the 16 KiB UTF-8 boundaries are unit-verified by existing coverage; no synthetic acceptance harness was added. Real out-of-set codes remained UNKNOWN.

Privacy: no UUID, known device identifier/serial, hostname or local/home path was found in any field of the 19 fresh stored records. All use redaction version 2. The actual Metro refusal contained no UDID, so this is not live proof of the unobserved UDID-bearing variant.

Reverse cleanup passed with cleanupProven=true: owned devices/Metro/transport removed, integrations restored, foreign inventories unchanged. The Android tutorial/launch-to-picker failure merits separate follow-up and is not attributed to this reporting diff. The subsystem-removal proposal was not evaluated.

Evidence: issue-981-pr1034-qa-a8a1/qa2-d4d7de43/, including the standalone report, original sanitized MCP results, both CLI formats and diffs, store-integrity/privacy checks, and cleanup proof.

iOS result after the passing saved action Android diagnostic picker after the failed native replay; not successful app navigation

@Lykhoyda

Lykhoyda commented Sep 14, 2026

Copy link
Copy Markdown
Owner Author

FAILED — independent exact-head QA of #1034 at d4d7de43667e9c00f53d25a608ecdf77c93bc581 against workspace 7a63707af91b8bec20cab76f902a1553985251e6.

A failed native Android journey cannot become a full PASS. This is evidence for the captain, not a merge gate. No approval or merge decision.

The Android first native surface after a fresh owned build/pin/attachOnly was the Expo development-server picker, not the app. Pin exhausted 120s with 0 Hermes targets. The APK does contain EXDevMenuIsOnboardingFinished=true and EXDevMenuShowsAtLaunch=false; there was no Expo tutorial. One public device_deeplink of the same managed URL then showed app onboarding with app-root and a successful pin, after which APP_INSTALL_IDENTITY_CHANGED blocked hideDevMenu and the unchanged qa-observe-screen-match replay (the YAML never ran). Repeat was not reachable. Product source, stopApp, routes, and assertions were not patched.

iOS at this same pair passed: usable Welcome baseline, hide surface:app, strict replay and repeat both passed:true with transport cdp-js / proofDomain=react-tree, route QaAcceptanceResult.

Reporting recheck on this run’s private store: four real codes (BUNDLE_HANDSHAKE_UNAVAILABLE, RUNNER_OWNERSHIP_MISMATCH, SESSION_AUTHORITY_REQUIRED, HANDOFF_NOT_AUTHORIZED). accept_handoff without credentials produced HANDOFF_NOT_AUTHORIZED: session is not a capability-bound recovery contender. SESSION is the active Metro+runner restore guard, not the pre-bind variant (NOT RUN). NON_GIT_MANIFEST_REQUIRED NOT RUN. METRO_ORIGIN_MISMATCH NOT RUN here. Live UDID-bearing Metro variant NOT RUN. Both shipped CLIs agree after normalizing generated timestamps; JSON has signatures/provenance; text omits them. Fresh store and legacy corpus hashes unchanged by reporting. 14 fresh rows redactionVersion: 2, privacy-clean for observed inputs.

Cleanup: owned runner/Metro/transport closed, integration restored, owned simulator and emulator removed. Foreign inventory unchanged (two booted iOS simulators; physical USB phone untouched).

Android first native surface: Expo development-server picker Android after public managed deeplink: app onboarding with app-root iOS usable baseline: app Welcome onboarding iOS result after strict saved-action replay

@Lykhoyda

Copy link
Copy Markdown
Owner Author

FAILED — QA input-correction experiment for #1034 at d4d7de43667e9c00f53d25a608ecdf77c93bc581 against workspace 7a63707af91b8bec20cab76f902a1553985251e6.

Binding the existing Expo Dev Client URL before the owned Android build did not produce a visible app result from the original saved action. This is evidence for the captain, not a merge gate. No approval or merge decision. No product source was patched.

What changed vs prior Android runs. Previous exact-head QA bound the device without a URL, then built, then pinned (0 Hermes targets / 120s; first native surface was the Expo picker). This run called rn_session bind_device with devClientUrl=exp+rn-dev-agent-test://expo-development-client/?url=http%3A%2F%2F10.0.2.2%3A8095 (the reserved session Metro port) before pnpm run android. The signed receipt carried that same devClientUrl. The adapter opened that URL on the owned emulator QA_PR1034_bindfirst. installIdentity stayed verified.

What still failed. pin_dev_client still returned BUNDLE_HANDSHAKE_UNAVAILABLE / CDP_TARGET_AUTHORITY_MISMATCH: expected one target on the exact device, found 0, Metro 8095, 120s deadline. Unchanged qa-observe-screen-match (stopApp: true, assertVisible app-root; YAML sha256 2f3521de6b5a0e6007d82d17fc1bf33d28109df563ae084ece4d8707b78132a1) never executed: cdp_run_action refused RUNNER_OWNERSHIP_MISMATCH after device_snapshot open could not start the Android runner (RN_ANDROID_RUNNER_DOWN, no /health within 30s). Repeat was not reachable. No native screenshot this run.

Picker-first and APP_INSTALL_IDENTITY_CHANGED are the known separate Android launch issue from prior runs. This experiment does not use them as verdict reasoning. APP_INSTALL_IDENTITY_CHANGED did not appear here.

iOS, CLI/reporting, and Desktop were NOT RUN (preserved / out of scope). Foreign iOS simulators and the attached USB phone were not touched. Owned emulator and Metro were removed; package scripts restored to expo run:ios / expo run:android.

The bind-first input correction did not avoid the Android saved-action wall. No product patch is justified from this run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: repeated authority refusals never surface as one systemic issue

2 participants