Skip to content

fix: make durable PR waits callback-driven#335

Draft
JWThewes wants to merge 3 commits into
mainfrom
fix/durable-pr-wait-recovery
Draft

fix: make durable PR waits callback-driven#335
JWThewes wants to merge 3 commits into
mainfrom
fix/durable-pr-wait-recovery

Conversation

@JWThewes

Copy link
Copy Markdown
Contributor

Closes #334

Important

This PR must not be merged until #333 has merged. After #333 merges, this branch must be rebased onto the latest main and the checks rerun before this PR is made ready for review.

Problem

PR-per-unit review used durable timer polling. One production-like execution exhausted Lambda's 3,000-operation limit while waiting for review, leaving process state active after the durable execution had failed.

The affected intent also demonstrated why repair must preserve state: one reviewed unit PR had merged remotely while local state lagged, and another unit's construction stages and draft PR were already complete and reusable.

Approach

  • Replace repeated durable PR polling with one persisted callback per observed provider state.
  • Add sparse DynamoDB maintenance partitions for active executions and parked PR waits.
  • Reconcile GitHub/GitLab state on a one-minute schedule and wake only for merge, closure, branch movement, or queued authenticated feedback.
  • Atomically claim callback delivery so scheduler and feedback wakeups cannot double-resume a run.
  • Detect terminal durable executions before their configured local expiry without overwriting a concurrently completed process row.
  • Reconcile provider truth during repair, validate merged reviewed-head ancestry, preserve successful stages and artifacts, and resume existing draft PRs through the current orchestrator alias.

Recovery verification

The deployed dev recovery was exercised against the failed execution that motivated #334:

  • the prior durable execution was confirmed FAILED with ExecutionLimitExceeded;
  • the merged reviewed head was confirmed reachable from the intent branch and the corresponding unit was marked merged;
  • ten completed construction stage rows remained SUCCEEDED at attempt 0 and no artifacts were archived;
  • the existing draft PR was reconciled in place, promoted to ready, and parked on exactly one persisted callback in the sparse PR_WAITS index.

The review PR was deliberately left unmerged.

Testing

  • npm test - 117 files, 2,331 tests passed
  • focused shared/orchestrator/intents suites - 44 files, 887 tests passed
  • commit hook - formatting, lint, secret scan, production dependency audits, and 561 targeted tests passed
  • affected intents and v2-orchestrator Lambda bundles built successfully
  • terraform fmt -check -recursive terraform

JWThewes added 3 commits July 21, 2026 10:17
Members could not start intents without a personal GitHub/GitLab
connection: the orchestrator resolved the starting user's token and
init-ws failed asynchronously (checkout_failed / intent_branch_push_failed).

Replace per-user runtime auth with project-level source-control bindings:

- Owners/admins bind each repository once, choosing GitHub App (per-repo
  installation discovery) or explicitly confirmed github-oauth /
  gitlab-oauth delegation at step 1 of the create-space flow; the App
  path needs no personal connection at all (repo discovery runs on
  App-JWT-minted metadata-read installation tokens via /github/app/*).
- Bindings hold opaque credential refs in a new DynamoDB table, never
  tokens, and auto-invalidate when the delegator disconnects, loses
  scopes, or leaves the project.
- Intent starts validate bindings up front (409 SOURCE_CONTROL_NOT_READY
  with per-repo reasons) instead of failing mid-run.
- AgentCore runtime is token-free: git operations obtain short-lived
  credentials from a credential-broker lambda (IAM-scoped to the runtime
  role, validates execution liveness) via a throwaway GIT_ASKPASS
  helper; tokens no longer enter invocation payloads or durable history.
- Provider API operations (branches, PRs, issues) route through a
  source-control service lambda; per-provider API proxy routes and the
  platform-wide GitHub auth mode are removed.

Also bumps js-yaml 4.2.0 -> 4.3.0 in the lockfile (GHSA-52cp-r559-cp3m,
pre-existing advisory that now trips the pre-commit audit).

Closes #332
…ng (#332)

- Clear-text logging (js/clear-text-logging, alerts 17-20): error objects in
  the credential-resolution path can carry provider-derived text. Logging
  sites in credential-broker and source-control now emit only allowlisted
  error-code constants via loggableErrorCode() and no longer log
  error.message.
- Polynomial regex (js/polynomial-redos): canonicalRepo trimmed leading and
  trailing slashes with /^\/+|\/+$/g over caller-supplied repo refs; replaced
  with a linear-time scan.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prevent durable PR waits from exhausting execution operation limits

1 participant