[audit-workflows] Agentic Workflow Audit — 2026-05-30 (56 runs, 91.1% success, 5 distinct failures)

[github-actions[bot]]

Daily audit of the last 24h of agentic workflow runs in github/gh-aw. 56 completed runs, 91.1% success (51/56) — a healthy full-window result. The 5 failures are all distinct one-off classes (no systemic regression), and yesterday's critical cache-memory git bug did not recur. The standout concerns are a token-budget 429 on Linter Miner (ties to open #35661) and the Failure Investigator itself timing out at 60 min / $7.15 after finding zero failures.

Summary

Metric	Value
Completed runs	56 (51 ✅ / 5 ❌) + 1 in-progress (this audit)
Success rate	91.1% (full window)
Total tokens / effective	49.9M / 366.3M
Measured cost	~$24.55 (claude-engine only; copilot/codex/gemini cost not reported)
Turns / action-minutes	1,100 / 472
Engines	copilot 39, claude 12, codex 3, antigravity 1, gemini 1, pi 1
Firewall block rate	16.6% (706/4,253) — healthy
Agent-logged errors	0 (all 5 failures were step-level)

Critical / Actionable Findings

1. 🔴 Token-budget 429 — Linter Miner (#35661 recurrence) · §26690626184
CAPIError: 429 Maximum effective tokens exceeded (25.13M / 25M). After 59 turns the run hit the 25M effective-token cap, then burned 4 --continue retries (each re-hitting the cap, ~92s apart) before giving up exitCode=1. Continuation cannot recover a hard budget cap. Fix: reduce Linter Miner's per-run scope (chunk the linter aggregation) and make the harness fail-fast on the budget-429 signature instead of retrying. Maps to the open token-budget issue #35661 — which is therefore not resolved.

2. 🟠 Failure Investigator timed out at 60 min / $7.15 (most expensive run) · §26692427111
Ironically the day's only "100% failure-rate" workflow. It concluded "zero failures in the 6h window" early (~19:11 UTC) yet kept running the full 60 minutes (7.8M tokens, $7.15) before hitting the Claude CLI step timeout. A clean window should converge in minutes. Fix: investigate why it doesn't early-exit on a no-failure window; confirm the 60-min ceiling is intended.

3. 🟠 add_comment target="*" with no number — Contribution Check (REACTIVATED) · §26689948645
Agent succeeded, but the safe_outputs step hard-failed the whole job: Target is "*" but no item_number/... specified in add_comment item — discarding a successful create_issue. On a schedule event there's no triggering PR, so a target=* comment is structurally unsatisfiable, yet the model emitted one despite an explicit prompt guardrail. Fix: (a) make Process Safe Outputs skip-with-warning on one bad item when ≥1 succeeded; (b) validate target=* at the MCP emit boundary so the agent self-corrects in-loop. (Also: create_issue with temporary_id registered 0 temp ids, breaking add_labels chaining.) Was in "watch" — reactivated on this full-window recurrence.

4 & 5 — golangci-lint download flake + Smoke Claude timeout (lower priority)

4. 🟡 golangci-lint non-gzip download — PR Sous Chef · §26690499578
Failed before the agent ran (0 turns): make install-golangci-lint piped a non-gzip release body into tar (exit 2) in the shared Install development dependencies step. Transient, but exposes every workflow using that setup. Fix: download to temp, verify HTTP 200 + gzip magic bytes, retry w/ backoff, pin a checksum (Makefile ~403). The Failure Investigator already filed a durable tracking issue for this.

5. 🟡 Smoke Claude 10-min CLI timeout + MCP EOF — monitor · §26690105542
Execute Claude Code CLI timed out after 10 minutes with 21× MCP error 0: client is closing: EOF. This was run_attempt 2 on the now-merged firewall/gateway-bump PR #35973; awf-squid was Healthy (not #35780). PR-branch smoke artifact — monitor whether the MCP-EOF cluster recurs on main after the gateway bump.

✅ Resolved / Recovered

• cache-memory git setup fatal (yesterday's critical, failed 2/2 workflows) — did not recur. Both Chaos PR Bundle Fuzzer (§26694560325) and Copilot PR Prompt Pattern Analysis (§26694706324) passed today.
• Changeset Generator — succeeded again on gpt-5.4 (2nd clean cycle after the 9-day alpha-routing 404). Closed.
• Copilot CLI 1.0.55 — 39 runs, 0 ENOENT / anthropic-beta / model-not-supported errors.

📊 Trend Charts (30-day window)

Success rate recovered to 91.1%, above the recent band and well clear of the 05-23 dip (41.6%). The 12-day trend is stable in the high-80s/low-90s; today's 5 failures are independent one-offs rather than a clustered regression.

Daily tokens (49.9M) and claude-measured cost ($24.55) sit near the 30-day average; the 3-day moving average is flat. Cost is understated — copilot/codex/gemini runs report EstimatedCost=0, so the true spend is higher than the claude-only line. Two single runs dominate: Failure Investigator ($7.15) and Daily Safe Output Tool Optimizer ($5.24).

🌐 Firewall

Healthy at 16.6% (down from 24.9% on 05-27). Blocks are dominated by (unknown) SNI (555) and Google/Chrome browser telemetry (www.google.com, content-autofill, accounts.google.com, safebrowsing) — none caused a run failure. Highest pressure: Smoke Copilot 121/304, Linter Miner 63/277, Smoke Antigravity 10/12 (by-design).

Recommendations (priority order)

1. (High) Cut Linter Miner's effective-token footprint and fail-fast on budget-429 ([aw-failures] Token-budget exhaustion (25M effective-tokens cap) recurring across 6+ scheduled workflows — 2026-05-29 02:00–07:32 UTC #35661).
2. (High) Safe-output partial-failure tolerance + emit-time target=* validation (Contribution Check class).
3. (Med) Investigate the Failure Investigator's missing early-exit on clean windows (60-min/$7.15 waste).
4. (Med) Harden the golangci-lint download (validate/retry/checksum) — shared-step flake.
5. (Med) Monitor Smoke Claude MCP-EOF on main; re-verify Avenger max-turns when it next runs.

References: §26690626184 · §26692427111 · §26689948645

Generated by 🔍 Agentic Workflow Audit Agent · opus48 4.1M · ◷

• expires on May 31, 2026, 9:46 PM UTC

[github-actions[bot]]

💥 KA-POW! 🦸 The Smoke Test Agent ZOOMED through here! WHOOSH! ⚡ All systems checked, all gizmos GO! Claude engine nominal — THWIP! 🕸️ Stay heroic, gh-aw! 🚀

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• contentautofill.googleapis.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

💥 [THE END] — Illustrated by Smoke Claude · opus48 887.5K · ◷

[github-actions[bot]]

Smoke goblin was here.
Tiny club. Big test. Repo still standing.

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• contentautofill.googleapis.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · gpt54 12.3M · ◷

[github-actions[bot]]

Smoke goblin was here.
Tiny club. Big test. Repo still standing.

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• contentautofill.googleapis.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · gpt54 12.3M · ◷

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #36085.