Commit 8132ef6
committed
fix(ci,4-4): Story 6-7 review patches + Story 4-4 hash stability + P15
Lands two independent threads in one commit:
(1) Story 6-7 post-merge code review fixes — hardens the CI workflow,
removes a dangerous gitleaks allowlist, modernizes cargo-deny, and
updates the ADR + CONTRIBUTING.md to match.
(2) Story 4-4 Tasks 5 + 6 — closes the byte-exact hash stability gap
(refined P21) and the cache-seeding consistency gap (refined P15).
With this commit, 4-4 has only Task 8 (the 6-6 regression test)
still gating it.
----------------------------------------------------------------------
(1) STORY 6-7 REVIEW FIXES
----------------------------------------------------------------------
.github/workflows/ci.yml:
- Pin every `dtolnay/rust-toolchain@master` to `@1.88.0`. The action's
own README discourages `@master` because it is a moving target and a
supply-chain risk. This includes the toolchain-matrix split below.
- Remove the always-true `fuzz-smoke` soft gate. The
`if: hashFiles('fuzz/Cargo.toml') != ''` guard was permanently true
the moment 6-7 merged because Story 6-4 had already landed
`fuzz/Cargo.toml`. Now `fuzz-smoke` is a hard gate from Day 1.
- Coverage job: drop `needs: unit` (no correctness benefit, just
serializes two builds), drop `fetch-depth: 0` (delta gate is
descoped — see ADR-0002 D3), use a separate `solarix-ci-coverage`
cache key so `RUSTFLAGS=-Cinstrument-coverage` artifacts don't
thrash the regular cache, switch to `taiki-e/install-action@v2`
for cargo-llvm-cov (saves ~3–5 min cold install), and add
`if-no-files-found: error` so a silent `cargo llvm-cov` no-op
fails the upload step instead of warning.
- Integration job: drop the unused `SOLANA_RPC_URL: devnet` env var
with an explanatory comment. The integration tests today
(bootstrap_test.rs, registration_test.rs) don't hit the RPC, and
devnet is too rate-limited / flaky for the per-PR critical path.
- Security job: switch to `taiki-e/install-action@v2` for cargo-audit
+ cargo-deny (cached binaries), and run `cargo deny check
advisories bans sources` explicitly instead of the bare `cargo deny
check`. The bare form would also run `licenses` which is fail-soft
for Sprint 5 (see ADR-0002 D5).
- Docker-smoke: bump `curl --retry 12 --retry-delay 5` to `--retry 30
--retry-delay 10 --retry-all-errors --connect-timeout 5` so a brief
404 during axum route mounting doesn't fail the wait step. Scan
EVERY captured log line for valid JSON (not just `head -5`) so a
pretty-format regression that lands after the startup banner is
still caught. Fail loudly when zero log lines are captured (silent
container crash). Drop `|| true` from the cleanup step so a real
teardown failure isn't masked. `if: always()` already runs cleanup
on both branches.
- Toolchain matrix: split the `strategy.matrix: [stable, beta]` into
two explicit `toolchain-stable` and `toolchain-beta` jobs. GitHub
Actions does not allow expressions in `uses:`, so a matrix forces
the action ref back to `@master`. Two explicit jobs trade YAML
lines for the elimination of the supply-chain hole. Beta job uses
`continue-on-error: true` to surface upcoming breakages as warnings.
.github/workflows/nightly.yml:
- Permissions: `pull-requests: write` → `issues: write`. The
notification step calls `github.rest.issues.createComment`, which
is on the issues API. The previous permission set would let the
workflow load but the createComment call would 403 at runtime.
- Move `continue-on-error` from the JOB level to the test STEP. With
it on the job, the overall job conclusion was forced to success,
which made `if: failure()` on the notification step never trigger.
The new pattern: `continue-on-error: true` on the run step, and
the notification step branches on `steps.run_smoke.outcome ==
'failure'` so it actually fires when the test fails.
- Belt-and-braces guard: a `feature_check` step grep's Cargo.toml for
`^mainnet-smoke\b` and short-circuits the rest of the job via
`if:` outputs if the cargo feature isn't declared yet. Pairs with
the existing `hashFiles('tests/mainnet_smoke.rs')` job-level guard
for the case where the test file and the cargo feature land in
separate PRs.
.gitleaks.toml:
- Remove the global `regexes = ['''[1-9A-HJ-NP-Za-km-z]{32,44}''']`
allowlist. The intent was to allowlist Solana pubkeys, but a regex
at the top-level [allowlist] block applies to every detected
secret across the whole repo — and base58 of length 32–44 also
matches the base58 encoding of many real secrets (Stripe keys,
GCP service-account fingerprints, ECDSA signatures). Future regex
allowlists must live under a specific [[rules]] block. Add a
comment explaining the gotcha so the next contributor doesn't
re-introduce it.
- Drop the `*.md$` path allowlist. It was too broad — a real secret
pasted into any `.md` file outside `_bmad-output/` would be missed.
Path allowlist now scopes to `tests/fixtures/`, `_bmad-output/`,
and `docs/research/` only.
deny.toml:
- Add `version = 2` to both `[advisories]` and `[licenses]`.
Without it, key semantics drift between cargo-deny releases (the
legacy `vulnerability/unmaintained/notice/unsound` knobs were
removed in v2 in favor of "any advisory match is an error unless
ignored"). v2 semantics are the only reason the
`RUSTSEC-2025-0012` (backoff) advisory that motivated this whole
story would actually fail the build.
- Update the section header comment to make it explicit that the
CI security job runs `cargo deny check advisories bans sources`
(NOT licenses), so the [licenses] block below is documentation
only for Sprint 5. Revisit post-bounty.
CONTRIBUTING.md:
- Security row matches the new `cargo deny check advisories bans
sources` invocation.
- Add `rustup toolchain install 1.88` + `rustup component add` to
the local prerequisites section so contributors don't trip over
a missing MSRV toolchain.
docs/adr/0002-ci-pipeline.md:
- Drop `fuzz-smoke` from the soft-gate table (now a hard gate).
- Update the table's note column to reflect the belt-and-braces
feature_check guard added to nightly.yml.
- Reflect the toolchain matrix split (stable + beta as two explicit
jobs) in the hard-gate list.
README.md:
- Add an MSRV badge linking to `rust-toolchain.toml`.
- Fix CI badge URL.
----------------------------------------------------------------------
(2) STORY 4-4 TASKS 5 + 6 (HASH STABILITY + CACHE SEEDING CONSISTENCY)
----------------------------------------------------------------------
src/idl/mod.rs:
- `CachedIdl` gains a `raw_json: String` field that holds the ORIGINAL
fetched/uploaded JSON bytes — not a re-serialization of the parsed
`Idl`. This is what `idl_hash` was computed from and what gets
persisted into `programs.idl_json`. Holding the raw bytes is what
gives Story 4.4 AC5 (hash stability) its byte-exact guarantee:
`compute_idl_hash(raw_json) == hash`. Re-serializing through
`serde_json::to_string(&idl)` would silently drop fields that are
not modeled by `anchor_lang_idl_spec::Idl` and shuffle Option
None-vs-absent representations, breaking the round trip for any
future feature that re-hashes persisted bytes (e.g., on-chain IDL
drift detection).
- `upload_idl` and `insert_fetched_idl` both wire the raw bytes into
the new field. Doc comments cross-reference Story 4.4 AC5.
src/registry.rs:
- `RegistrationData` gains an `idl_json: String` field that carries
the raw bytes from `CachedIdl::raw_json` through to
`commit_registration`. The previous implementation called
`serde_json::to_string(&idl)` inside `commit_registration` —
doc-comment now explains why that was wrong and why the raw bytes
win.
- New `mark_program_error(pool, program_id, error_message)` helper:
flips `programs.status = 'error'`, stashes the failure message in
`indexer_state.error_message`, and commits both updates in a single
transaction. Returns a boxed Send future for the same reason as
`update_program_status` — keeps the in-flight Executor reference
from leaking through the opaque return type and breaking Send
inference at the caller's await.
- Two new unit tests:
* `test_mark_program_error_future_is_send` (Send-safety compile-
time check, follows the AC9 pattern)
* `registration_data_idl_json_hashes_to_idl_hash` — pin the byte-
exact invariant. Uses deliberately unusual whitespace + key
order in the input JSON, then asserts both that `data.idl_json`
matches the input verbatim AND that re-serializing the parsed
`Idl` produces DIFFERENT bytes (so the test would actually fail
if `commit_registration` ever silently re-serialized).
src/main.rs:
- Cache seeding loop: now reads `p.idl_json` (the raw bytes from the
DB) instead of computing `serde_json::to_string(&p.idl)`. The bytes
go straight into `IdlManager::insert_fetched_idl` so the in-memory
cache holds exactly what's on disk.
- Per-program success tracking: collects a `Vec<bool>` of seed
outcomes inside the registry write lock, then drops the lock before
any DB await. Programs whose seeding failed are dropped from the
`programs_to_start` list AND get their `programs.status` flipped to
`'error'` via the new `mark_program_error` helper. This is Story 4.4
Task 6 (refined P15): keeps the API consistent so an operator never
sees a 200 from `/api/programs/{id}` alongside 404s from
`/api/programs/{id}/instructions/{name}`.
- `StartupProgram` gains an `idl_json` field. `query_registered_programs`
populates it from the same `idl_json` column it already reads.
src/pipeline/mod.rs:
- `test_backfill_progress_eta` strengthened. The previous version was
weakened in a clippy cleanup pass to "just confirm the call path
doesn't panic" because the old assertion (`eta.as_secs() >= 0`)
triggered `clippy::absurd_extreme_comparisons`. The new version
asserts the actual contract: a freshly-constructed BackfillProgress
with elapsed time < 1ms returns `Duration::from_secs(0)` (zero-rate
sentinel), and a finished progress (start == end) also returns
`Duration::from_secs(0)` regardless of rate. A regression that
swapped the sentinel for `Duration::MAX` or panicked on
division-by-zero would now fail the test.
tests/registration_test.rs:
- New `mark_program_error_transitions_status_and_records_message`
integration test (#[ignore], requires running PostgreSQL): seeds a
successful registration, calls `mark_program_error`, asserts
`programs.status` flipped to 'error' AND
`indexer_state.{status, error_message}` reflect the failure.
- New `idl_json_persisted_bytes_are_byte_exact` integration test:
end-to-end byte preservation. Uploads JSON with deliberately
unusual whitespace, calls `commit_registration`, reads the
persisted bytes back via `SELECT idl_json FROM programs`, and
asserts byte equality + hash equality. This is the integration-
level pin for Story 4.4 AC5.1 parent 9fae115 commit 8132ef6
12 files changed
Lines changed: 580 additions & 129 deletions
File tree
- .github/workflows
- docs/adr
- src
- idl
- pipeline
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
35 | | - | |
| 35 | + | |
36 | 36 | | |
37 | | - | |
38 | 37 | | |
39 | 38 | | |
40 | 39 | | |
| |||
54 | 53 | | |
55 | 54 | | |
56 | 55 | | |
57 | | - | |
58 | | - | |
59 | | - | |
| 56 | + | |
60 | 57 | | |
61 | 58 | | |
62 | 59 | | |
| |||
84 | 81 | | |
85 | 82 | | |
86 | 83 | | |
87 | | - | |
88 | 84 | | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
89 | 92 | | |
90 | 93 | | |
91 | 94 | | |
92 | 95 | | |
93 | 96 | | |
94 | | - | |
95 | | - | |
96 | | - | |
| 97 | + | |
97 | 98 | | |
98 | 99 | | |
99 | 100 | | |
| |||
116 | 117 | | |
117 | 118 | | |
118 | 119 | | |
119 | | - | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
120 | 124 | | |
121 | 125 | | |
122 | 126 | | |
123 | | - | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
124 | 131 | | |
125 | | - | |
| 132 | + | |
126 | 133 | | |
127 | | - | |
128 | 134 | | |
129 | 135 | | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
130 | 141 | | |
131 | 142 | | |
132 | | - | |
133 | | - | |
134 | | - | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
135 | 151 | | |
136 | 152 | | |
137 | 153 | | |
| |||
142 | 158 | | |
143 | 159 | | |
144 | 160 | | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
145 | 164 | | |
146 | 165 | | |
147 | 166 | | |
148 | 167 | | |
149 | 168 | | |
150 | | - | |
151 | | - | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
152 | 175 | | |
153 | 176 | | |
154 | 177 | | |
| |||
159 | 182 | | |
160 | 183 | | |
161 | 184 | | |
162 | | - | |
163 | | - | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
164 | 189 | | |
165 | 190 | | |
166 | 191 | | |
| |||
175 | 200 | | |
176 | 201 | | |
177 | 202 | | |
178 | | - | |
179 | | - | |
180 | | - | |
| 203 | + | |
181 | 204 | | |
182 | 205 | | |
183 | 206 | | |
184 | 207 | | |
185 | | - | |
186 | | - | |
187 | | - | |
188 | | - | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
189 | 212 | | |
190 | 213 | | |
191 | | - | |
192 | | - | |
| 214 | + | |
| 215 | + | |
193 | 216 | | |
194 | 217 | | |
195 | 218 | | |
| |||
213 | 236 | | |
214 | 237 | | |
215 | 238 | | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
216 | 245 | | |
217 | | - | |
| 246 | + | |
| 247 | + | |
218 | 248 | | |
219 | 249 | | |
220 | 250 | | |
| |||
223 | 253 | | |
224 | 254 | | |
225 | 255 | | |
226 | | - | |
| 256 | + | |
227 | 257 | | |
228 | 258 | | |
229 | 259 | | |
230 | | - | |
231 | | - | |
| 260 | + | |
232 | 261 | | |
233 | 262 | | |
234 | 263 | | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
235 | 268 | | |
236 | 269 | | |
237 | | - | |
238 | | - | |
239 | | - | |
240 | | - | |
241 | | - | |
242 | | - | |
243 | | - | |
244 | | - | |
245 | | - | |
246 | | - | |
247 | | - | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
248 | 287 | | |
249 | 288 | | |
250 | 289 | | |
| |||
263 | 302 | | |
264 | 303 | | |
265 | 304 | | |
| 305 | + | |
266 | 306 | | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
267 | 312 | | |
268 | | - | |
| 313 | + | |
269 | 314 | | |
270 | 315 | | |
271 | 316 | | |
| |||
276 | 321 | | |
277 | 322 | | |
278 | 323 | | |
279 | | - | |
280 | | - | |
281 | | - | |
| 324 | + | |
282 | 325 | | |
283 | 326 | | |
284 | 327 | | |
285 | 328 | | |
286 | 329 | | |
287 | 330 | | |
288 | 331 | | |
289 | | - | |
290 | | - | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
291 | 341 | | |
292 | 342 | | |
293 | | - | |
294 | | - | |
295 | | - | |
296 | | - | |
297 | | - | |
298 | 343 | | |
299 | 344 | | |
300 | 345 | | |
301 | 346 | | |
302 | | - | |
303 | | - | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
304 | 351 | | |
305 | | - | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
306 | 374 | | |
307 | 375 | | |
308 | 376 | | |
309 | | - | |
| 377 | + | |
310 | 378 | | |
311 | | - | |
312 | | - | |
313 | | - | |
314 | 379 | | |
315 | | - | |
| 380 | + | |
316 | 381 | | |
0 commit comments