Commit 2153b0b
authored
* fix(gl): make /encrypted-blobs recovery parsing schema-strict
Deserialize the /encrypted-blobs 200 body into a typed EncryptedBlobsResponse
instead of poking at a serde_json::Value and defaulting blobs to [] via
unwrap_or_default(). A server schema regression now fails closed (a hard parse
error) rather than silently yielding nothing to recover, mirroring the existing
WithheldPathsResponse convention. The sole caller still does not abort the clone
but now surfaces the error instead of swallowing it into an empty result.
Part of #49.
* fix(gl): paginate Arweave manifest discovery and rank ties by block height
Discovery hard-coded a single first:100 GraphQL query with no pagination, so a
repo with more than 100 encrypted-manifest anchors (one per push) silently lost
recovery coverage for any blob whose most recent seal fell outside that window.
Follow cursor pagination (after + pageInfo) until coverage is complete, deduping
transaction ids and bounding the loop (1000 pages / 10000 ids, degenerate-cursor
guard) so a hostile or buggy gateway cannot grow the downstream per-id fetch loop
without limit. A full page returned without pageInfo, a cap hit, or a failure
after discovery began now warns; a page-1 failure (unconfigured/unreachable
gateway) stays silent as before.
Pagination concatenates pages in gateway order, which made merge_manifests'
equal-timestamp tie-break (previously insertion-order, i.e. accidentally newest
on single-page HEIGHT_DESC) gateway-dependent. Rank candidates by the composite
key (timestamp, Arweave block height, cid): timestamp primary keeps the existing
distinct-timestamp latest-wins; block height breaks equal-timestamp ties by
anchor recency (a not-yet-mined anchor counts as newest), preserving the
pre-pagination behavior; cid is a final deterministic tiebreak for same-block
anchors. Height is parsed defensively so a present-but-unreadable height (e.g. a
gateway stringifying large ints) cannot masquerade as a pending/newest anchor.
Part of #49.
* fix(review): bound edges per Arweave page and cover degenerate-cursor/dedup
Code-review follow-ups on the #49 recovery change:
- Cap edges parsed per GraphQL page at ARWEAVE_PAGE_SIZE so a hostile gateway
returning one oversized page cannot defeat the MAX_TX_IDS budget or flood the
downstream per-id manifest fetch loop (adversarial review).
- Collapse the two identical gateway-failure arms in the discovery loop into a
single warn site, and hoist PAGE_SIZE to a named const tied to the query's
first:100 (maintainability).
- Add tests: per-page edge bound, non-advancing-cursor termination (infinite-loop
guard), and cross-page tx-id dedup fetched exactly once (testing).
Part of #49.
* fix(review): make the degenerate-cursor test assert, and pin the page-size invariant
Round-2 review follow-ups:
- The non-advancing-cursor test bound its mock to a _-prefixed handle with
expect_at_most(3) and never asserted it; mockito defaults assert_on_drop=false,
so the test passed even if the guard were removed. Bind it, tighten to expect(2)
(the guard breaks on the second page, so discovery makes exactly two POSTs), and
assert_async.
- Correct the parse_tx_page docstring: the per-page take() bounds the TxRef vec and
fetch loop, not the response-body deserialization (an unbounded-body concern
shared by every gateway read here).
- Hoist the discovery query to a const and add query_first_matches_page_size so the
query first:N literal and ARWEAVE_PAGE_SIZE are kept equal by a test rather than
a comment.
Part of #49.
1 parent 96fcfdb commit 2153b0b
1 file changed
Lines changed: 838 additions & 64 deletions
0 commit comments