feat(remote): add local fallback when api.wiki.codes is unreachable - #535
feat(remote): add local fallback when api.wiki.codes is unreachable#535idea404 wants to merge 4 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a8d9f0f36b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (std.mem.eql(u8, action, "tree")) { | ||
| handleTree(alloc, out, ctx.explorer); | ||
| } else if (std.mem.eql(u8, action, "outline")) { | ||
| handleOutline(alloc, args, out, ctx.explorer); |
There was a problem hiding this comment.
When api.wiki.codes is unavailable, a documented remote outline call such as {"action":"outline","query":"src/foo.zig"} reaches this fallback with only query populated, but handleOutline reads the file path from path and will return error: missing 'path' argument. Normalize query into the local handler's path argument before dispatching so the advertised fallback actually works for valid remote outline requests.
Useful? React with 👍 / 👎.
| } else if (std.mem.eql(u8, action, "search")) { | ||
| handleSearch(alloc, args, out, ctx.explorer); | ||
| } else if (std.mem.eql(u8, action, "read")) { | ||
| handleRead(io, alloc, args, out, ctx.explorer); |
There was a problem hiding this comment.
For action=read, the remote API schema and forwarding logic use a lines string like "10-60", but this fallback passes the original args directly to handleRead, which only honors line_start/line_end. When the upstream is down and a caller requests a range, the fallback ignores it and returns the entire file, which can produce much larger responses than requested; parse/translate lines before calling the local read handler.
Useful? React with 👍 / 👎.
| } else if (std.mem.eql(u8, action, "read")) { | ||
| handleRead(io, alloc, args, out, ctx.explorer); | ||
| } else if (std.mem.eql(u8, action, "symbol")) { | ||
| handleSymbol(alloc, args, out, ctx.explorer); |
There was a problem hiding this comment.
A valid remote symbol request supplies the identifier as query (and the code above validates that), but this fallback calls handleSymbol unchanged even though the local handler requires name. In the upstream-down case, {"action":"symbol","query":"Foo"} therefore falls back to error: missing 'name' argument instead of returning symbols; translate query to name for the local fallback path.
Useful? React with 👍 / 👎.
|
Hi @idea404 — thank you for tackling this! 🙏 The api.wiki.codes outages (#508/#534) are a real pain, and a local clone-and-index fallback is a genuinely good answer. 1. Could you retarget the base 2. Verified: it compiles and the full suite stays green, no regressions. The security-sensitive parts are well done — A few things before we can merge:
All very doable and we're happy to help. If we don't hear back in ~1 day, we may pick these up in a follow-up building on your work, with you credited (author/co-author + release notes). Thanks again! 🙏 |
When the remote API fails, automatically fall back to a local shallow clone + index for tree, outline, search, read, symbol, and deps actions. - Add src/remote_cache.zig for cache management (~/.codedb/remote-cache/) - Modify handleRemote to try remote first, then local fallback - Add clean_cache action to purge cached repos - Update tool schema to document fallback behavior Closes justrach#534
- Enforce TTL: ensureCached uses isCacheFresh, evicts stale entries - Enforce MAX_CACHED_REPOS: evictIfNeeded runs LRU eviction before clone - Clone timeout: git -c http.lowSpeedLimit/Time prevents hanging clones - Size cap: --single-branch --no-tags minimizes clone size - Race fix: clone to temp dir then atomic rename prevents concurrent clobber - Add tests for TTL enforcement and clone-to-temp-then-rename
a8d9f0f to
774bd1e
Compare
|
Thanks for the thorough review! All four points addressed in 774bd1e:
|
Address Codex bot review feedback: - outline: map query → path before calling handleOutline - symbol: map query → name before calling handleSymbol - read: parse lines '10-60' → line_start/line_end before calling handleRead Add translateRemoteArgs() function and test coverage.
|
Also addressed the Codex bot suggestions — |
Resolve test_mcp.zig conflict by keeping issue-534 remote-cache tests alongside upstream release-branch tests (justrach#570–justrach#592). Co-authored-by: Cursor <cursoragent@cursor.com>
|
@justrach quick bump |
api.wiki.codes has been down 64+ days (HTTP 530, dead cloudflared tunnel) and handleRemote had a single hardcoded endpoint with no fallback. idea404's PR adds src/remote_cache.zig (shallow clone to temp then rename, isCacheFresh TTL, cleanStaleEntries/evictIfNeeded) and wires it into handleRemote: the network path is still tried first, and only a transport error or a non-2xx response falls through to the local clone-and-index cache. Non-2xx errors the fallback cannot serve still get the #508 appendRemoteErrorHint diagnostics. Adapted while merging onto the MCP-2026-07-28 branch: - remote_cache.zig read cio.CaptureResult.Term.Exited without checking the tag. A git or indexer process killed by a signal (the http.lowSpeedLimit abort path makes this reachable) would panic on illegal union access. Now uses the repo convention `term != .Exited or term.Exited != 0`, matching nuke.zig and update.zig. - handleRemoteFallback now prefixes a provenance note. Local results are codedb's text format rather than wiki JSON and can lag the live repo by up to the cache TTL, so they are labelled rather than passed off as a live remote answer. - Added test_mcp.zig "issue-508: remote fallback cache dir is an indexable root". getRemoteCacheDir's output is fed straight into ProjectCache.get, which rejects any path root_policy refuses, so a cache dir under /tmp or equal to $HOME would make every fallback silently return false with no diagnostic. No std.Io API drift needed adapting; the PR's io/allocator threading already matches current usage. zig build test: 23/23 steps, 534 pass, 4 skip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TD448QW4vLZ2UqPic6gR5a
…epo role codedb is deliberately local-only. The codedb_remote tool (api.wiki.codes) is removed entirely: Tool enum, tools/list schema, dispatch, handleRemote, the curl-based fetchRemote, wiki slug helpers, appendRemoteErrorHint (#508), AND this branch's PR #535 local-clone fallback — src/remote_cache.zig, handleRemoteFallback, translateRemoteArgs, the lib.zig export, and the issue-508/issue-534 tests. Docs retargeted: README/architecture/quickstart tool tables drop the row, hooks-labs guard examples now demo codedb_search + max_results, and the public-repo story points at the installer-registered DeepWiki server (mcp__deepwiki__* for hook matchers). Website installer copies re-synced to keep the byte-identity test green. BREAKING CHANGE: the codedb_remote MCP tool no longer exists; use the DeepWiki server (read_wiki_structure / read_wiki_contents / ask_question) for public-repo questions. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
2 participants
Closes #534
When api.wiki.codes is unreachable, automatically fall back to a local shallow clone + index for tree, outline, search, read, symbol, and deps actions.
Changes:
src/remote_cache.zigmodule for cache management (~/.codedb/remote-cache/)handleRemoteto try remote first, then local fallback on failureclean_cacheaction to purge cached reposFallback chain:
api.wiki.codes(existing behavior)git clone --depth=1+codedb snapshot)Verification:
zig build— compiles cleanzig build test— 709/710 pass (1 pre-existing flaky perf test)