Cache-server concurrent writers: threaded requests, first-writer-wins verdicts

[davet47]

Second item of the v0.5 hosted-store theme (ROADMAP: "Concurrent writers — the cache server is single-threaded-serialised; real teams need atomic verdict/blob writes under concurrency (CAS or equivalent), not politeness").

Threading — python -m hashloom.cache_server is now a ThreadingHTTPServer:

• Requests (network IO, JSON parsing, auth) run concurrently; one threading.Lock serialises the single SQLite connection, so every db write stays an atomic single-statement commit — the exact semantics the store's check_same_thread=False comment promises, now enforced by the lock rather than by single-threading. (An explicit lock rather than trusting SQLite serialized mode: sqlite3.threadsafety introspection is unreliable on the 3.10 floor.)
• daemon_threads = False, so server_close() joins in-flight request threads before serve() closes the store — the shutdown use-after-close that threading would otherwise introduce never exists.
• Unexpected handler errors now return structured JSON 500s ({"error": {"code": "internal", ...}}) instead of a bare stack trace swallowed by the silenced request log — the _respond discipline, extended to the cache server.

First-writer-wins verdicts — the "CAS or equivalent", resolved by observing that the verification key is the compare:

• A verdict row is keyed by the verification key (sha256 over contract, impl, test source, toolchain identity, and the transitive dep-hash closure), so two publishes for one key are independent runs of a bit-identical closure — status is always pass (the server rejects the rest) and summaries are empty on green. Last-writer-wins buys nothing.
• Worse, the old ON CONFLICT DO UPDATE refreshed ran_at (a verdict looking fresher than the run that produced it) and reset stale=0 — which becomes a served-stale-green race the moment cross-graph invalidation (theme item 3) can mark shared rows stale.
• New CacheStore(SqliteStore) subclass in cache_server.py: ON CONFLICT(key) DO NOTHING. Duplicates are no-ops. Local stores keep their overwrite behavior; the Store protocol and its contract are untouched by construction.

Deferred, documented: WAL journal mode + per-thread connections + bounded busy-retry — the throughput path if a team ever saturates the lock (docs/hosted-store.md item 4 names it).

Tests (4 new in test_remote_store.py, fixtures now exercise CacheStore): deterministic first-writer-wins proof (duplicate leaves summary and ran_at untouched); the soundness pin — a duplicate publish cannot un-stale a row mark_stale just marked; a 16-worker mixed publish/read storm with no 5xx, every key landing exactly once, and one whole winner on the contested key; structured-500 with no Traceback in the body.

Live smoke: 250 parallel requests against the real server process — status codes exactly as designed (100×204, 100×200, 50×403, zero 5xx), 51 rows for 51 keys, one whole winner on the 50-way contested key.

Full suite 243 passed, coverage 90.49% (gate 85), benchmark ≥5x. Wire protocol unchanged — clients need no update.

Next theme items: cross-graph invalidation, then the dependency set in the verification key.

🤖 Generated with Claude Code