feat(node): add durable replicated KV foundation

[windsornguyen]

Summary

Ships Cloud9's first durable replicated database surface and aligns the public specifications with the product thesis: an open-source Spanner and MLIR for databases.

The branch adds a bounded segmented WAL, a restart-safe Raft node, a ConnectRPC key-value API, Jepsen coverage, and fail-closed CI checks. It also replaces obsolete SQL/KV-only and HLC-fallback language with the multi-dialect IR and capability-gated TrueTime architecture.

Motivation

main contains the reviewed Raft correctness fixes but not the durable node, database API, WAL, or Jepsen harness built on top of them. Keeping those commits outside main leaves the repository without the runnable system described by its current tests.

The existing specifications also claimed completed features and an HLC fallback that the product no longer intends to support. The rewritten specifications separate implemented behavior from target architecture and make unsupported consistency modes explicit errors.

No linked issue.

Changes

• Add a bounded segmented WAL with checksummed recovery, exclusive writers, capacity limits, and poisoned-handle semantics.
• Add a persistent Raft runtime and ConnectRPC key-value API with idempotent sessions, bounded requests, authenticated peer messages, restart recovery, and three-node failover tests.
• Add a Jepsen db/DB harness, failure-detection tests, and CI jobs that reject client exceptions and false-green recovery.
• Define Cloud9 as an MLIR for SQL, key-value, document, object, and analytical dialects with specialized physical engines.
• Define capability-gated bounded time, AWS ClockBound requirements, commit-wait, MVCC, distributed transactions, placement, consensus, and projection invariants.
• Document the newly public node and WAL APIs and enforce missing_docs for those crates.

Testing

• cargo test --workspace passes
• Manual testing (the remote multi-host Jepsen workload was not rerun for this delivery commit)

cargo fmt --all -- --check
cargo clippy --workspace --all-targets --locked -- -D warnings
cargo test --workspace --locked
cargo doc --workspace --no-deps --locked
RUSTDOCFLAGS='-D missing-docs' cargo doc -p cloud9-node -p cloud9-wal --no-deps --locked
(cd jepsen && env -u CLASSPATH lein check)
(cd jepsen && env -u CLASSPATH lein test)
typos README.md CONTRIBUTING.md docs/README.md spec jepsen/README.md

Jepsen unit result: 9 tests, 32 assertions, 0 failures, 0 errors.

Security Checklist

• No secrets/keys hardcoded or logged; the checked-in example key is explicitly non-secret test material
• Input validation on all untrusted data
• Constant-time comparison for secrets/signatures
• Error messages don't leak sensitive info
• No integer overflow in externally sized arithmetic; WAL arithmetic is checked and protocol inputs are bounded
• No unbounded allocations from untrusted input; ConnectRPC, Raft messages, responses, commands, and WAL recovery are bounded

Code Quality

• cargo fmt --all applied
• cargo clippy --workspace passes with no warnings
• New handwritten public items have rustdoc comments; generated protocol types are source-documented in protobuf
• No unwrap() in non-test code (use expect() with context or ?)

Breaking Changes

• No breaking changes
• Breaking change (describe below):

Documentation

• README updated (if needed)
• Rustdoc comments added/updated
• docs/ updated (if needed)

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.