Status: CURRENT (forward-looking reference). Written against the full target design, so some component counts (storage layers, planner modes, channels) describe more than the seeded demo ships today. The scaling breakpoints, cost math, and per-tier upgrade paths are the live reference for the enterprise-scale roadmap (see README → Roadmap and the scale review in
architecture.md). For what runs today, start witharchitecture.md.
Date: 2026-05-21 Purpose: the honest "is it perfect?" pass. Concrete numbers per dimension across five corpus scales (10K → 100M docs); honest weaknesses identified; upgrade path stated for each scale ceiling. Methodology: compose published numbers (pgvector HNSW behavior, ParadeDB pg_search benchmarks, Cohere Rerank latency, Gemini Flash pricing) with the system's design points; identify where each dimension breaks and what we'd do about it.
| Dimension | Verdict | Where it breaks | Cost to fix |
|---|---|---|---|
| Scalable | ✓ to 1M docs out-of-the-box · ✓ to 10M with documented graduation · ⚠ beyond | PG single-instance + HNSW ceiling at ~50M chunks (≈10M docs); concurrent ingest/query I/O contention | Vector graduation (Turbopuffer/Qdrant, ~1-day swap behind adapter) + PgBouncer + 2 read replicas |
| Optimal | ✓ for our explicit use case · ✗ if all you need is Q&A over a small fixed set | Complexity cost: 14 layers + 12 planner modes + 10 retrieval channels = real engineering surface area | None — complexity is the explicit trade for the capability envelope. Acknowledged in the writeup. |
| Fast | ✓ 3–5s for typical queries (with streaming, perceived ~1.5s) · ⚠ 18–22s on agentic deep-research opt-in · ✓ aggregation queries ~3–5s | Multi-hop with IRCoT escalation hits ~6s (we capped at 2 hops); B2 agentic mode is 15–30s by design (opt-in only) | Streaming (already in §6) + IRCoT cap + per-stage progress UI (already in chat plan inspector) |
| Cost-effective | ✓ per-query $0.005–0.01 typical · ✓ per-doc ingest $0.06 · ⚠ per-corpus at Reliance scale ($600K ingest at 10M docs) | The L2b open-vocabulary extraction is +50% over fixed-type-list extraction (~$1,500 at 100K-doc scale) | None — the +$1,500 is the cost of "schema emerges from data" honestly. Worth it. |
Net: the architecture is near-optimal for the explicit problem we set (domain-agnostic enterprise KB, heterogeneous docs, audit-grade citations, schema-emerges-from-data). It is deliberately suboptimal for adjacent problems (live source sync, permissions-aware multi-tenancy, agentic actions, single-domain Q&A). The deviations from "perfect" are mostly acknowledged trade-offs, not unknown unknowns.
Each tier shows: storage, latency posture, breaking point, and upgrade path. Numbers compose pgvector HNSW behavior (4× halfvec storage overhead), ParadeDB pg_search benchmarks (28ms at 100M rows; 1.6–2× faster index builds than alternatives), and the per-doc figures from red_team.md §3 / architecture.md §13.
| Resource | Size | Notes |
|---|---|---|
| Raw files (MinIO) | ~50 GB | trivial single bucket |
| Postgres total | ~3.5 GB | single small instance |
| HNSW index (~500K chunks × 768d halfvec) | ~0.7 GB | builds in minutes |
| pg_search BM25 | ~0.3 GB | builds in minutes |
| Audit log (1y) | ~0.5 GB | — |
Latency: indistinguishable from architecture target — 3-5s typical query, retrieval well under 1s.
Where it breaks: cold-start statistics (red-team F4). Below ~200 docs, L3 rarity scoring is noise (centroid is itself); anomaly retrieval channel ⑥ returns garbage. Mitigation already in plan: anomaly filter OFF by default until corpus crosses threshold.
Cost: one-time $600 ingest. Operational ~$0.001/query.
Verdict: ✓ no architectural changes needed; just guardrails.
| Resource | Size | Notes |
|---|---|---|
| Raw files (MinIO) | ~500 GB | — |
| Postgres total | ~35 GB | single instance fits |
| HNSW index (~5M chunks × 768d halfvec) | ~7 GB | builds in ~30–60 min on first ingest. Dimension note: Gemini Embedding 001 is 3072d native via Matryoshka Representation Learning (MRL) — truncatable to 768/1536/3072 at the API. We chose 768d-halfvec for 8× storage saving (60GB → 7.5GB at 100K-doc scale); 768d retains ~95% of 3072d quality per Google's MRL paper. Configurable per Design 9 — workspaces that need maximum recall can opt up to 3072d-halfvec or 1536d. |
| pg_search BM25 | ~3 GB | — |
| Audit log (1y) | ~5 GB | — |
Latency: factoid 3.3s · vague needle 4.9s · multi-hop 6.3s · IRCoT-escalated 12-15s · B2 agentic opt-in 15–30s (designed for).
Cost: $6,000 one-time ingest (+$1,500 of which is L2b emergent-fields = the cost of honest schema-emergence). Per-query $0.005–0.01 typical.
Concurrent load: single PG instance handles maybe 50–100 simultaneous query sessions before connection-pool contention. Beyond that, need PgBouncer + 1 read replica — small lift, well-documented PG pattern.
Verdict: ✓ the demo target; everything works on a single tuned PG instance.
| Resource | Size | Notes |
|---|---|---|
| Raw files (MinIO) | ~5 TB | object store scales horizontally; trivial |
| Postgres total | ~350 GB | still single instance with proper tuning (PG handles TB-scale fine) |
| HNSW index (~50M chunks × 768d halfvec) | ~70 GB | builds in ~6–10 hours on cold start; incremental adds in seconds |
| pg_search BM25 | ~30 GB | per ParadeDB benchmarks: 28ms query latency at 100M rows; we're at 50M chunks — fine |
| Audit log (1y, 10M queries) | ~50 GB | shard by month; archive older to S3 |
Latency: mostly identical to Tier 2; HNSW query latency grows logarithmically with index size, not linearly. Real concern: first-token latency on cold cache for large index pages.
Concurrent load — real: at 1M docs and 200+ active users, need:
- PgBouncer (transaction pooling, ~100ms overhead amortized away)
- 2 read replicas for vector + BM25 queries
- Audit log on separate logical replication outbox (don't compete with hot reads)
Cost: $60,000 one-time ingest. $5K–10K/year operational at 1M queries/year.
Where it could break: HNSW index build time on cold start (6–10 hours). Once built, queries are fine. Mitigation: incremental builds, never cold-start the full index after migration.
Verdict: ✓ runs on a single beefy PG instance + read replicas + PgBouncer. None of this is novel engineering. Pattern is well-trodden.
| Resource | Size | Notes |
|---|---|---|
| Raw files (MinIO) | ~50 TB | regional sharding by then |
| Postgres total | ~3.5 TB | sharding becomes relevant; or move heavy tables to dedicated DB |
| HNSW (~500M chunks × 768d halfvec) | ~700 GB | HNSW ceiling reached — vector index needs to graduate |
| pg_search BM25 | ~300 GB | ParadeDB still handles this; or shard by doc_type |
| Audit log | ~500 GB/year | partitioned + cold-tiered |
The HNSW graduation: pgvector's HNSW is wonderful up to ~50M vectors per index. Beyond that, build/maintenance time and memory pressure become real. The architecture stated this from day one (§7: "Graduation path: vectors → Turbopuffer or Qdrant at ~50M chunks (single swap behind adapter interface, no other rewrites).")
Concrete graduation:
- Turbopuffer (newer, serverless, ~10× cheaper than Pinecone)
- OR Qdrant (open source, self-hostable, mature)
- Both expose hybrid filter + vector query
- Migration: implement
VectorAdapter.search()against new store; backfill from PG-stored chunks; cut over. - ~1 week of engineering, predictable and well-documented.
Cost: $600,000 one-time ingest. This is the number nobody talks about. Per red_team.md §3.2 the figure is honest — at Reliance's scale, the LLM extraction bill is real.
Per-query cost: still ~$0.01 typical. Per-query economics scale beautifully because each query is independent.
Verdict: ⚠ needs vector graduation. Documented. Engineering well-understood.
| Concern | Solution at this scale |
|---|---|
| PG single-instance ceiling | Citus/CockroachDB sharding, OR split per-doc-type |
| Vector store | Distributed Turbopuffer/Qdrant cluster |
| BM25 | Quickwit (same Tantivy engine, distributed) |
| Procrastinate queue → ingest throughput | Kafka or NATS JetStream |
| Audit log | Append-only object store + Iceberg/Delta |
| Multi-region | Regional shards + cross-region replication |
Verdict: ✗ explicitly beyond our architecture's documented scope. Statable in the writeup as "100M+ requires distributed re-architecture; not a 1-day swap." Few real KBs reach this scale.
| Scale | Status | Action needed |
|---|---|---|
| 10K | ✓ runs on a laptop | nothing |
| 100K | ✓ single beefy PG | nothing (the demo target) |
| 1M | ✓ + read replicas + PgBouncer | small ops lift |
| 10M | ⚠ vector graduation | ~1-week engineering swap |
| 100M | ✗ distributed re-arch | months; out of scope |
| Optimization target | How we achieved it |
|---|---|
| Domain agnosticism | L0–L7 layers work the same way for contracts, emails, drawings, xlsx, scans; L3 atomic-unit per-doc-type plug-in |
| Schema emergence | L2b open-vocabulary extraction + cross-doc clustering + auto-promotion; no day-zero schema requirement |
| Audit-grade citations | Universal citation envelope across 10 modalities (Design 5); every claim points to a span; audit log is immutable hash-chained |
| Cost control per query | Hard-capped IRCoT at 2 hops; per-query cost ceiling on B2 agentic mode; intent classifier gates expensive paths |
| Transparency | Planner JSON shown to user; channel scores in audit log; "How I answered" inspector in chat |
| Refuse-don't-hallucinate | Astute RAG + HHEM gate + semantic fallback alongside typed filter (red-team F5 fix) |
| Adjacent use case | Better tool | Why we didn't optimize for it |
|---|---|---|
| Prototype Q&A over 10 PDFs | OpenAI File Search, Anthropic File Search | Zero-engineering wins for trivial scope; ours is overkill |
| Single-source thinking partner | NotebookLM | Different scale (per-collection vs per-corpus); their UX is right for that scope |
| Permission-aware enterprise search | Glean ($7.2B-valued for a reason) | ACL is deployment-integration + Wave C; we chose schema-emergence focus |
| Multi-agent finance/legal analyst | Hebbia Matrix (30% of asset managers) | Wave B B1 + B4 close this gap; default is single-question deep retrieval |
| Live source connectors (Slack/SharePoint/Gmail) | Glean, Onyx | Connectors are deployment integration, not architecture |
| Agentic action-taking (send email, place order) | Computer-use agents, action-LLMs | Read-only by design; KB is substrate, actions are a layer above |
| Real-time analytics over POS/SCADA/ATM streams | OLAP / streaming platform | KB ≠ OLAP, by design |
| CAD/BIM geometry queries | Domain-specific CAD tools | Visual-only via ColPali (Wave C); geometry is out of scope |
The architecture has:
- 10 main storage layers (L0, L0.5, L1, L2, L2b, L3, L4, L5, L6, L7) plus 4 L1 sub-layers (L1a, L1b, L1c, L1d)
- 12 planner modes (E/F/S/H/T/M/G/D/C/A/Q/K)
- 10 parallel retrieval channels
- 9 tier-1 gap designs with their own pipelines (Q-mode, conflicts, doc chains, feedback, citations, vocabulary, lineage, chat context, layered config)
- 4 Wave B additions committed (batch mode, agentic loop, DSPy, multi-agent decomposition)
This is a lot of engineering surface area. The simpler alternative — single chunk-and-embed pipeline with one retrieval channel — would be 10% the code.
Why we chose this: every layer earns its keep for a specific query class:
- L1a Contextual Retrieval: vocabulary-mismatch needles (
docs/archive/red_team.mdedge case 1) - L1d RAPTOR: vague/abstract queries
- L2b emergent fields: schema-emerges principle
- L3 atomic units: rare-clause needles (edge case 2)
- L4–L5: identity resolution + relationships across docs
- L6 HippoRAG: multi-hop
- L7 LazyGraphRAG (Wave C): global/thematic queries
- Q-mode: aggregations and set operations
- K-mode (chains): "latest revision" / amendment supersession
If we remove a layer, one specific query class fails. The evaluation set deliberately includes all these query classes. So the complexity is the cost of admission.
Is it more complex than Hebbia? Glean? Yes. Hebbia hides much of this under their multi-agent abstraction; Glean hides it under their knowledge graph + ranker. We expose it intentionally for transparency. Trade-off chosen deliberately.
For pure CUAD-only contract Q&A:
- Drop L0.5 (no amendment chains in CUAD demo data)
- Drop L7 (no global thematic queries)
- Drop ColPali / L1c (no scanned docs needed)
- Drop Q-mode (CUAD eval is retrieval, not aggregation)
- Keep L1a / L1d / L2 / L2b / L3 / L4 / L5 / L6
That cuts ~30% of code. We didn't, because the demo corpus is mixed and the problem brief explicitly says "domain-agnostic."
Numbers compose: Gemini Flash typical latencies (intent ~100ms, planner ~300-500ms, generation ~1-3s with 20-doc context streamed), Cohere Rerank 3.5 80–150ms p50 (verified), pgvector HNSW (sub-100ms at 100K-doc scale), pg_search BM25 (28ms at 100M rows per ParadeDB).
| Class | Wall-clock | Perceived (with streaming) | OK? |
|---|---|---|---|
| Simple factoid | 3.3s | ~1.5s to first token | ✓ |
| Aggregation Q-mode | 3.5s (templated answer, no LLM synthesis) | ~1.5s | ✓ |
| Vague needle | 4.9s | ~1.8s | ✓ |
| Multi-hop, no IRCoT | 6.3s | ~2s | ⚠ |
| Multi-hop WITH IRCoT (capped 2 hops) | 8-12s | ~3.5s | ⚠ |
| Conflict-resolution query | 6-8s | ~2.5s | ⚠ |
| Chain-aware "latest revision" | 4s | ~1.8s | ✓ |
| B2 opt-in agentic deep_research | 15–30s | ~5s (with progress strip) | acceptable for opt-in only |
| B1 batch query (400 docs × 1 question) | 30–90s for the batch; per-cell ~5s parallel | shown as progress UI | acceptable for batch UX |
| Schema-swap re-extraction at 80 docs | ~3 min | shown in /upload page | ✓ |
| Schema-swap re-extraction at 100K docs | ~3 hours | background job | acknowledged honestly |
For a typical 5s vague-needle query (parallel where possible):
0.10s intent classifier (Gemini Flash, ~200 tok)
0.60s rewriting (parallel) (HyDE×3 + Step-Back + Q2D, max-of-4 Flash)
0.40s planner JSON (Gemini Flash with JSON schema)
0.80s parallel retrieval (10 chans) (BM25 + HNSW + clause + PPR + ...)
0.10s RRF (in-process)
0.40s Cohere Rerank top-50 (vendor API, p50 80-150ms; ours is 50 items)
0.05s CRAG gate (in-process)
0.30s conflict detector (Flash, on candidates with field disagreement)
2.50s generation (Flash, 20-doc ctx, streamed)
0.30s HHEM faithfulness (local model)
0.20s audit log + render (in-process)
─────
5.75s wall-clock
Three latency levers we already use:
- Stream generation → first token at ~2s perceived (vs. 5.75s wall-clock). Already in
architecture.md§6 step 8. - Cap rerank at top-50 (not top-200) → -300ms. Already in red-team #2.
- Cap IRCoT at 2 hops (not 4) → bounds worst-case. Already in red-team #2.
Two we could add if needed (not blocking):
- Run intent classifier + planner speculatively in parallel (planner has to wait for intent today). Saves ~300ms.
- Pre-warm Cohere Rerank session pool. Saves ~100ms cold-start.
| Workload | Throughput | Bottleneck | Note |
|---|---|---|---|
| Ingest (Docling digital PDF) | ~3–5 docs/min/worker (steady-state with async pipelining); single-doc end-to-end latency ~90s (walkthrough.md trace) | parser + embedder + vendor LLM I/O | A worker has ~5–10 docs in flight at different pipeline stages; per-doc latency != per-worker throughput |
| Ingest (Mistral OCR scanned) | ~1–2 docs/min/worker | OCR vendor latency (single API call dominates) | OCR is mostly serial — less pipelining benefit |
| Concurrent queries | 50–100 / instance | PG connection pool | Out-of-the-box |
| Concurrent queries with PgBouncer | 500+ / instance | Cohere Rerank API rate limits | Tier 3+ ops setup |
| Batch-mode B1 (cells/sec) | ~5/sec parallel | Gemini Flash rate limit | Wave B feature |
Verdict on speed: the typical query is competitive with production RAG systems. The IRCoT-escalated and B2 agentic paths are slower by design — they buy quality on hard queries at a clear cost. We don't claim to be fast on agentic; we claim to be correct on agentic.
| Stage | Per query | Notes |
|---|---|---|
| Intent classifier | $0.00005 | ~200 tok Flash |
| Rewriting (worst: 5 calls) | $0.00125 | ~1500 output tok across HyDE/StepBack/Q2D |
| Planner JSON | $0.00030 | ~400 tok Flash |
| Parallel retrieval | $0 | local SQL + vector ops |
| Cohere Rerank top-50 | $0.0005 | $2/1K queries × 50/200 batch ratio |
| Conflict detector (10% of queries) | $0.00020 | Flash, only when needed |
| CRAG gate | $0 | local |
| Generation (Flash, 20-doc context, ~600 out) | $0.0035 | dominates |
| HHEM-2.1 | $0 | local model |
| Audit | $0 | DB writes |
| Typical total | ~$0.006 | matches earlier estimate |
| Worst case (IRCoT 2 hops) | ~$0.018 | full pipeline × 2 |
| Worst case (B2 agentic, 5 hops with Pro synthesis) | ~$0.08 | opt-in, capped |
| Stage | Per doc | Notes |
|---|---|---|
| Parse (Docling local) | $0 | open source |
| Parse (Mistral OCR if scanned) | $0.002 | $2/1K pages, ~1 page/doc avg overhead |
| Contextual prefix (Flash, prompt-cached) | $0.0012 | 12 chunks × $0.0001 |
| Chunk embedding | $0.0006 | Gemini Embedding 001 |
| RAPTOR per-doc build | $0.005 | 4–5 cluster summary Flash calls + 1 doc summary call; ~5–8s wall-clock per doc with parallel cluster summarization |
| Cross-doc RAPTOR apex (corpus themes) | ~$1 amortized | GMM clustering on 100K doc summaries (~3 min single-threaded sklearn) + 50–100 theme summary Flash calls. ~5–10 min one-time wall-clock at 100K-doc scale; only re-runs when apex thresholds shift |
| L2 mention extraction | $0.012 | 1 Flash call over 30 pages |
| L2b emergent fields extraction | $0.018 | +50% over previous — the cost of schema-emergence |
| L3 atomic-unit extraction | $0.008 | 1 Flash call |
| Identity resolution (LLM judge) | $0.005 | ~25 Flash calls per doc across all entity types (ORG + PERSON + LOCATION + PRODUCT + EVENT); deterministic name-normalize catches ~65% of mentions before LLM-judge is needed |
| Schema-driven extraction | $0.003 | 1 Flash call |
| Doc-chain detection | $0.001 | LLM judge on borderline |
| Artifact gen (async) | $0.005 | briefing + FAQ + suggested Qs |
| Per-doc total | ~$0.06 | up from ~$0.04 pre-L2b |
| Corpus | One-time ingest | Schema re-extract (per version) | Per-query (1M queries/year) | Storage/year |
|---|---|---|---|---|
| 10K docs (pilot) | ~$600 | ~$30 | ~$5,000 | ~$1,000 (MinIO + PG) |
| 100K docs (architecture target) | ~$6,000 | ~$300 | ~$5,000 | ~$2,000 |
| 1M docs (10×) | ~$60,000 | ~$3,000 | ~$5,000 | ~$10,000 |
| 10M docs (Reliance) | ~$600,000 | ~$30,000 | ~$10,000 | ~$80,000 |
| 100M docs | ~$6M | ~$300K | ~$50K | ~$800K |
Two observations:
-
Per-query cost is essentially flat with corpus size — because retrieval is O(log n) on HNSW, BM25 is fast at 100M-row scale, and generation cost is determined by context window not corpus size. Per-query economics scale beautifully.
-
One-time ingest cost is the eyebrow-raising number. $600K at Reliance scale is the figure to state honestly in the writeup. Mitigations: prompt caching on Contextual Retrieval (already in pipeline), batched embedding (Gemini batch mode is 50% cheaper), retiring artifact generation as Wave B opt-in not default.
| System | 100K docs / year | 1M queries / year | Notes |
|---|---|---|---|
| Our architecture | $6,000 ingest + $5,000/yr ops | included | ~$11K total year 1 |
| OpenAI File Search | $50K/month storage at 5MB/doc × 100K = $600K/year storage alone | + $0.10/query × 1M = $100K | $700K+ |
| Glean (per-seat) | n/a | $30–$50/user/month × 1000 users = $360–600K/year | $360–600K |
| Hebbia (per-seat) | n/a | reported $100K+/seat/year | $1M+ |
We are competitively priced. The CFO of any enterprise serious about this will pick our pricing model if their use case matches ours.
In rough priority order:
- Cold-start statistics under ~200 docs (red-team F4). Anomaly retrieval channel ⑥ is noise. Mitigation: turn off until threshold. Acknowledged.
- Doc-type classifier blast radius (red-team F10). 95% accurate classifier = 5% of docs invisible to L3 channel for clause-level queries. Mitigation: multi-label + "unknown" route, planned Phase 5.
- Concurrent ingest + query saturates single PG (red-team F9). Mitigation: PgBouncer + read replicas at Tier 3+.
- HNSW ceiling at ~50M chunks. Mitigation: documented graduation to Turbopuffer/Qdrant.
- Embedding model lock-in. Swap to a non-Gemini embedder = re-embed all chunks. ~$1,500–$15K at 100K-doc scale depending on vendor. Mitigation: lock to Gemini Embedding 001 deliberately; revisit at MTEB leaderboard shifts.
- Cohere external dependency. Real downtime risk. Mitigation:
mxbai-rerank-large-v2fallback, but not wired in the failure path — needs to be (red-team F7). - Long-tail entity-resolution accuracy. L4 LLM-judge will miss some merges; some queries will say "no evidence" when evidence is under an un-merged alias. Mitigation: feedback loop (Design 4) closes this on user correction.
- Bi-temporal AS-OF queries. Doc chains (Design 3) handle latest-revision/supersession. Full
AS OF '2023-06-15'fact-level history is Wave B/C. - Schema operations beyond add (rename/split/merge/delete). Cascade through audit log + saved queries needs design. Not designed yet — tier-1 gap.
- L2b open-vocabulary extraction cost (+50% over fixed-list). ~$1,500 at 100K-doc scale. Accepted as cost of honest schema-emergence.
- Multi-tenant isolation. Wave C, blocks production deployment in finance/healthcare/government. Acknowledged.
- Cross-lingual L3. L3 clause typer is English-only. Marathi land records have no L3, just L2 + RAPTOR. Wave C.
- Real-time freshness lag. 60–120s end-to-end ingest. Doc uploaded 30s ago is not queryable. Acknowledged design choice.
- No DSPy yet (prompts hand-written). Wave B B3 commits to fix this.
- No multi-agent decomposition yet (Hebbia gap). Wave B B4 commits.
- No opt-in agentic loop (Search-o1 / ReAct gap). Wave B B2 commits.
- No batch query UX (Hebbia spreadsheet gap). Wave B B1 commits.
- Schema-swap "in seconds" only at demo scale. At 100K docs, schema-swap re-extraction is hours, not seconds. Acknowledged honestly.
Of these:
- 8 are scoped out by design (1, 8, 11, 12, 13, 18 mostly, plus the 5 deliberate non-pursuits in README)
- 4 are committed Wave B fixes (14, 15, 16, 17)
- 5 are addressed by Wave B / planned Phases (2 → Phase 5, 3 → Phase 11 ops, 4 → graduation, 6 → adapter wire, 9 → next batch tier-1)
- 1 is an accepted trade-off (10, the L2b cost)
Net: every weakness has a stated mitigation or a deliberate "not in scope" framing. None is an unknown unknown.
Architecturally identical; ops scales up:
- PgBouncer (transaction pooling, ~1 hour of ops setup)
- 2 read replicas (split vector + BM25 reads from writes)
- Audit log to logical replication outbox (don't compete with hot reads)
- Schedule incremental HNSW maintenance windowed (avoid spike during queries)
- Storage tiering: chunks older than 1y move to compressed cold partition
- Cost: ~$60K ingest + ~$5K/year ops + ~$10K/year storage = $75K total year 1
Real architectural deltas (~1 week of engineering each):
- Vector graduation to Turbopuffer or Qdrant (planned; adapter swap)
- BM25: ParadeDB still works; shard by doc_type if needed
- PG sharding by doc_type or by date (Citus)
- Procrastinate → Kafka for ingest throughput
- Multi-region: regional shards + cross-region replication for audit + canonical entities
- Cohere Rerank: dedicated rate-limit tier OR local mxbai with bigger GPUs
- Cost: ~$600K ingest + ~$80K/year ops/storage = $680K total year 1
Beyond our architecture's documented scope. Would need a re-architecture pass (Iceberg/Delta for audit, distributed vector store fully, perhaps a graph database as L4–L6 backbone). Out of scope for this writeup.
Is the architecture perfect? No. Nothing this complex is. Specifically:
- It is near-optimal for our explicit problem (domain-agnostic enterprise KB, audit, schema emergence). 17/17 cited techniques are real, current, and accurately characterized.
- It has 18 known weaknesses, of which 17 are acknowledged trade-offs or scoped-out items, and 1 is an accepted cost (the L2b ~$1,500/100K-doc premium).
- It scales linearly to 1M docs out-of-the-box, with documented graduation to 10M (1-week engineering), and a re-architecture path beyond.
- It is fast where it matters (typical query 3-5s with streaming, perceived ~1.5s to first token). Slow on the deliberately slow paths (IRCoT escalation, B2 agentic, B1 batch).
- It is cost-effective per query ($0.005–0.01) and expensive but priced honestly per corpus ($6K at 100K docs, $600K at 10M).
The architecture is the result of explicit trade-offs. Each layer earns its keep. Each cost is named. Each scaling ceiling is documented. The deliberate non-pursuits (permissions, native CAD geometry, real-time streaming, bi-temporal AS-OF, agentic actions, vector graduation past 50M chunks, image content beyond layout, multi-tenant isolation, cross-lingual L3, live source connectors) are stated in the README. The four Wave B additions are committed to close real 2026 SOTA gaps.
If asked "is this perfect?", the truthful answer is:
"No system at this scope is perfect. The architecture is composed of 17 verified-real techniques from the 2024–2026 literature and competitive landscape, with every cost, latency, and scaling ceiling stated honestly in
docs/scale_perf_audit.md. Eighteen specific weaknesses are enumerated with mitigations. The four 2026 SOTA gaps that competitors are ahead on (Hebbia multi-agent, Search-o1 agentic, DSPy optimization, batch UX) have committed Wave B fixes. The architecture is deliberately positioned: optimal for our use case, deliberately suboptimal for adjacent ones, with the boundaries named explicitly. It is not perfect; it is honest."
That answer is defensible. It is grounded in this audit.