Alignment

Author: mortensi

docs/server-architecture.md

@@ -0,0 +1,208 @@
+# JsonDocStore Server Sketch
+
+This branch explores a server-based architecture for `jsondocstore`.
+
+## Goal
+
+- Keep one JSON file per document.
+- Keep indexes on disk.
+- Expose the store through a small native server process.
+- Let Python and JavaScript use thin client libraries instead of native bindings.
+
+## High-Level Model
+
+Components:
+
+- `jsondocstored`: native server, ideally written in C
+- Python client package
+- JavaScript client package
+
+The server owns:
+
+- document reads and writes
+- index maintenance
+- locking
+- crash recovery
+
+Clients only send requests and decode responses.
+
+## On-Disk Layout
+
+```text
+store/
+  docs/
+    user-1.json
+    user-2.json
+  indexes/
+    role.idx
+    email.idx
+  meta/
+    schema.json
+    journal.log
+    lock
+```
+
+Notes:
+
+- `docs/` remains the source of truth for document contents.
+- `indexes/` stores persistent secondary indexes.
+- `meta/schema.json` defines indexed fields and format version.
+- `meta/journal.log` supports recovery after crashes.
+
+## Transport
+
+Preferred order:
+
+1. Unix domain socket for local usage
+2. TCP for optional remote usage
+
+A local socket keeps the first version simpler and safer than exposing TCP by default.
+
+## Wire Protocol
+
+Use newline-delimited JSON messages.
+
+Request examples:
+
+```json
+{"op":"get","key":"user-1"}
+{"op":"insert","key":"user-1","doc":{"username":"alice","role":"admin"}}
+{"op":"update","key":"user-1","doc":{"username":"alice","role":"user"}}
+{"op":"delete","key":"user-1"}
+{"op":"query_by","field":"role","value":"admin"}
+{"op":"create_index","field":"role"}
+{"op":"delete_index","field":"role"}
+{"op":"list"}
+{"op":"list_indexes"}
+```
+
+Response examples:
+
+```json
+{"ok":true,"result":{"username":"alice","role":"admin"}}
+{"ok":true,"result":{"user-1":{"username":"alice","role":"admin"}}}
+{"ok":false,"error":"Document not found: user-1"}
+```
+
+Reasons for this choice:
+
+- easy to debug
+- trivial in Python and JavaScript
+- no custom binary framing needed at first
+
+## Process Model
+
+First version:
+
+- one server process per store
+- single writer
+- requests handled sequentially
+
+That is enough to guarantee consistent document and index updates without introducing concurrency complexity too early.
+
+Later improvements:
+
+- concurrent readers
+- request queue
+- worker threads
+
+## Operations
+
+`insert`
+
+1. validate key
+2. validate document
+3. compute indexed values
+4. append intent to journal
+5. write `docs/<key>.json` atomically
+6. update affected index files
+7. mark journal entry committed
+
+`update`
+
+1. read current document
+2. compute old and new indexed values
+3. append intent to journal
+4. rewrite `docs/<key>.json` atomically
+5. update only affected index entries
+6. mark journal entry committed
+
+`delete`
+
+1. read current document
+2. append intent to journal
+3. remove index entries
+4. delete the JSON file
+5. mark journal entry committed
+
+## Indexing
+
+For each indexed field, persist:
+
+- field value
+- document key
+
+Conceptually:
+
+```text
+role=admin -> [user-1, user-7]
+role=user  -> [user-2, user-3]
+```
+
+The first implementation can use one index file per field. The server should still update only the affected field entries during `insert`, `update`, and `delete`.
+
+## Recovery
+
+On startup:
+
+1. acquire store lock
+2. load schema
+3. inspect journal
+4. repair or roll forward incomplete operations
+5. open socket and accept requests
+
+Also provide:
+
+- `rebuild_indexes`
+- `verify_store`
+
+Those are maintenance commands, not normal client operations.
+
+## Client Libraries
+
+Python package:
+
+- `pip install jsondocstore`
+- thin client over socket communication
+
+JavaScript package:
+
+- `npm install jsondocstore`
+- thin client over the same protocol
+
+Both clients should mirror the same API shape:
+
+- `get(key)`
+- `insert(key, doc)`
+- `update(key, doc)`
+- `delete(key)`
+- `queryBy(field, value)`
+- `createIndex(field)`
+- `deleteIndex(field)`
+- `list()`
+- `listIndexes()`
+
+## Why This Direction
+
+Pros:
+
+- one native implementation
+- easier Python and JavaScript distribution
+- one document per JSON file remains intact
+- indexes can be persistent and real
+
+Cons:
+
+- requires a running process
+- local deployment is more operationally complex than an embedded library
+- protocol and process lifecycle become part of the product