more db design info

Author: wshands

docs/database.md

@@ -6,6 +6,35 @@ Schema is defined in [db/schema.py](../db/schema.py) and initialized by the `det
 
 ---
 
+## Schema Design
+
+**Entities and attributes.** Each table represents a distinct *entity* — a thing the system tracks. Each column is an *attribute* — a fact about that entity. The four entities are: a biological sample, a genomics file, a pipeline execution, and an audit event. When a concept accumulates its own attributes it earns its own table; otherwise it stays a column. `samples` exists as a separate table because a sample has its own properties (`patient_id`, `assay_type`, `organism`) beyond just an ID. If samples had no attributes, `sample_id` could remain a plain column in `genomics_files`.
+
+**Primary keys — surrogate vs natural.** `genomics_files`, `pipeline_runs`, and `audit_log` use `BIGSERIAL` surrogate keys (`file_id`, `run_id`, `log_id`) — auto-incrementing integers generated by PostgreSQL that have no business meaning. `samples` uses a natural key (`sample_id VARCHAR(64)`) because that identifier comes from the outside world and is already meaningful and stable. Surrogate keys are preferred when the natural identifier might change, is long, or is not guaranteed unique across platforms.
+
+**Foreign keys.** A foreign key is a column whose value must match a primary key that already exists in another table. PostgreSQL enforces this constraint on every insert and update. The relationship chain here is:
+
+```
+samples.sample_id  ←── genomics_files.sample_id (FK)
+                        genomics_files.file_id   ←── pipeline_runs.file_id (FK)
+```
+
+`audit_log.file_id` is deliberately **not** a foreign key — if a file record is deleted, the audit history must survive intact for HIPAA compliance. A foreign key would either block the deletion or cascade-delete the audit rows.
+
+**Generated column.** `pipeline_runs.duration_secs` is computed automatically by PostgreSQL from two other columns and never written manually:
+
+```sql
+duration_secs FLOAT GENERATED ALWAYS AS (
+    EXTRACT(EPOCH FROM (completed_at - started_at))
+) STORED
+```
+
+`completed_at - started_at` produces a PostgreSQL `INTERVAL`. `EXTRACT(EPOCH FROM ...)` converts it to total seconds as a float. `STORED` means the value is written to disk rather than recomputed on every read. The column is `NULL` while a run is in progress because `completed_at` is not yet set.
+
+**Normalization.** The schema avoids transitive dependencies — no column depends on another non-key column. Sample attributes (`patient_id`, `assay_type`) live in `samples`, not in `genomics_files`, so renaming a sample requires updating one row rather than thousands. This is Third Normal Form (3NF).
+
+---
+
 ## Tables
 
 ### `samples`
@@ -172,7 +201,7 @@ ORDER BY a.logged_at;
 **Recent pipeline run performance:**
 ```sql
 SELECT r.run_id, f.file_name, r.status,
-       ROUND(r.duration_secs, 1) AS secs,
+       ROUND(r.duration_secs::NUMERIC, 1) AS secs,
        r.error_message
 FROM pipeline_runs r
 JOIN genomics_files f ON f.file_id = r.file_id