Skip to content

Latest commit

 

History

History
99 lines (83 loc) · 15.8 KB

File metadata and controls

99 lines (83 loc) · 15.8 KB

Transparent Session Filesystem Implementation Alignment

Purpose

This document aligns the original product commitments, the canonical transparent-filesystem contract, the implementation plan, the current repository, and the available validation evidence.

It does not redesign CodexFold, authorize real-session enrollment, promote the capability above fs-engine-preview, or treat fixtures as production evidence. CodexFold remains a standalone public product. Native launchd, systemd --user, and Windows SCM supervision are part of the standalone runtime; private or external control-plane coupling remains outside it.

Baseline refreshed against the native FSKit development checkpoint and current Task 12 through Task 15 implementation on 2026-07-23.

The validation references in this document describe the completed signed build 102 App/extension and current helper isolated Canary. It passed installed mount-health, byte-identity, restart, host-reboot, rollback, and exact current-client gates. None of the status rows below authorize production activation or transfer that evidence to a later candidate.

Original Commitment To Requirement Mapping

Original product commitment Canonical requirement Alignment result
Codex Desktop and CLI open and resume a normal JSONL path without a materialization step TF-001, TF-002, TF-003 Preserved
The client remains unmodified and cannot distinguish managed storage from a supported native file TF-002, TF-003 Preserved
Exact duplicate content is stored once across sessions and forks TF-004 Preserved
Reuse is not limited to a strict shared prefix; repeated fields, records, and chunks at arbitrary positions are shareable TF-004 Clarified in the contract
Forks remain independently writable after sharing common content TF-004, TF-005, TF-006 Preserved
Normal writes append to a durable delta; non-append mutations use safe copy-on-write TF-005, TF-006 Preserved
Runtime reads use packed storage rather than tens of thousands of loose-object opens TF-007, TF-008 Preserved
Correctness includes performance, bounded memory, crash recovery, restart recovery, and exact rollback TF-008, TF-009, TF-010 Preserved
Stable production operation discovers existing sessions, new sessions, and forks automatically TF-011 Preserved and implemented behind explicit activation, health, writer, stability, and storage gates
macOS, Linux, and Windows share one storage engine but have independent adapters and readiness gates TF-012, TF-016, TF-017 Preserved
Capability language cannot overstate a storage engine, preview, or one successful canary TF-013 Preserved
Native sources and current recoverable bytes remain available until the relevant gates pass TF-010, TF-014, TF-015 Preserved
Useless or closed branches can be identified and archived, but the tool must not guess destructively TF-018 Implemented with evidence-only family reports and explicit recoverable archive transactions
A branch that is exactly 100% contained in another retained session can be deleted only after exact recovery proof TF-019 Added to the contract; implementation exists
Prompt cleanup, repair, and reconciliation are separate content-changing workflows, not storage folding TF-020 Added to the contract; implementation and static regression boundaries exist
Temporary files, recovery generations, retained snapshots, and repeated operations must not consume unbounded disk TF-021 Added to the contract; hard budgets, bounded retention, leases, and GC are implemented
Reported savings distinguish logical reuse from actual physical bytes reclaimed TF-021 Added to the contract; projected and actual physical accounting is implemented
CodexFold is an independent open-source product with no private control-plane dependency TF-022 Added to the contract; current repository is aligned

Requirement To Implementation And Evidence

Requirement Current implementation Tests or evidence Status
TF-001 internal/cli/fs.go, internal/mountfs, canonical migration, automatic enrollment, and routing Isolated CLI/Desktop direct-open, resume, and automatic-enrollment canaries Partial only at public release level: implemented and verified on macOS canaries; explicit real-home activation requires an external quiescent cutover
TF-002 internal/mountfs, internal/fskitproto, the Swift FSKit extension, canonical routing, and mount identity Native FSKit mounted behavior plus real current CLI/Desktop resume, append, fork, archive, restart, post-crash continuation, and controlled in-flight host-interruption recovery Implemented and verified in an isolated macOS canary; production promotion remains retention-gated
TF-003 Neutral operation layer plus diagnostic operation evidence in internal/compat Real macOS traces and adapter canaries plus real Linux FUSE3 operations Partial: current installed macOS clients and Linux adapter operations are covered; real Linux Codex clients and Windows are not validated
TF-004 internal/scan, internal/cdc, internal/fold, internal/pack Repeated field, record, CDC, fork, and non-prefix corpus tests Implemented
TF-005 internal/vfs append delta and writer leases Append-without-hydration tests and real CLI/Desktop append evidence Implemented
TF-006 internal/vfs copy-on-write backing and neutral write operations Random-write, truncate, interruption, native FSKit mounted mutation, and historical FUSE-T tests Implemented
TF-007 Immutable packs, in-memory index, bounded cache, random-read resolver Pack round-trip/corruption tests and 758 MiB packed-read benchmark Implemented
TF-008 Packed-read benchmark, mounted performance tests, and internal/testfs stress harness 758 MiB core benchmark, three native FSKit cold/warm F_NOCACHE rounds, cache-coherency matrix, bounded RSS, historical synchronous FUSE-T measurements, and Linux FUSE3 race performance Implemented and verified for the shared core and macOS adapter; Windows runtime metrics remain open
TF-009 Journal recovery, generation recovery, service keep-alive, restart-safe retirement Recovery tests, daemon restart canaries, managed Deep Idle sleep/wake, retained-source host reboot, and a durable journal / partial-tail / non-graceful-host-restart canary Implemented and verified for the isolated macOS transaction path; retention remains open
TF-010 Shadow compare, optimistic routes, retained snapshots, current-byte fallback 90,000 real random-range comparisons, rollback and failure-containment canaries, and one bounded retained-source user-home canary Implemented for macOS canaries; retention remains open
TF-011 internal/enroll, fs enroll, and the bounded standalone-service loop discover existing, new, and forked sessions, persist stability observations, take a fail-closed native-writer snapshot, and reuse fold/pack/migrate transactions Policy tests plus a real writable-descriptor probe and isolated canonical FUSE enrollment, daemon restart, real CLI append, version-change diagnostics, and failed-cutover evidence Implemented; explicit real-home activation still requires an external quiescent cutover
TF-012 Shared Go core, Apple-native Swift FSKit on macOS, Linux FUSE3, and Windows WinFsp adapters Native FSKit mounted behavior, current-client, crash, and rollback tests; historical FUSE-T evidence; real Linux adapter tests; and Windows cross-compiles Partial at the cross-platform product level: macOS real-client acceptance passes, while all real Windows gates remain open
TF-013 Canonical capability type in internal/fsctl/status.go Status rejection tests and CLI status tests Implemented; current status is fs-engine-preview
TF-014 Snapshot retention and destructive-action guards Migration, rollback, and retention tests Implemented as a safety rule; retention promotion gates remain open
TF-015 Non-blocking compatibility diagnostics plus writer, fingerprint, mount, and journal guards Unknown-version no-reroute, direct-migration writer, and isolated canary tests Implemented; versions do not gate filesystem access
TF-016 Native FSKit Host/extension packaging, non-elevating launchd, systemd --user, and Windows SCM lifecycle Signed native FSKit app installation/rollback, launchd crash matrix, real Linux service, and Windows cross-compile evidence Implemented; native release packaging and Windows runtime execution remain unverified
TF-017 Canonical namespace, write-sealed backing, mount identity, route normalization, daemon/supervisor locks, and build identity Native FSKit mounted namespace and current-client tests, four-process crash matrix, app/binary rollback, controlled in-flight host interruption, real Linux FUSE3, and historical FUSE-T evidence Partial at the cross-platform product level: isolated macOS current-client gates pass; retention and Windows gates remain open
TF-018 internal/codex spawn edges, internal/family graph/content evidence, internal/archive guarded transactions, and public fork-family plus archive commands Diverse relationship fixtures, repeated-record performance regression, source-change rejection, official archive trace, native apply/recovery, and isolated managed FUSE-T archive/unarchive plus daemon restart Implemented
TF-019 internal/contain and internal/prune; public contains and remove-contained commands Exact containment, archived-only apply, transaction rollback, and recovery-manifest tests Implemented
TF-020 Exact fold/migrate paths are byte-preserving; repair-rollout and reconcile-rollout write separate explicit outputs; a static production-import boundary prevents other workflows from invoking reconciliation internal/reconcile, CLI behavior, and AST boundary tests Implemented
TF-021 internal/storage provides physical inventory, configurable hard budgets, mutation preflight, generation and retired-state retention, lease-aware startup/explicit GC, and projected versus actual reclamation Hard-link accounting, low-space refusal, lease retention, interrupted cleanup, repeated GC, cross-platform compile, and live read-only inventory evidence Implemented; destructive retention remains promotion-gated
TF-022 Standalone CLI, daemon, launchd/systemd/SCM service management, configuration, storage, doctor, GC, rollback, and enrollment code Public coupling scan and sanitization test Implemented

Implementation Plan Task Status

Task Status Evidence Exact remaining scope
Task 1: packed object generation and resolver Complete Commit 17564e9; internal/pack tests pass None in Task 1
Task 2: exact immutable virtual byte view Complete Commit 039e6b9; exact and 10,000 random-read tests pass None in Task 2
Task 3: append and copy-on-write engine Complete Commit 9a7f1e8; append, COW, writer, reopen, and interruption tests pass None in Task 3
Task 4: journal, compaction, and fallback Complete Commit 076d772; recovery, compaction, and latest-byte fallback tests pass None in Task 4
Task 5: shadow, doctor, benchmark, and status Complete Commit 35a53fc; focused and shared-core evidence exists Real-platform promotion remains outside Task 5
Task 6: compatibility diagnostics and route transactions Complete Commit 5be10d2; route race and version-diagnostic tests pass New client evidence improves regression coverage but does not change runtime access
Task 7: neutral filesystem and tagged FUSE host Complete Commit 3f51aa5; neutral, real macOS FUSE-T, and real Linux FUSE3 tests pass; Windows WinFsp cross-compiles Windows real-adapter execution remains platform work
Task 8: standalone CLI and automatic enrollment Complete Command surface, guarded lifecycle, bounded planner/apply loop, native service arguments, and isolated real FUSE enrollment evidence Production enablement remains outside Task 8
Task 9: service lifecycle and update guard Complete Commit 4589ffa; launchd, real systemd --user, Windows SCM compile, and preflight tests pass Windows runtime and stronger automatic update claims remain release-gated
Task 10: synthetic, crash, performance, and compile gates Complete for the shared engine Commit a1ac76e; preview validation report It cannot satisfy real-adapter or retention gates
Task 11: real macOS trace, adapter, shadow, and canary Partial Native FSKit build 102 mounted behavior, performance/coherency, bounded RSS, four-process and host-reboot recovery, controlled durable-journal host interruption, atomic update rollback, and complete real current CLI/Desktop acceptance Incident-free retention window remains
Task 12: bounded automatic discovery and enrollment Complete Planner/apply/service tests plus isolated canonical FUSE enrollment, daemon restart, real CLI append, version diagnostics, and failed-cutover evidence Explicit real-home activation requires an external quiescent cutover
Task 13: conservative branch lifecycle and content-change boundary Complete Spawn-edge family reports, exact relationship comparison, official-compatible guarded archive and recovery, separate exact-contained deletion, and static content-change boundaries pass unit, race, native, and managed FUSE-T validation None in this task
Task 14: hard storage budgets, retention, cleanup, and accounting Complete Platform-neutral inventory, hard preflight, lease-aware bounded GC, truthful accounting, low-space and repeated-GC tests Destructive retention remains platform-gated
Task 15: remaining platform and retention gates Partial Current macOS regression evidence, real-client acceptance, and controlled in-flight host interruption; real Linux FUSE3 operation, crash, performance, backing policy, and systemd lifecycle; Windows WinFsp/SCM cross-compile Retention period, Linux client/upgrade/rollback validation, and all real Windows gates remain

Remaining Product Behavior And Exact Next Work

Missing behavior Exact implementation work Required verification
Remaining macOS promotion gates Complete the incident-free retention window only on disposable or explicitly approved data No new route loss, exact full-file SHA where applicable, and the required incident-free window
Linux remaining readiness Keep the implemented FUSE3 adapter and systemd lifecycle behind platform gates Real Linux Codex traces, client upgrade transparency, rollback, and retention
Windows readiness Execute the implemented WinFsp adapter and SCM host without moving shared behavior out of the core Native operations, crash/restart, performance, real Codex traces, client upgrade transparency, rollback, and retention on a real Windows host

Current Capability Decision

The shared storage, virtual-file, bounded automatic-enrollment, and physical-space governance engines are implemented and validated strongly enough for fs-engine-preview. Linux now has real adapter and native service evidence, while Windows remains implementation and cross-compile only. The repository still lacks the remaining macOS retention and disruptive gates, Linux real-client and lifecycle promotion gates, and all real Windows gates. Therefore:

  • Keep the capability at fs-engine-preview.
  • Keep the public capability label at preview while allowing an operator to perform an explicit, externally quiesced real-home cutover.
  • Do not claim physical disk reclamation from logical deduplication alone.
  • Keep automatic enrollment bounded by mount health, writer state, stability, and storage budgets; no client-version approval is required.
  • Do not introduce a private control-plane dependency into any public surface.