Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,17 @@ follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]

### Added
- Cache-server auth scoping (v0.5 theme, item 1): `python -m
hashloom.cache_server` takes an optional `--publish-token` (or
`HASHLOOM_CACHE_PUBLISH_TOKEN`). When set, publishes (POST) require it and
reads (GET) accept either token — publish implies read — so CI writes
greens and a laptop with the read token can only consume them. A valid
read token on a publish route gets 403 `read_only` (vs 401 `unauthorized`
for an unknown token). Back-compat: with only `--token`, that token grants
both roles, byte-identical to before. Client side, `.hashloom/config.json`
`shared` takes an optional `"publish": false` for read-only clients to
skip publish requests entirely (a rejected publish was already a swallowed
no-op). Per-project/team tokens remain deferred.
- Strict provenance mode (#49): opt-in `.hashloom/config.json`
`{"strict_provenance": true}`. `verify` refuses any unit whose dependency
closure contains a `status: inferred` contract — a structured
Expand Down
10 changes: 7 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,9 +169,13 @@ every teammate's agent gets `cached-pass`:

Run the backend anywhere with `python -m hashloom.cache_server` (a small
bearer-token HTTP service over a SQLite store; an operational process, not a CLI
command). Only greens are published — failures never cross the boundary — and the
toolchain-in-key rule above keeps a shared green sound across machines. If the
shared store is unreachable, verify degrades silently to local.
command). Auth is scoped: give the server an optional `--publish-token` and only
holders of it can write greens (CI publishes, laptops with the read token only
consume; a single token still grants both roles, as before). A read-only client
can set `"publish": false` to skip publish requests entirely. Only greens are
published — failures never cross the boundary — and the toolchain-in-key rule
above keeps a shared green sound across machines. If the shared store is
unreachable, verify degrades silently to local.

## CLI

Expand Down
7 changes: 5 additions & 2 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,11 @@ this is the prioritization.
v0.2 made the cache shareable; v0.5 makes sharing it safe at team scale. The
remaining hard parts from [docs/hosted-store.md](docs/hosted-store.md):

- **Auth scoping** — split publish from read: CI can write greens, a laptop can
only consume them. Today one bearer token does both.
- **Auth scoping** — ✓ **Shipped** (unreleased): the cache server takes an
optional `--publish-token`; reads accept either token (publish implies read),
publishes require the publish token — CI writes greens, a laptop with the
read token only consumes. A single token still grants both roles, so existing
deployments are unchanged. Per-project/team tokens remain deferred.
- **Concurrent writers** — the cache server is single-threaded-serialised;
real teams need atomic verdict/blob writes under concurrency (CAS or
equivalent), not politeness.
Expand Down
25 changes: 19 additions & 6 deletions docs/hosted-store.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,15 @@ and `RemoteStore` makes the shared side an HTTP backend (item 1 below and
shared-store outage degrades silently to local verify (the local path stays the
default). Single-threaded for now; concurrency stays in #4. Server-side `ran_at`
stamping (the server's own `SqliteStore`) already covers #6.
2. **Auth** *(partially shipped)*. A single shared bearer token gates the server
(constant-time `hmac.compare_digest`). Still deferred: a *stronger* check on the
publish path than the read path, and per-project/team tokens.
2. **Auth** *(read/publish split shipped)*. Bearer tokens gate the server
(constant-time `hmac.compare_digest`), and the roles are now split by verb:
`--token` grants reads; an optional `--publish-token` is required for the
publish routes (POST), and implies read. With only `--token`, that one token
does both — the original deployment shape, unchanged. A read token on a
publish route is 403 `read_only`; an unknown token is 401 `unauthorized`.
Clients can also declare themselves read-only (`"publish": false` in the
shared config) and skip publish requests entirely. Still deferred:
per-project/team tokens.
3. ✓ **Trust: toolchain in the key (shipped).** A shared stale-green is worse than
a solo one, which is why test source is already in the verification key (#18):
a verdict is only as portable as its key is complete. The key now also folds in
Expand Down Expand Up @@ -76,14 +82,21 @@ The shared backend is an operational process, **not** a `hashloom` subcommand (t
5-CLI surface is fixed):

```bash
python -m hashloom.cache_server --db cache.db --token SECRET # --host/--port optional
python -m hashloom.cache_server --db cache.db --token READ_SECRET --publish-token CI_SECRET
```

It refuses to start without a token (`--token` or `HASHLOOM_CACHE_TOKEN`) and binds
`127.0.0.1` by default. Point each developer's `.hashloom/config.json` at it:
`127.0.0.1` by default. `--publish-token` (or `HASHLOOM_CACHE_PUBLISH_TOKEN`) is
optional — omit it and the one token grants both roles. CI gets the publish token;
a developer laptop gets the read token and, optionally, opts out of doomed
publish attempts:

```json
{ "shared": { "url": "http://cache.host:8770", "token": "SECRET" } }
{ "shared": { "url": "http://cache.host:8770", "token": "CI_SECRET" } }
```

```json
{ "shared": { "url": "http://cache.host:8770", "token": "READ_SECRET", "publish": false } }
```

Then `hashloom verify` (and the MCP `verify` tool) publish greens to, and read them
Expand Down
70 changes: 58 additions & 12 deletions src/hashloom/cache_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,17 @@
developer's `.hashloom/config.json` `{"shared": {...}}` at one of these, so a unit
verified green once is served to everyone (see docs/hosted-store.md).

Auth is scoped by verb: GET is a read, POST is a publish. With only `--token`,
that one token does both (the original single-token deployment). Add
`--publish-token` to split the roles: reads accept either token (publish
implies read), publishes require the publish token -- so CI writes greens and
a laptop with the read token can only consume them. A read token on a publish
route is 403 `read_only`; an unrecognized token is 401 `unauthorized`.

It is intentionally NOT a `hashloom` subcommand (the 5-CLI surface is fixed); run it
as a separate operational process:

python -m hashloom.cache_server --db cache.db --token SECRET [--host H --port P]
python -m hashloom.cache_server --db cache.db --token SECRET [--publish-token SECRET2] [--host H --port P]

Single-threaded by design: a `SqliteStore` holds one sqlite connection (not
thread-safe), and the verdict/blob writes are idempotent upserts, so
Expand All @@ -34,12 +41,19 @@


class CacheServer(HTTPServer):
"""An HTTPServer that holds the store and expected token for the handler."""

def __init__(self, addr: tuple[str, int], store: SqliteStore, token: str):
"""An HTTPServer that holds the store and expected token(s) for the handler."""

def __init__(
self,
addr: tuple[str, int],
store: SqliteStore,
token: str,
publish_token: str | None = None,
):
super().__init__(addr, _Handler)
self.store = store
self.token = token
self.publish_token = publish_token # None -> `token` gates both verbs


class _Handler(BaseHTTPRequestHandler):
Expand All @@ -60,12 +74,27 @@ def _send(self, status: int, body: dict | None = None) -> None:
def _error(self, status: int, code: str, message: str) -> None:
self._send(status, {"error": {"code": code, "message": message}})

def _authed(self) -> bool:
def _presented(self) -> str | None:
header = self.headers.get("Authorization", "")
prefix = "Bearer "
if not header.startswith(prefix):
return header[len(prefix):] if header.startswith(prefix) else None

def _authed(self, publish: bool = False) -> bool:
"""Constant-time check against the token(s) the verb accepts.

Reads accept the read token or the publish token (publish implies
read); publishes require the publish token when one is configured.
Both comparisons always run — no short-circuit string equality.
"""
presented = self._presented()
if presented is None:
return False
return hmac.compare_digest(header[len(prefix):], self.server.token)
is_read = hmac.compare_digest(presented, self.server.token)
pub = self.server.publish_token
is_publish = pub is not None and hmac.compare_digest(presented, pub)
if publish and pub is not None:
return is_publish
return is_read or is_publish

def _read_body(self) -> dict | None:
length = int(self.headers.get("Content-Length") or 0)
Expand Down Expand Up @@ -94,7 +123,11 @@ def do_GET(self) -> None:
return self._error(404, "not_found", "unknown route")

def do_POST(self) -> None:
if not self._authed():
if not self._authed(publish=True):
# a valid read token on a publish route is a scope problem, not an
# identity problem — tell the client which one it has
if self._authed():
return self._error(403, "read_only", "publishing requires the publish token")
return self._error(401, "unauthorized", "missing or invalid bearer token")
store = self.server.store
body = self._read_body()
Expand All @@ -117,9 +150,15 @@ def do_POST(self) -> None:
return self._error(404, "not_found", "unknown route")


def serve(db: str, token: str, host: str = "127.0.0.1", port: int = DEFAULT_PORT) -> None:
def serve(
db: str,
token: str,
host: str = "127.0.0.1",
port: int = DEFAULT_PORT,
publish_token: str | None = None,
) -> None:
store = SqliteStore(db, check_same_thread=False) # used on the serve_forever thread
httpd = CacheServer((host, port), store, token)
httpd = CacheServer((host, port), store, token, publish_token=publish_token)
print(f"hashloom cache server on http://{host}:{httpd.server_address[1]} (db: {db})", file=sys.stderr)
try:
httpd.serve_forever()
Expand All @@ -141,12 +180,19 @@ def main(argv: list[str] | None = None) -> int:
p.add_argument(
"--token",
default=os.environ.get("HASHLOOM_CACHE_TOKEN"),
help="bearer token clients must present (or set HASHLOOM_CACHE_TOKEN)",
help="bearer token clients must present (or set HASHLOOM_CACHE_TOKEN); "
"with no --publish-token it grants reads and publishes both",
)
p.add_argument(
"--publish-token",
default=os.environ.get("HASHLOOM_CACHE_PUBLISH_TOKEN"),
help="optional second token required to publish (or set HASHLOOM_CACHE_PUBLISH_TOKEN); "
"when set, --token becomes read-only and this token grants reads and publishes",
)
args = p.parse_args(argv)
if not args.token:
p.error("a --token (or HASHLOOM_CACHE_TOKEN env var) is required; refusing to run an unauthenticated cache")
serve(args.db, args.token, host=args.host, port=args.port)
serve(args.db, args.token, host=args.host, port=args.port, publish_token=args.publish_token)
return 0


Expand Down
5 changes: 4 additions & 1 deletion src/hashloom/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,4 +143,7 @@ def resolve_shared_store(root: Path) -> dict | None:
raise HashloomError("bad_config", "shared.url must be a non-empty string")
if not isinstance(token, str) or not token:
raise HashloomError("bad_config", "shared.token must be a non-empty string")
return {"url": url, "token": token}
publish = cfg.get("publish", True) # false: read-only client, publishes are skipped
if not isinstance(publish, bool):
raise HashloomError("bad_config", f"shared.publish must be true or false, got {publish!r}")
return {"url": url, "token": token, "publish": publish}
13 changes: 10 additions & 3 deletions src/hashloom/remote.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,14 +28,18 @@


class RemoteStore:
def __init__(self, url: str, token: str, timeout: float = 5.0):
def __init__(self, url: str, token: str, timeout: float = 5.0, publish: bool = True):
self._base = url.rstrip("/")
self._token = token
self._timeout = timeout
# a read-only client (`shared.publish: false`) skips publishes outright
# instead of collecting one rejected POST per verdict/blob; a *rejected*
# publish was already a swallowed no-op, this just saves the round trip
self._publish = publish

@classmethod
def from_config(cls, cfg: dict) -> RemoteStore:
return cls(cfg["url"], cfg["token"])
return cls(cfg["url"], cfg["token"], publish=cfg.get("publish", True))

# -- the four team-portable methods -------------------------------------

Expand All @@ -44,6 +48,8 @@ def get_verification(self, key: str) -> dict | None:
return body if st == 200 and isinstance(body, dict) else None

def record_verification(self, key: str, contract_name: str, status: str, summary: str) -> None:
if not self._publish:
return
# fire-and-forget; _request swallows any transport error. The layer has
# already written the verdict locally, so a publish failure loses nothing.
self._request("POST", "/verification", {
Expand All @@ -55,7 +61,8 @@ def get_blob(self, blob_hash: str) -> str | None:
return body.get("content") if st == 200 and isinstance(body, dict) else None

def put_blob(self, content: str) -> str:
self._request("POST", "/blob", {"content": content})
if self._publish:
self._request("POST", "/blob", {"content": content})
# the layer uses the *local* hash; return ours regardless of the network
return hashlib.sha256(content.encode("utf-8")).hexdigest()

Expand Down
105 changes: 103 additions & 2 deletions tests/test_remote_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,103 @@ def test_auth_rejected_but_verify_still_degrades(cache, tmp_path):
local.close()


# -- auth scoping: split publish from read ----------------------------------

_PUBLISH_TOKEN = "publish-token"


@pytest.fixture
def scoped_cache(tmp_path):
"""A cache_server with split read/publish tokens; yields (base_url, server_store)."""
store = SqliteStore(tmp_path / "cache.db", check_same_thread=False)
httpd = CacheServer(("127.0.0.1", 0), store, _TOKEN, publish_token=_PUBLISH_TOKEN)
port = httpd.server_address[1]
thread = threading.Thread(target=httpd.serve_forever, daemon=True)
thread.start()
try:
yield f"http://127.0.0.1:{port}", store
finally:
httpd.shutdown()
thread.join(timeout=5)
httpd.server_close()
store.close()


def test_read_token_reads_but_cannot_publish(scoped_cache):
base, _ = scoped_cache
assert _raw("GET", base, _TOKEN, "/verification/whatever") == 404 # authed, no such row
assert _raw("POST", base, _TOKEN, "/blob", {"content": "x"}) == 403
assert _raw("POST", base, _TOKEN, "/verification",
{"key": "k", "contract_name": "c", "status": "pass", "summary": ""}) == 403


def test_publish_token_implies_read(scoped_cache):
base, store = scoped_cache
assert _raw("GET", base, _PUBLISH_TOKEN, "/verification/whatever") == 404 # reads too
assert _raw("POST", base, _PUBLISH_TOKEN, "/blob", {"content": "x"}) == 200
assert _raw("POST", base, _PUBLISH_TOKEN, "/verification",
{"key": "k", "contract_name": "c", "status": "pass", "summary": ""}) == 204
assert store.get_verification("k")["status"] == "pass"


def test_unknown_token_is_401_on_both_verbs(scoped_cache):
base, _ = scoped_cache
assert _raw("GET", base, "wrong-token", "/verification/whatever") == 401
assert _raw("POST", base, "wrong-token", "/blob", {"content": "x"}) == 401


def test_single_token_server_still_grants_both_roles(cache):
# back-compat pin: with no publish token configured, --token publishes too
base, token = cache
assert _raw("POST", base, token, "/blob", {"content": "x"}) == 200


def test_ci_publishes_laptop_consumes(scoped_cache, tmp_path):
base, server_store = scoped_cache
root = tmp_path / "proj"
root.mkdir()
_make_project(root)

# CI: publish token. verify runs pytest once and publishes the green.
ci_local = SqliteStore(root / ".hashloom" / "ci.db")
ci = LayeredStore(ci_local, RemoteStore(base, _PUBLISH_TOKEN))
index(root, ci)
assert api.verify(root, ci, ["total"])["results"][0]["status"] == "pass"

# laptop: read token. the green is served over HTTP without pytest.
lap_local = SqliteStore(root / ".hashloom" / "lap.db")
lap = LayeredStore(lap_local, RemoteStore(base, _TOKEN))
index(root, lap)
assert api.verify(root, lap, ["total"])["results"][0]["status"] == "cached-pass"
assert lap_local.counters().get("test_runs", 0) == 0

# the laptop's own new green stays local: its publish is rejected (403),
# swallowed, and verify still passes
(root / "src" / "x.py").write_text("def total(xs):\n return sum(xs) + 0\n")
r = verify_one(root, lap, "total")
assert r["status"] == "pass"
assert server_store.get_verification(r["key"]) is None
ci_local.close()
lap_local.close()


def test_publish_false_client_skips_posts_entirely(cache, tmp_path):
base, token = cache
root = tmp_path / "proj"
root.mkdir()
_make_project(root)
# even with a fully-capable token, publish=False never POSTs
local = SqliteStore(root / ".hashloom" / "x.db")
store = LayeredStore(local, RemoteStore(base, token, publish=False))
index(root, store)
r = verify_one(root, store, "total")
assert r["status"] == "pass"
assert RemoteStore(base, token).get_verification(r["key"]) is None # nothing landed
blob_hash = local.get_impl("total")["blob_hash"]
assert RemoteStore(base, token).get_blob(blob_hash) is None # blobs suppressed too
local.close()


def test_resolve_shared_store_validation(tmp_path):
init_project(tmp_path)
cfg = tmp_path / ".hashloom" / "config.json"
Expand All @@ -169,12 +266,16 @@ def test_resolve_shared_store_validation(tmp_path):
assert resolve_shared_store(tmp_path) is None # no shared block -> local only

cfg.write_text(json.dumps({"shared": {"url": "http://h", "token": "t"}}))
assert resolve_shared_store(tmp_path) == {"url": "http://h", "token": "t"}
assert resolve_shared_store(tmp_path) == {"url": "http://h", "token": "t", "publish": True}

cfg.write_text(json.dumps({"shared": {"url": "http://h", "token": "t", "publish": False}}))
assert resolve_shared_store(tmp_path)["publish"] is False

for bad in ({"shared": "nope"},
{"shared": {"token": "t"}}, # missing url
{"shared": {"url": "http://h"}}, # missing token
{"shared": {"url": "", "token": "t"}}): # empty url
{"shared": {"url": "", "token": "t"}}, # empty url
{"shared": {"url": "http://h", "token": "t", "publish": "no"}}): # non-bool publish
cfg.write_text(json.dumps(bad))
with pytest.raises(HashloomError) as e:
resolve_shared_store(tmp_path)
Expand Down
Loading