Skip to content

Commit 9390ef0

Browse files
connortsui20claude
andauthored
chore: delete the v3 tree now that v3 is decommissioned (#7)
The v3 generation was fully decommissioned on 2026-07-08: monorepo PR vortex-data/vortex#8683 removed the v3 ingest steps and made the v4 Postgres ingest required, and the EC2 host, its S3 backups, and its IAM/SG resources were deleted. This removes the code that served it: server/ (the axum ingest/read server), migrate/ (the v2->v3->v4 migrator, whose backfill job finished on 2026-07-07), ops/ (the host deploy scripts), the Cargo workspace, the Rust CI workflow, and the v3 runbooks. The measurement_id hash contract the Rust server owned moves to the monorepo: the golden vectors and a Python pin test now live there (vortex-data/vortex#8687), so scripts/measurement_id_golden.json is deleted here and web/lib/schema-version.test.ts drops its server/src/schema.rs anchor, making web/lib/schema-version.ts the in-repo source of truth. Docs are updated to describe the single v4 ingest path and to mark the v3 sections as historical. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 065478c commit 9390ef0

110 files changed

Lines changed: 281 additions & 33737 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/rust-ci.yml

Lines changed: 0 additions & 73 deletions
This file was deleted.

AGENTS.md

Lines changed: 47 additions & 92 deletions
Original file line numberDiff line numberDiff line change
@@ -5,82 +5,51 @@ SPDX-FileCopyrightText: Copyright the Vortex contributors
55

66
# AGENTS.md - `benchmarks-website/`
77

8-
Read [`README.md`](README.md) first for the architecture. Then this file. The
9-
root [`CLAUDE.md`](../CLAUDE.md) covers Rust style, test layout, commit
10-
conventions. Everything about the legacy generations (v2, v3) is in
11-
[`docs/legacy.md`](docs/legacy.md).
12-
13-
## The legacy v2 site
14-
15-
The v2 Vite/React site is retired from serving (`bench.vortex.dev` now points at
16-
v4) and was deployed from elsewhere — its source never lived in this repo. The
17-
migrator still reads the v2 S3 dump as its historical source (see
18-
[`migrate/README.md`](migrate/README.md)).
19-
20-
The v3 deploy lives entirely under `server/`, `migrate/`, and `ops/`.
21-
The operator runbook is [`ops/README.md`](ops/README.md).
22-
23-
## v3 specifics
24-
25-
- **Wire shapes are a coordinated change.** [`server/src/records.rs`](server/src/records.rs),
26-
[`vortex-bench/src/v3.rs`](../vortex-bench/src/v3.rs), and (until cutover)
27-
[`migrate/src/classifier.rs`](migrate/src/classifier.rs) must agree.
28-
Bumping a shape means changing all three plus the snapshot fixtures in
29-
one commit. `SCHEMA_VERSION` is the version literal coupled across
30-
[`server/src/schema.rs`](server/src/schema.rs) (in-repo source of
31-
truth), [`web/lib/schema-version.ts`](web/lib/schema-version.ts) (the
32-
in-repo web mirror), and [`scripts/post-ingest.py`](../scripts/post-ingest.py)
33-
(the monorepo CI ingest wrapper, which hardcodes it as a Python literal); a
34-
bump also coordinates with the monorepo producer wire shape in
35-
`vortex-bench/src/v3.rs`. Bump in lockstep or every CI ingest run 400s/409s
36-
(400 if the server is ahead, 409 if the emitter is ahead). The server-side validation in
37-
`records.rs` + `ingest.rs` and the echo in `/health` all consume the
38-
constant through `crate::schema`. The full versioned contract lives in
39-
[`CONTRACT.md`](CONTRACT.md).
40-
- **Numeric `?n=` is clamped to 1000; `?n=all` is the uncapped escape
41-
hatch.** HTML routes hydrate from the materialized latest-100 shard
42-
artifact by default; `?n=all` is an explicit opt-in
43-
(chart-init.js's full-history zoom-out hop uses it once, and curl
44-
power users can request it). The numeric `?n=` path is bounded by
45-
`MAX_NUMERIC_COMMIT_WINDOW` in [`server/src/api/window.rs`](server/src/api/window.rs)
46-
as a DoS-protection floor against `curl ...?n=99999999`. If you need
47-
full history, use `?n=all`. Do NOT raise the numeric cap or remove it
48-
without thinking about the DoS surface.
49-
- **`measurement_id` is server-internal.** Never put it on the wire. It is
50-
a deterministic hash over `commit_sha` plus the dim tuple, computed in
51-
[`server/src/db.rs`](server/src/db.rs) and reused by the migrator via
52-
the same crate.
53-
- **Don't write a server-side classifier for live ingest.** The emitter
54-
produces v3-shape records directly; the migrator's classifier only
55-
exists to translate v2 records once and goes away after cutover.
56-
- **Don't reach for WASM.** SSR + a thin hydration script in
57-
[`server/static/chart-init.js`](server/static/chart-init.js) is the
58-
whole client.
59-
- **v3 ingest is no longer best-effort in CI.** The `Ingest results to v3
60-
server` step in [`bench.yml`](../.github/workflows/bench.yml),
61-
[`sql-benchmarks.yml`](../.github/workflows/sql-benchmarks.yml), and
62-
[`v3-commit-metadata.yml`](../.github/workflows/v3-commit-metadata.yml)
63-
no longer carries `continue-on-error: true`. A v3-server outage on a
64-
develop push now fails the bench workflow and triggers the existing
65-
`incident.io` alert. The gate is `vars.V3_INGEST_URL != ''` so forks
66-
and unconfigured environments are unaffected.
67-
- **Don't re-introduce a server-side commit cap on `?n=all`.** `?n=all`
68-
is the uncapped escape hatch (chart-init.js fetches it once for the
69-
zoom-out path); visual downsampling happens client-side via LTTB on
70-
the visible commit range only. Numeric `?n=` is clamped per the bullet
71-
above. Default fetches from chart-init.js use the materialized
72-
latest-100 shard artifact, not `?n=all`.
73-
- **Don't refetch on every scope change.** Once a chart's payload is in
74-
memory, pan/zoom/slider/range-strip all rebuild in place via the
75-
in-memory LTTB pass on the cached payload. The single exception is the
76-
latest-100 to full-history zoom-out path: charts initially hydrate from
77-
the materialized latest-100 group shard artifact (served from
78-
`/api/artifacts/{generation}/groups/{slug}/shards/{i}`); when the user
79-
zooms past that window for the first time, `chart-init.js` lazy-fetches
80-
`?n=all` once and replaces the latest-100 payload in place.
8+
Read [`README.md`](README.md) first for the architecture. Then this file. The web app has its own
9+
deep-dive in [`web/README.md`](web/README.md).
10+
11+
## The legacy generations
12+
13+
The previous site generations (v2 and v3) are documented in [`docs/legacy.md`](docs/legacy.md).
14+
The v3 Rust tree (`server/`, `migrate/`, `ops/`, the Cargo workspace) has been deleted from the
15+
working tree; recover it from git history if ever needed. Do not resurrect it piecemeal — the
16+
rules below are the parts that were worth keeping.
17+
18+
## Load-bearing rules
19+
20+
- **`SCHEMA_VERSION` is a coordinated cross-repo change.**
21+
[`web/lib/schema-version.ts`](web/lib/schema-version.ts) is the in-repo source of truth; the
22+
monorepo's `scripts/post-ingest.py` hardcodes the same integer, and its `vortex-bench/src/v3.rs`
23+
owns the producer wire shape. Bump in lockstep across both repos. The versioned contract lives
24+
in [`CONTRACT.md`](CONTRACT.md), and
25+
[`web/lib/schema-version.test.ts`](web/lib/schema-version.test.ts) asserts the in-repo anchors
26+
agree.
27+
- **`measurement_id` is internal — never on the wire.** It is a deterministic xxhash64 over
28+
`commit_sha` plus the dim tuple, computed by the monorepo's `scripts/_measurement_id.py` just
29+
before INSERT and used as the `ON CONFLICT` upsert key. Its output is frozen forever: the
30+
monorepo's golden-vector test (`scripts/tests/test_measurement_id.py`) pins it, because a
31+
drifted hash would insert duplicates next to the rows already in production.
32+
- **Numeric `?n=` is clamped to 1000; `?n=all` is the uncapped escape hatch.** Pages hydrate from
33+
the materialized latest-100 shard artifact by default; `?n=all` is an explicit opt-in (the
34+
chart's full-history zoom-out hop uses it once, and curl power users can request it). The
35+
numeric path is bounded by `MAX_NUMERIC_COMMIT_WINDOW` in
36+
[`web/lib/window.ts`](web/lib/window.ts) as a DoS-protection floor against `?n=99999999`. Do
37+
NOT raise the numeric cap or remove it without thinking about the DoS surface, and do NOT
38+
re-introduce a server-side commit cap on `?n=all` — visual downsampling happens client-side via
39+
LTTB on the visible commit range only.
40+
- **Don't refetch on every scope change.** Once a chart's payload is in memory,
41+
pan/zoom/slider/range-strip all rebuild in place via the in-memory LTTB pass on the cached
42+
payload ([`web/components/Chart.tsx`](web/components/Chart.tsx)). The single exception is the
43+
latest-100 to full-history zoom-out path: when the user first zooms past the materialized
44+
window, the chart lazy-fetches `?n=all` once and replaces the payload in place.
45+
- **Don't write a server-side classifier for live ingest.** The emitter produces structured
46+
records directly. Classifying loose name strings at read time was the v2-era weakness every
47+
later generation existed to escape; it belongs nowhere in the live pipeline.
8148

8249
## Footguns we have already hit
8350

51+
All in [`web/components/Chart.tsx`](web/components/Chart.tsx):
52+
8453
- **Reverse predecessor walk in the tooltip.** `payload.commits[]` is
8554
sorted oldest-first by SQL - `commits[0]` is the oldest, `commits[N-1]`
8655
is the newest. For per-row delta the predecessor of `commits[idx]` is
@@ -96,25 +65,11 @@ The operator runbook is [`ops/README.md`](ops/README.md).
9665
## Local dev
9766

9867
```bash
99-
# Public-only run (read API + ingest only, admin routes 404):
100-
INGEST_BEARER_TOKEN=dev cargo run -p vortex-bench-server
68+
# The web app needs BENCH_DB_* env vars for a real database — see web/README.md.
69+
cd web && pnpm install && pnpm dev
10170

102-
# With admin endpoints mounted on a separate loopback listener:
103-
INGEST_BEARER_TOKEN=dev ADMIN_BEARER_TOKEN=dev \
104-
cargo run -p vortex-bench-server
105-
106-
cargo nextest run -p vortex-bench-server -p vortex-bench-migrate
107-
INSTA_UPDATE=auto cargo nextest run -p vortex-bench-server # update snapshots
71+
pnpm test # vitest unit tests
72+
pnpm build # deliberately works without a database, so CI needs no secrets
10873
```
10974

110-
For the full env-var contract (admin bind, snapshot dir, extension dir,
111-
logging spec, PaaS `PORT` fallback) see [`ops/config/vortex-bench.env.example`](ops/config/vortex-bench.env.example)
112-
and the lib-level `//!` doc on [`server/src/main.rs`](server/src/main.rs).
113-
114-
For the migrator end-to-end against the real S3 dump:
115-
116-
```bash
117-
cargo run -p vortex-bench-migrate -- run --output ./bench.duckdb
118-
VORTEX_BENCH_DB=./bench.duckdb INGEST_BEARER_TOKEN=dev \
119-
cargo run -p vortex-bench-server
120-
```
75+
For the full env-var contract see [`web/README.md`](web/README.md).

0 commit comments

Comments
 (0)