bench(indexer): add slow query benchmark script for world_id_registry_events#806
bench(indexer): add slow query benchmark script for world_id_registry_events#806agentotto[bot] wants to merge 8 commits into
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Adds scripts/seed_benchmark_db.sh that populates world_id_registry_events with ~10k realistic rows using a single INSERT ... SELECT with generate_series. Follows the same DATABASE_URL convention as the benchmark script. Uses ON CONFLICT DO NOTHING for idempotency. Updates README_benchmark.md with a 'Seeding test data' section.
|
Added What it does:
Also updated |
- Standalone compose file that starts Postgres + runs migrations via a one-shot service (no sqlx-cli or Rust toolchain needed) - Iterates over all migration files in services/indexer/migrations/ so new migrations are picked up automatically - Updated README_benchmark.md with end-to-end steps: start, seed, benchmark on both branches, compare, teardown
| SELECT 1 | ||
| FROM world_id_registry_events | ||
| WHERE event_type = 'root_recorded' | ||
| AND event_data->>'root' = 'BENCH_ROOT_PLACEHOLDER' |
There was a problem hiding this comment.
Query one uses literal placeholder
Medium Severity
Query 1’s root filter uses the fixed string BENCH_ROOT_PLACEHOLDER, which is never substituted. Seeded root_recorded rows store 0x-prefixed hashes, so the benchmark always exercises a non-matching lookup instead of the production root_exists path with a real root value.
Reviewed by Cursor Bugbot for commit 524a8ba. Configure here.
|
|
||
| for run in $(seq 1 $RUNS); do | ||
| echo " Running $NAME — iteration $run/$RUNS ..." | ||
| OUTPUT=$(psql "$DB_URL" -X -A -c "$SQL" 2>&1) || true |
There was a problem hiding this comment.
psql failures still exit zero
Low Severity
Each benchmark iteration runs psql with || true, so connection errors, missing tables, or bad SQL do not fail the script under set -e. The run can finish with exit code 0 and summary averages of N/A, which looks like a successful benchmark when nothing valid was measured.
Reviewed by Cursor Bugbot for commit 524a8ba. Configure here.
sed -E with \s is not supported by BSD awk on macOS, so when running
the script on macOS the sed substitution failed to strip the line down
to a bare number. The full 'Execution Time: X.XXX ms' string was then
interpolated into the downstream awk arithmetic expression, producing:
awk: syntax error at source line 1
context is
\tBEGIN {printf "%.3f", 0 + Execution >>> Time: <<<
Replace the grep|sed pipeline with a single awk invocation that both
matches the pattern and extracts field $3 — compatible with BSD awk
(macOS) and GNU awk (Linux).
| if [[ "$EXISTING" -gt 0 ]]; then | ||
| echo "" | ||
| echo "WARNING: world_id_registry_events already contains $EXISTING rows." | ||
| read -rp "Truncate the table before seeding? [y/N] " ANSWER |
There was a problem hiding this comment.
Reseed aborts without TTY
Low Severity
When the table already has rows, the script prompts with read while set -e is enabled. Non-interactive runs (no stdin) make read exit non-zero and terminate the script before the insert, even though the README says re-running is safe via ON CONFLICT DO NOTHING.
Reviewed by Cursor Bugbot for commit cddd035. Configure here.
Rewrites benchmark_slow_queries.sh to perform a complete before/after comparison in a single run: Phase 1 — Drop perf indexes, run baseline (slow) queries Phase 2 — Create perf indexes Phase 3 — Run optimized (fast) queries Phase 4 — Print side-by-side comparison with min/max/avg and speedup - Tracks min/max/avg for each query in each phase - Saves full EXPLAIN output and summary comparison to timestamped file - Uses portable awk approach (works on macOS BSD tools) - Keeps existing header info (timestamp, git branch/commit, database)
- generate_series range: 0..5999 → 8 × 0..59999 batches (480,000 total) - ~400,000 blocks kept (83% filter) × ~2.5 avg log entries ≈ 1,000,000 rows - Block range now [1_000_000 .. 1_479_999] - Batching into 8 × ~125,000-row inserts avoids large memory pressure - Updated header comment and inline comments to reflect new counts - Same data distribution, SQL structure, and macOS compatibility
PR #800 (fix/slow-queries-world-id-registry-events) no longer rewrites the paginated-fetch query as UNION ALL. Benchmarks showed the original OR query is already fast (~0.066 ms) via the primary key on (block_number, log_index), while the UNION ALL version was ~6000x slower because it prevents PG early termination with LIMIT. Script now benchmarks only the two queries that are actually affected by the indexes added in 0002_perf_indexes.sql: - Query 1: root existence check (partial expression index) - Query 3: insert (index maintenance overhead) A clear explanatory comment is added at the top of the script.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 4 total unresolved issues (including 3 from previous reviews).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 76910ef. Configure here.
| psql "$DB_URL" -X -c " | ||
| CREATE INDEX IF NOT EXISTS idx_world_id_registry_events_block_number_hash | ||
| ON world_id_registry_events (block_number, block_hash); | ||
| " 2>&1 | tee -a "$OUTFILE" |
There was a problem hiding this comment.
Aborted run leaves indexes dropped
Medium Severity
The script drops idx_world_id_registry_events_root and idx_world_id_registry_events_block_number_hash before the baseline phase and only recreates them in Phase 2. If the process exits or is interrupted between those steps, the database can remain without those indexes until someone fixes them manually.
Reviewed by Cursor Bugbot for commit 76910ef. Configure here.
Summary
Adds a benchmark script to measure query performance on the
world_id_registry_eventstable. This is a companion to PR #800 which introduces index and query changes to improve paginated event fetches.Queries benchmarked
event_typeand JSONevent_data->>'root'get_afterquery usingORget_afterquery from PR #800Queries 2 and 3 are the before/after versions of the same logical query so their results can be directly compared.
How to use
DATABASE_URL='postgres://user:pass@host:5432/dbname' ./scripts/benchmark_slow_queries.shEach query is run 5× with
EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT). Results are written tobenchmarks/results_<ISO_TIMESTAMP>.txtwith a summary of average execution times at the bottom.See
scripts/README_benchmark.mdfor full instructions including how to compare results across branches.Note
Low Risk
Dev-only scripts and compose file; no production indexer or query logic changes, aside from optional local DB index drop/create during benchmarks.
Overview
Adds a local benchmarking workflow for indexer
world_id_registry_eventsperformance (companion to PR #800), without changing application or migration code in this diff.docker-compose.benchmark.ymlspins up Postgres onindexer_testsand appliesservices/indexer/migrations/*.sqlvia a one-shotmigrateservice.scripts/seed_benchmark_db.shbulk-loads ~10M synthetic rows (batchedgenerate_series, mixedroot_recorded/identity_updatedJSONB).scripts/benchmark_slow_queries.shruns a single-run before/after: drops the two perf indexes from0002_perf_indexes.sql, benchmarks, recreates them, benchmarks again, and writesbenchmarks/results_<timestamp>.txt(gitignored).Measured workloads are root existence (
event_type+event_data->>'root') and a noop INSERT (index maintenance cost). Paginatedget_after/ UNION ALL is not run—the script documents that the OR path is already fast on the PK and UNION ALL was far slower in testing.scripts/README_benchmark.mddocuments Docker → seed → run/compare across branches.Reviewed by Cursor Bugbot for commit 76910ef. Bugbot is set up for automated code reviews on this repo. Configure here.