perf(db): explicit column list in profile-buffer CH fetch (drop SELECT *)

[ayushjhanwar-png]

Summary

Replaces `SELECT *` with an explicit column list in `profile-buffer.ts` `fetchFromClickhouse`.

Why

This query fires per-event during /track ingestion when the Redis profile cache misses. At ~25K events/min steady load, the per-query cost multiplies.

`SELECT *` currently loads:

• Materialized columns (`os`, `campaign`, `quotaPlan`) — unused by downstream merge/cache
• Full `properties` Map (retained, needed for merge)

The explicit column list keeps everything downstream uses (`id, project_id, first_name, last_name, email, avatar, is_external, properties, created_at`) and drops the materialized-column reads plus their serialization cost.

Context — today's /track 5xx spike (2026-07-16 06:41 UTC)

Post-backfill cluster contention slowed profile queries from ~50ms to 15-20s each. With per-event lookups at 25K/min, concurrent queries ballooned past ClickHouse's `max_concurrent_queries=447` cap on replica -37 → INSERTs rejected with Code 202 → API returned 5xx to /track.

This PR is a small step toward reducing per-query cost. It does NOT fix the underlying architecture (per-event lookup vs batched). Full upstream port to batch fetch is planned as a separate PR.

Follow-ups (separate PRs)

• Full port from upstream openpanel — batch fetch via `(id, project_id) IN (...)` tuples with `argMax(col, created_at) GROUP BY id, project_id`. Reduces CH profile queries by ~50-100× under peak load.
• Extend Redis TTL from 1h → 6h for active profiles (reduces cache miss rate).
• Skip lookup for anonymous events (profile_id === device_id).

Test plan

• Merge triggers dev deploy
• Verify profile lookups still succeed (spot-check dashboard profile pages)
• Confirm `_firstSeen` events still fire for new profiles
• Confirm profile merges still work (test /identify flow end-to-end)

Summary by CodeRabbit

• Performance Improvements
  • Reduced unnecessary profile data retrieved from storage, improving query efficiency and minimizing payload size.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a15d030f-2793-43fb-99e1-d7da3e8e8d07

📥 Commits

Reviewing files that changed from the base of the PR and between bc42f57 and 30f7cbd.

📒 Files selected for processing (1)

• packages/db/src/buffers/profile-buffer.ts

---

📝 Walkthrough

Walkthrough

The profile buffer’s ClickHouse query now selects only the profile columns used by the application, replacing SELECT * and documenting the reduced payload.

Changes

Profile query optimization

Layer / File(s)	Summary
Explicit profile column selection packages/db/src/buffers/profile-buffer.ts	fetchFromClickhouse replaces SELECT * with an explicit list of identifiers, profile attributes, properties, and creation metadata, with comments explaining the narrower payload.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main performance change: replacing SELECT * with an explicit column list in profile-buffer ClickHouse fetches.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/profile-buffer-select-star-optimization

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}