Skip to content

safety: gate property-MV route on explicit project allowlist - #358

Merged
ayushjhanwar-png merged 2 commits into
mainfrom
perf/chart-property-mv-safety-allowlist
Jul 7, 2026
Merged

safety: gate property-MV route on explicit project allowlist#358
ayushjhanwar-png merged 2 commits into
mainfrom
perf/chart-property-mv-safety-allowlist

Conversation

@ayushjhanwar-png

Copy link
Copy Markdown

Stacked on #356. Adds a project allowlist gate to canUsePropertyMV so the MV route only fires for projects with verified ≥90% identified traffic on their hot events.

Why this is needed

Both MVs (profile_event_property_summary_mv, profile_event_summary_mv) filter `profile_id != device_id` on write, so anonymous events are excluded. The identified/anonymous split is project-specific:

Project Event Total (7d) Anonymous Identified % Identified
dashreels showOpen (millions) 0.2% 99.8% 99.8%
dashreels show1Activated 169,915 140 169,775 99.9%
shortreels showOpen 2,659,905 2,241,881 418,024 15.7%
shortreels show1Activated 73,731 72,259 1,472 2%

Without this gate, PR #356 would silently under-count by ~6× on shortreels filtered charts. A `showOpen` breakdown chart on shortreels would show 15.7% of actual events. A `show1Activated` chart would show 2%.

What ships

Two-line hard gate:

```typescript
const MV_ROUTING_ALLOWED_PROJECTS = new Set(['dashreels']);
if (!MV_ROUTING_ALLOWED_PROJECTS.has(projectId)) return false;
```

Unknown projects fall through to the events-table path — always correct, sometimes slower.

Alternative considered

Runtime coverage probe: query MV vs `events_daily_stats` counts over last 7d, cache 24h in Redis per (project, event). Auto-adapts as new projects onboard. Rejected because:

  • ~60 lines vs 2
  • Adds CH probe on cache miss
  • Adds Redis dependency in the gate
  • Manual curation is fine when new projects onboard rarely

If the allowlist grows unwieldy (10+ projects, frequent additions), swap to the probe. Shape documented in `docs/conversion-chart-perf.md`.

Verified locally against prod CH

Same-shape chart on both projects (`showOpen` filtered by `properties.source`):

Project Range Route Elapsed
dashreels 7d MV 0.9 s
dashreels 3m MV 2.0 s
shortreels 7d events (safe fallback) 4.1 s
shortreels 3m events (safe fallback) 8.0 s

Onboarding a new project

Run the diagnostic SQL in `docs/conversion-chart-perf.md` per hot event. If `pct_identified ≥ 90`, add the project to `MV_ROUTING_ALLOWED_PROJECTS`.

Base branch

Based on `perf/chart-property-mv-routing` (#356). When #356 merges, this PR's base auto-updates to `main`. Cleaner review boundary vs. bundling the guard into #356 or #357.

Related

Test plan

  • `pnpm --filter db typecheck` — clean (only pre-existing `getCohortMembershipSubquery` error)
  • Local API smoke test against prod CH — dashreels routes to MV; shortreels falls through to events with correct numbers
  • Reviewer: verify `dashreels` is the only intended allowlisted project at this time
  • Reviewer: confirm the gate rejects unknown projects (fail-closed default)

@coderabbitai

coderabbitai Bot commented Jul 7, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0847a632-f599-4111-8da6-aabb3d5fecf2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch perf/chart-property-mv-safety-allowlist

Comment @coderabbitai help to get the list of available commands.

Both MVs the router uses filter `profile_id != device_id` on write, so
anonymous events are excluded. The identified/anonymous split is
project-specific:
  dashreels showOpen        → 99.8% identified (MV complete)
  shortreels showOpen       → 15.7% identified (MV misses 84%)
  shortreels show1Activated → 2%    identified (MV misses 98%)

Without a gate, a shortreels filtered/breakdown chart on showOpen
would silently under-count by ~6x. Gate on an explicit allowlist so
only projects with verified ≥90% identify rates on their hot events
route through the MV.

Considered a runtime coverage probe (mv_count / events_count over
last 7d) cached 24h in Redis. Auto-adapts per project but adds ~60
lines of code + a CH probe on cache miss + Redis dependency in the
gate. Allowlist is 2 lines; onboarding a new project is a one-liner
after running the diagnostic SQL. Trade-off documented in
docs/conversion-chart-perf.md — swap to the probe if the list grows
beyond ~10 projects.

Fail-closed default: unknown project → falls through to events-table
path (correct, sometimes slower).
The three console.log calls in getChartSqlFromPropertyMV matched the
existing getChartSqlFromMaterializedView pattern but bloat production
logs on every filter/breakdown chart request. CodeRabbit flagged this
as a potential PII leak (filter values printed verbatim). Removing
here rather than leaving inconsistent with the other builders — those
should follow up separately.
@ayushjhanwar-png
ayushjhanwar-png force-pushed the perf/chart-property-mv-safety-allowlist branch from e8f0785 to b425049 Compare July 7, 2026 12:49
@ayushjhanwar-png
ayushjhanwar-png changed the base branch from perf/chart-property-mv-routing to main July 7, 2026 12:49
@ayushjhanwar-png
ayushjhanwar-png merged commit 5f9a286 into main Jul 7, 2026
5 checks passed
ayushjhanwar-png added a commit that referenced this pull request Jul 16, 2026
)

* Revert "perf(db): extend property-MV router to breakdown-by-property charts (redo) (#359)"

This reverts commit 8213214.

* Revert "safety: gate property-MV route on explicit project allowlist (#358)"

This reverts commit 5f9a286.

* Revert "perf(db): route event-property-filter charts to profile_event_property_summary_mv (#356)"

This reverts commit a1d884f.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant