perf(events): route event list to events_v2 + merge anonymous identit… (#424)

* perf(events): route event list to events_v2 + merge anonymous identity on profile page

Two changes to the profile/events event list (getEventList + getEventsCount):

1. Route reads to events_v2 (name-first ORDER BY) via getEventsTableForRange —
   parity with events within the TTL window, but profile_id / name IN (...)
   lookups prune to a fraction of the rows (measured 13x fewer for name+profile
   on the profile page). Falls back to events for pre-EVENTS_V2_MIN_DATE ranges
   or when the flag is off. Also skip the 0.5d expanding-window retries when a
   specific event name is searched — the name+profile sort-key pruning makes a
   single full-range pass cheap, replacing up to ~10 doubling re-scans.

2. Profile page identity merge: show the full pre-login + post-login journey by
   listing events for the canonical profile_id AND its anonymous device aliases.
   getProfileIdCluster resolves canonical (dictGet, in-RAM) then reverse-looks-up
   its aliases from profile_aliases (cached 10min, so the scan runs once per
   profile). getEventList/getEventsCount take a profileIds[] -> profile_id IN
   (...) with literal ids (no per-query alias subquery scan). Wired via a new
   mergeIdentity flag on the event.events procedure, set by the profile events
   route. Anon-only events ($ae_first_open, loginPrompt, logInInitiate, ...)
   are now visible on the profile timeline instead of split across device ids.

* fix(profile): resolve identity cluster via profile_aliases, not the alias dict

The forward step of getProfileIdCluster used dictGet, but the alias RAM dict
lags profile_aliases inserts (LIFETIME 30-60min). A freshly-aliased anon device
dictGet-MISSes on every replica, so opening that anon profile resolved to itself,
found no reverse aliases, and showed only the pre-login events — the post-login
half silently dropped.

Forward-resolve via a profile_aliases alias lookup instead — a cheap
(project_id, alias) sort-key point lookup that is authoritative and near
real-time. The reverse step already scans profile_aliases, so the resolver now
reads a single consistent source. Verified: opening anon d6021004 now resolves
to canonical X8JG and returns the full cluster.

* feat(profile-events): default to last 15 days with a visible range picker

Profile events feed now defaults to the last 15 days (like the profiles list,
#371) so landing on a busy profile stays fast, while searching a specific event
spans all days to find old occurrences.

- getEventList: when an explicit [startDate, endDate] is passed, that BETWEEN
  drives the window and the rolling expanding-window is skipped (only the cursor
  upper-bound remains for pagination) — a real range bounds the scan cleanly.
- Profile events route: default window = last 15 days; searching a specific event
  = all days; a user-picked range always wins. Memoized so the 'now' anchor is
  stable.
- EventsTable: swapped the old DateRangerPicker modal for the #371 LastSeenPicker
  dual-month calendar, added an emptyRangeLabel prop (profile page shows
  'Last 15 days' / 'All time' while searching) and a clear button.

* fix(profile-events): address CodeRabbit — datetime bounds + reverse-alias dedup

- getEventList/getEventsCount: compare created_at against toDateTime bounds
  instead of toDate, so the LastSeenPicker's hour-level ranges aren't widened to
  whole calendar days.
- getProfileIdCluster: the reverse alias lookup kept any alias whose profile_id
  row equalled the canonical, but a reassigned device retains its old row
  (profile_id is in the sort key) — so it could resurrect a device now owned by
  another user. Restrict to candidate aliases that ever pointed here, then keep
  only those whose latest argMax(profile_id, created_at) is still this canonical.

Author: ayushjhanwar-png

apps/start/src/components/events/table/index.tsx

@@ -1,11 +1,10 @@
 import { EventsFilters } from '@/components/events/filters/events-filters';
 import { FullPageEmptyState } from '@/components/full-page-empty-state';
+import { LastSeenPicker } from '@/components/profiles/last-seen-picker';
 import { Skeleton } from '@/components/skeleton';
-import { Button } from '@/components/ui/button';
 import { useDataTableColumnVisibility } from '@/components/ui/data-table/data-table-hooks';
 import { DataTableToolbarContainer } from '@/components/ui/data-table/data-table-toolbar';
 import { DataTableViewOptions } from '@/components/ui/data-table/data-table-view-options';
-import { pushModal } from '@/modals';
 import type { RouterInputs, RouterOutputs } from '@/trpc/client';
 import { cn } from '@/utils/cn';
 import type { IServiceEvent } from '@openpanel/db';
@@ -14,8 +13,8 @@ import type { Table } from '@tanstack/react-table';
 import { getCoreRowModel, useReactTable } from '@tanstack/react-table';
 import { useWindowVirtualizer } from '@tanstack/react-virtual';
 import type { TRPCInfiniteData } from '@trpc/tanstack-react-query';
-import { format } from 'date-fns';
-import { CalendarIcon, Loader2Icon } from 'lucide-react';
+import { format, parse } from 'date-fns';
+import { CalendarIcon, Loader2Icon, XIcon } from 'lucide-react';
 import { parseAsIsoDateTime, useQueryState } from 'nuqs';
 import { last } from 'ramda';
 import { memo, useCallback, useEffect, useMemo, useRef, useState } from 'react';
@@ -32,10 +31,15 @@ type Props = {
     >,
     unknown
   >;
+  // Label for the date button when no explicit range is picked. The profile page
+  // passes "Last 15 days" (its default feed window) / "All time" (while searching).
+  emptyRangeLabel?: string;
 };
 
 const LOADING_DATA = [{}, {}, {}, {}, {}, {}, {}, {}, {}] as IServiceEvent[];
 const ROW_HEIGHT = 40;
+// DB-format the LastSeenPicker reads/writes (naive, project timezone).
+const DB_FMT = 'yyyy-MM-dd HH:mm:ss';
 
 interface VirtualizedEventsTableProps {
   table: Table<IServiceEvent>;
@@ -265,7 +269,7 @@ const VirtualizedEventsTable = ({
   );
 };
 
-export const EventsTable = ({ query }: Props) => {
+export const EventsTable = ({ query, emptyRangeLabel }: Props) => {
   const { isLoading } = query;
   const columns = useColumns();
   const [expandedIds, setExpandedIds] = useState<Set<string>>(() => new Set());
@@ -333,7 +337,11 @@ export const EventsTable = ({ query }: Props) => {
 
   return (
     <>
-      <EventsTableToolbar query={query} table={table} />
+      <EventsTableToolbar
+        query={query}
+        table={table}
+        emptyRangeLabel={emptyRangeLabel}
+      />
       <VirtualizedEventsTable
         table={table}
         data={data}
@@ -358,40 +366,55 @@ export const EventsTable = ({ query }: Props) => {
 function EventsTableToolbar({
   query,
   table,
+  emptyRangeLabel,
 }: {
   query: Props['query'];
   table: Table<IServiceEvent>;
+  emptyRangeLabel?: string;
 }) {
   const [startDate, setStartDate] = useQueryState(
     'startDate',
     parseAsIsoDateTime,
   );
   const [endDate, setEndDate] = useQueryState('endDate', parseAsIsoDateTime);
+  const hasRange = !!(startDate && endDate);
 
   return (
     <div className="flex flex-col gap-2 mb-4">
       <DataTableToolbarContainer className="mb-0">
         <div className="flex flex-1 flex-wrap items-center gap-2">
           <EventListener onRefresh={() => query.refetch()} />
-          <Button
-            variant="outline"
-            size="sm"
-            icon={CalendarIcon}
-            onClick={() => {
-              pushModal('DateRangerPicker', {
-                onChange: ({ startDate, endDate }) => {
-                  setStartDate(startDate);
-                  setEndDate(endDate);
-                },
-                startDate: startDate || undefined,
-                endDate: endDate || undefined,
-              });
+          {/* Reuses the #371 dual-month "Last seen" calendar. DB-format strings
+              in/out; the events feed stores the range as ISO dates. */}
+          <LastSeenPicker
+            startDate={startDate ? format(startDate, DB_FMT) : null}
+            endDate={endDate ? format(endDate, DB_FMT) : null}
+            onApply={(start, end) => {
+              setStartDate(parse(start, DB_FMT, new Date()));
+              // "Since" mode returns a null end — roll to now so both bounds are
+              // set (the feed query treats [start, end] as an explicit window).
+              setEndDate(end ? parse(end, DB_FMT, new Date()) : new Date());
             }}
+            className="inline-flex h-8 items-center gap-2 rounded-md border bg-card px-3 text-sm font-medium transition-colors hover:bg-accent"
           >
-            {startDate && endDate
-              ? `${format(startDate, 'MMM d')} - ${format(endDate, 'MMM d')}`
-              : 'Date range'}
-          </Button>
+            <CalendarIcon className="size-4 text-muted-foreground" />
+            {hasRange
+              ? `${format(startDate!, 'MMM d')} - ${format(endDate!, 'MMM d')}`
+              : (emptyRangeLabel ?? 'Date range')}
+          </LastSeenPicker>
+          {hasRange && (
+            <button
+              type="button"
+              title="Clear date range"
+              onClick={() => {
+                setStartDate(null);
+                setEndDate(null);
+              }}
+              className="flex size-8 items-center justify-center rounded-md border bg-card text-muted-foreground transition-colors hover:bg-accent hover:text-foreground"
+            >
+              <XIcon className="size-4" />
+            </button>
+          )}
         </div>
         <DataTableViewOptions table={table} />
       </DataTableToolbarContainer>

apps/start/src/routes/_app.$organizationId.$projectId.profiles.$profileId._tabs.events.tsx

@@ -8,7 +8,15 @@ import { useTRPC } from '@/integrations/trpc/react';
 import { PAGE_TITLES, createProjectTitle } from '@/utils/title';
 import { useInfiniteQuery } from '@tanstack/react-query';
 import { createFileRoute } from '@tanstack/react-router';
+import { subDays } from 'date-fns';
 import { parseAsIsoDateTime, useQueryState } from 'nuqs';
+import { useMemo } from 'react';
+
+// Default the profile events feed to the last 15 days (like the profiles list,
+// #371) so landing on a busy profile stays fast. A user-picked range wins, and
+// when searching a specific event we span all days (no default window) so old
+// events are found.
+const DEFAULT_WINDOW_DAYS = 15;
 
 export const Route = createFileRoute(
   '/_app/$organizationId/$projectId/profiles/$profileId/_tabs/events',
@@ -33,14 +41,34 @@ function Component() {
   const [endDate] = useQueryState('endDate', parseAsIsoDateTime);
   const [eventNames] = useEventQueryNamesFilter();
   const columnVisibility = useReadColumnVisibility('events');
+
+  const isSearching = (eventNames?.length ?? 0) > 0;
+  // A user-picked range always wins (a lone start = "since" gets `now` as end).
+  // Otherwise: searching a specific event spans all days; the default feed is the
+  // last 15 days. Memoized so the "now" anchor is stable across renders.
+  const { rangeStart, rangeEnd } = useMemo(() => {
+    if (startDate) {
+      return { rangeStart: startDate, rangeEnd: endDate ?? new Date() };
+    }
+    if (isSearching) {
+      return { rangeStart: undefined, rangeEnd: undefined };
+    }
+    const now = new Date();
+    return { rangeStart: subDays(now, DEFAULT_WINDOW_DAYS), rangeEnd: now };
+  }, [startDate, endDate, isSearching]);
+
   const query = useInfiniteQuery(
     trpc.event.events.infiniteQueryOptions(
       {
         projectId,
         profileId,
+        // Show the full journey: identified profile + its anonymous device
+        // aliases merged into one timeline (pre-login events live under the
+        // anon device id, resolved via profile_aliases).
+        mergeIdentity: true,
         filters,
-        startDate: startDate || undefined,
-        endDate: endDate || undefined,
+        startDate: rangeStart,
+        endDate: rangeEnd,
         events: eventNames,
         columnVisibility: columnVisibility ?? {},
       },
@@ -51,5 +79,10 @@ function Component() {
     ),
   );
 
-  return <EventsTable query={query} />;
+  return (
+    <EventsTable
+      query={query}
+      emptyRangeLabel={isSearching ? 'All time' : 'Last 15 days'}
+    />
+  );
 }

packages/db/src/services/event.service.ts

@@ -13,6 +13,7 @@ import {
   chQuery,
   convertClickhouseDateToJs,
   formatClickhouseDate,
+  getEventsTableForRange,
 } from '../clickhouse/client';
 import { type Query, clix } from '../clickhouse/query-builder';
 import type { EventMeta, Prisma } from '../prisma-client';
@@ -436,6 +437,10 @@ export async function createEvent(payload: IServiceCreateEventPayload) {
 export interface GetEventListOptions {
   projectId: string;
   profileId?: string;
+  // Identity-merge: when set, list events for ALL of these profile_ids (a
+  // canonical id + its anonymous device aliases) so the profile page shows the
+  // full pre-login + post-login journey. Takes precedence over `profileId`.
+  profileIds?: string[];
   sessionId?: string;
   take: number;
   cursor?: number | Date;
@@ -454,6 +459,7 @@ export async function getEventList(options: GetEventListOptions) {
     take,
     projectId,
     profileId,
+    profileIds,
     sessionId,
     events,
     filters,
@@ -465,21 +471,44 @@ export async function getEventList(options: GetEventListOptions) {
   } = options;
   const { sb, getSql, join } = createSqlBuilder();
 
+  // Route reads to events_v2 (name-first ORDER BY) — parity with `events` within
+  // the retained TTL window, but a `profile_id` / `name IN (...)` lookup prunes to
+  // a fraction of the rows (measured 13x fewer for name+profile). Explicit ranges
+  // predating EVENTS_V2_MIN_DATE fall back to `events`; cursor-window reads anchor
+  // at "now" so they always resolve to events_v2.
+  const routeStartDate = startDate
+    ? formatClickhouseDate(startDate)
+    : formatClickhouseDate(new Date());
+  sb.from = `${getEventsTableForRange(routeStartDate)} e`;
+
   const MAX_DATE_INTERVAL_IN_DAYS = 365;
+  // When searching a *specific* event (name filter, not the "*" wildcard), the
+  // name+profile pruning on events_v2 makes a single full-range scan cheap — so
+  // skip the 0.5d expanding-window retries, which otherwise re-scan the whole
+  // project once per doubling step to reach an event that fired weeks ago.
+  const hasSpecificEventFilter =
+    !!events && events.length > 0 && !events.includes('*');
   // Cap the date interval to prevent infinity
-  const safeDateIntervalInDays = Math.min(
-    dateIntervalInDays,
-    MAX_DATE_INTERVAL_IN_DAYS,
-  );
+  const safeDateIntervalInDays = hasSpecificEventFilter
+    ? MAX_DATE_INTERVAL_IN_DAYS
+    : Math.min(dateIntervalInDays, MAX_DATE_INTERVAL_IN_DAYS);
+
+  // When the caller passes an explicit [startDate, endDate] (e.g. the profile
+  // page's default "last 15 days", or a user-picked range), that BETWEEN clause
+  // below bounds the scan — so skip the rolling expanding-window entirely and let
+  // the range drive it (only the cursor upper-bound is needed for pagination).
+  const hasExplicitRange = !!(startDate && endDate);
 
   if (typeof cursor === 'number') {
     sb.offset = Math.max(0, (cursor ?? 0) * take);
   } else if (cursor instanceof Date) {
-    sb.where.cursorWindow = `created_at >= toDateTime64(${sqlstring.escape(formatClickhouseDate(cursor))}, 3) - INTERVAL ${safeDateIntervalInDays} DAY`;
+    if (!hasExplicitRange) {
+      sb.where.cursorWindow = `created_at >= toDateTime64(${sqlstring.escape(formatClickhouseDate(cursor))}, 3) - INTERVAL ${safeDateIntervalInDays} DAY`;
+    }
     sb.where.cursor = `created_at <= ${sqlstring.escape(formatClickhouseDate(cursor))}`;
   }
 
-  if (!cursor) {
+  if (!cursor && !hasExplicitRange) {
     sb.where.cursorWindow = `created_at >= toDateTime64(${sqlstring.escape(formatClickhouseDate(new Date()))}, 3) - INTERVAL ${safeDateIntervalInDays} DAY`;
   }
 
@@ -595,7 +624,12 @@ export async function getEventList(options: GetEventListOptions) {
     sb.select.revenue = 'revenue';
   }
 
-  if (profileId) {
+  if (profileIds && profileIds.length > 0) {
+    sb.where.profileId = `profile_id IN (${join(
+      profileIds.map((id) => sqlstring.escape(id)),
+      ',',
+    )})`;
+  } else if (profileId) {
     sb.where.profileId = `profile_id = ${sqlstring.escape(profileId)}`;
   }
 
@@ -604,7 +638,7 @@ export async function getEventList(options: GetEventListOptions) {
   }
 
   if (startDate && endDate) {
-    sb.where.created_at = `toDate(created_at) BETWEEN toDate('${formatClickhouseDate(startDate)}') AND toDate('${formatClickhouseDate(endDate)}')`;
+    sb.where.created_at = `created_at BETWEEN toDateTime('${formatClickhouseDate(startDate)}') AND toDateTime('${formatClickhouseDate(endDate)}')`;
   }
 
   const selectedEventNames = events?.includes('*') ? [] : (events ?? []);
@@ -661,19 +695,30 @@ export const getEventsCountCached = cacheable(getEventsCount, 60 * 10);
 export async function getEventsCount({
   projectId,
   profileId,
+  profileIds,
   events,
   filters,
   startDate,
   endDate,
 }: Omit<GetEventListOptions, 'cursor' | 'take'>) {
   const { sb, getSql, join } = createSqlBuilder();
+  // Match getEventList: read from events_v2 for the same sort-key pruning.
+  const routeStartDate = startDate
+    ? formatClickhouseDate(startDate)
+    : formatClickhouseDate(new Date());
+  sb.from = `${getEventsTableForRange(routeStartDate)} e`;
   sb.where.projectId = `project_id = ${sqlstring.escape(projectId)}`;
-  if (profileId) {
+  if (profileIds && profileIds.length > 0) {
+    sb.where.profileId = `profile_id IN (${join(
+      profileIds.map((id) => sqlstring.escape(id)),
+      ',',
+    )})`;
+  } else if (profileId) {
     sb.where.profileId = `profile_id = ${sqlstring.escape(profileId)}`;
   }
 
   if (startDate && endDate) {
-    sb.where.created_at = `toDate(created_at) BETWEEN toDate('${formatClickhouseDate(startDate)}') AND toDate('${formatClickhouseDate(endDate)}')`;
+    sb.where.created_at = `created_at BETWEEN toDateTime('${formatClickhouseDate(startDate)}') AND toDateTime('${formatClickhouseDate(endDate)}')`;
   }
 
   const selectedEventNames = events?.includes('*') ? [] : (events ?? []);