Cap the purpose meter tag to prevent unbounded cardinality

[vanrogu]

Summary

Adds a configurable cap on the number of distinct purpose tag values that event store meters can report, preventing unbounded metric cardinality growth when purpose is used as an entity identifier (e.g., customer ID).

Problem

Every meter the store registers is tagged with context and purpose. When purpose is used as documented in examples — forContext("customer").withPurpose("123") — it takes one value per entity. Since Micrometer never evicts meters, the cost grows with every distinct purpose the process has ever seen:

• ~5.5 KB of heap per distinct purpose
• 15 meters per purpose (plus 2 more per additional event type)
• 18 Prometheus series and ~2.4 KB of scrape body per purpose

At 100,000 customers this extrapolates to ~550 MB of heap, 1.8M series, and a 234 MB scrape — nothing fails, but the process gets heavier for as long as it runs.

Solution

Introduces MeterOptions to cap how many distinct purposes get their own meter tag value before the rest are pooled under _other:

• Default cap: 1000 purposes per store (configurable via MeterOptions)
• Admission is first-come-first-served and permanent — a purpose that gets a tag value keeps it for the store's lifetime, so dashboards built on those series remain stable
• Purposes beyond the cap are pooled under _other with a one-time warning log
• Events are still counted correctly — pooled purposes contribute to the _other series
• The cap is applied at one point (purposeTagValueFor) so it bounds everything downstream: per-stream meters, the eventtype cross product, and the store's internal gauge state map

Key Changes

• New MeterOptions class (sliceworkz-eventstore-api):

  • withMaxPurposeTagValues(int) — cap at a specific number
  • withoutPurposeBreakdown() — pool every purpose (cap = 0)
  • withUnlimitedPurposeTagValues() — no cap (for low-cardinality purposes by construction)
  • defaults() — returns cap of 1000
  • Validates that cap is non-negative

• Updated EventStoreImpl (sliceworkz-eventstore-impl):

  • New three-argument constructor accepting MeterOptions
  • Two-argument constructor delegates to three-argument with MeterOptions.defaults()
  • purposeTagValueFor(String) method implements the capping logic with CAS-based admission
  • Tracks admitted purposes in a ConcurrentHashMap and count in AtomicInteger
  • One-time warning logged when cap is first exceeded

• Updated EventStoreFactory (sliceworkz-eventstore-api):

  • New three-argument eventStore() method accepting MeterOptions
  • Default implementation delegates to two-argument for backward compatibility

• Updated storage builders (in-memory, PostgreSQL):

  • Added meterOptions() builder method
  • Passed through to buildStore() which creates the EventStoreImpl

• Comprehensive test suite (MeterPurposeCardinalityTest):

  • Verifies purposes below cap keep their own tag value
  • Verifies purposes beyond cap are pooled under _other
  • Confirms pooled purposes still count their events
  • Confirms admitted purposes are not demoted after cap is reached
  • Verifies internal gauge state map is also bounded
  • Tests withoutPurposeBreakdown() behavior
  • Tests default behavior for unconfigured stores
  • Tests concurrent first-use of distinct purposes holds the cap exactly
  • Tests wildcard and default purposes are treated as ordinary values

• Documentation (CLAUDE.md):

  • Added "Metrics: what the stream meters cost, and the cap on purpose" section
  • Explains the cost per distinct purpose, why the cap exists, and how to configure it

Implementation Details

https://claude.ai/code/session_019wmQNmjGJvQXnoFFPDReQk