File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -26,6 +26,37 @@ REDIS=redis://localhost:6379
2626REDIS_PREFIX=yhub
2727```
2828
29+ ### Eviction policy
30+
31+ Configure Redis / Valkey with the ` volatile-lru ` eviction policy:
32+
33+ ```
34+ maxmemory-policy volatile-lru
35+ ```
36+
37+ y/hub sets an expiry only on cached HTTP API responses. Update streams are
38+ written without a TTL, so ` volatile-lru ` reclaims memory from the cache while
39+ leaving the streams untouched.
40+
41+ Do ** not** use ` allkeys-lru ` , ` allkeys-lfu ` , or ` allkeys-random ` — these evict
42+ update streams, which causes ** data loss** .
43+
44+ ### Memory sizing
45+
46+ Redis is not just a cache — it is the authoritative store for updates that have
47+ not yet been persisted. Client updates are appended to a stream, and the worker
48+ only trims a stream after it has merged and written the document to S3 and
49+ PostgreSQL. Anything evicted or lost before that point is gone.
50+
51+ Size the instance so that all transient updates fit in memory at peak load,
52+ with headroom. If Redis cannot hold them, ` volatile-lru ` will start failing
53+ writes once the cache is exhausted (which surfaces as errors) rather than
54+ silently dropping streams — but the safe configuration is enough memory for the
55+ full working set.
56+
57+ Persistence (AOF/RDB) is recommended as well, so that a Redis restart does not
58+ discard updates that the worker has not yet persisted.
59+
2960---
3061
3162## 2. Set Up PostgreSQL
You can’t perform that action at this time.
0 commit comments