-
Notifications
You must be signed in to change notification settings - Fork 31
Open
Description
Description
There is a memory leak in RedisKeysSourceTask
where the sourceOffsets
list continuously grows without ever being cleared, leading to memory exhaustion in long-running connector deployments.
Steps to Reproduce
- Deploy a Redis Keys Source Connector monitoring high-volume key changes
- Let it run for an extended period
- Observe memory usage continuously increasing
- Memory leak occurs due to
sourceOffsets.add()
calls incommitRecord()
with no corresponding cleanup
Expected Behavior
Memory usage should remain stable for long-running connectors.
Actual Behavior
Memory continuously increases due to unbounded list growth, eventually leading to OutOfMemoryError in production deployments.
Root Cause
commitRecord()
method adds offsets tosourceOffsets
list viaaddSourceOffset()
commit()
method is a no-op and never clears the list- The collected offsets are never used since Redis key monitoring doesn't require acknowledgment
Environment
- Component:
RedisKeysSourceTask
- Impact: Production deployments with high-volume key monitoring
Fix
I've submitted a fix in PR [#53] that removes the unused sourceOffsets
field and related code, eliminating the memory leak while preserving all functionality.
Metadata
Metadata
Assignees
Labels
No labels