Feature Request: Add built-in connection creation rate limiter with idle-retry capability for burst traffic
Background and Problem
Currently, when burst traffic arrives and all idle connections are exhausted, the connection pool allows all pending requests to initiate TCP dialing concurrently, as long as TotalConns < PoolSize. This causes a connection creation storm:
- Hundreds of goroutines dial Redis at the same instant, causing TCP handshake spikes on both client and server side.
- In scenarios where connection creation cost (e.g. ~10ms per dial) is much higher than per-request execution cost (e.g. ~1ms), creating large numbers of connections is far less efficient than waiting briefly and reusing existing connections.
- The pool fills to
PoolSize almost immediately, even for very short traffic spikes, after which most connections sit idle and eventually expire, creating periodic destroy-rebuild cycles.
Why a custom Dialer is not sufficient
The common community workaround is wrapping Dialer with a rate.Limiter, but this approach has a critical limitation:
Once a request passes the idle-check and enters the "new connection" branch, it will never go back to check for newly returned idle connections, even while waiting for a dial token. This means:
- Even if another request returns a connection to the idle pool a millisecond later, the waiting request will still create a brand new connection, wasting the opportunity for reuse.
- The rate limiter only slows down connection creation, it cannot redirect requests back to the idle queue, which is the more efficient path for short-lived requests.
This is suboptimal for workloads where reuse is cheaper than creation.
Proposed Solution
Add a built-in connection creation rate limiter to the connection pool with the following behavior:
- Add a new option, e.g.
DialRateLimit / DialRateBurst, to control the maximum number of new connections created per second.
- When idle connections are exhausted and the pool would normally create a new connection, first acquire a token from the rate limiter.
- If a token is not immediately available, instead of blocking in the dial path, requeue the request to wait for an idle connection (the same
waitForConn path used when TotalConns == PoolSize).
- When a connection is returned to the idle pool, wake up waiting requests as usual; if a waiting request can take the idle connection, it never needs to dial at all.
- Only if the request waits longer than some threshold (or if idle connections remain unavailable) does it proceed to create a new connection.
In short: when dialing would be throttled, prefer waiting for reuse over waiting to create.
Expected Benefits
- Smooths out connection creation spikes during traffic bursts, eliminating dial storms.
- Significantly improves overall efficiency for short-request workloads, where reuse is much cheaper than creation.
- Reduces unnecessary connection churn caused by short traffic spikes.
- Avoids the "can't go back to idle queue" limitation of custom Dialer-based rate limiting.
- Remains fully backward compatible: disabled by default, only active when explicitly configured.
Additional Considerations
- The implementation should work with both single-node and cluster clients.
- The limiter should interact correctly with existing
PoolTimeout semantics.
- Metrics (
PoolStats) should expose rate-limited dial attempts for observability.
- This complements the existing
ConnMaxLifetimeJitter feature: jitter prevents mass-expiration storms, while this feature prevents mass-creation storms.
Thank you for considering this feature. I'd be happy to discuss implementation details or contribute a draft PR if the maintainers think this is worthwhile.
Feature Request: Add built-in connection creation rate limiter with idle-retry capability for burst traffic
Background and Problem
Currently, when burst traffic arrives and all idle connections are exhausted, the connection pool allows all pending requests to initiate TCP dialing concurrently, as long as
TotalConns < PoolSize. This causes a connection creation storm:PoolSizealmost immediately, even for very short traffic spikes, after which most connections sit idle and eventually expire, creating periodic destroy-rebuild cycles.Why a custom Dialer is not sufficient
The common community workaround is wrapping
Dialerwith arate.Limiter, but this approach has a critical limitation:Once a request passes the idle-check and enters the "new connection" branch, it will never go back to check for newly returned idle connections, even while waiting for a dial token. This means:
This is suboptimal for workloads where reuse is cheaper than creation.
Proposed Solution
Add a built-in connection creation rate limiter to the connection pool with the following behavior:
DialRateLimit/DialRateBurst, to control the maximum number of new connections created per second.waitForConnpath used whenTotalConns == PoolSize).In short: when dialing would be throttled, prefer waiting for reuse over waiting to create.
Expected Benefits
Additional Considerations
PoolTimeoutsemantics.PoolStats) should expose rate-limited dial attempts for observability.ConnMaxLifetimeJitterfeature: jitter prevents mass-expiration storms, while this feature prevents mass-creation storms.Thank you for considering this feature. I'd be happy to discuss implementation details or contribute a draft PR if the maintainers think this is worthwhile.