dropped internal Raft message since sending buffer is full (overloaded network)

[rahulbapumore]

Hi @ahrtr
We are facing below error messages and etcd is restarting because it could not maintain quorum.
2025-03-18T12:23:53.018+01:00 dropped internal Raft message since sending buffer is full (overloaded network) 2025-03-18T12:23:53.050+01:00 dropped internal Raft message since sending buffer is full (overloaded network) 2025-03-18T12:23:53.094+01:00 dropped internal Raft message since sending buffer is full (overloaded network) 2025-03-18T12:23:53.099+01:00 dropped internal Raft message since sending buffer is full (overloaded network) 2025-03-18T12:23:53.100+01:00 dropped internal Raft message since sending buffer is full (overloaded network) 2025-03-18T12:23:53.101+01:00 dropped internal Raft message since sending buffer is full (overloaded network) 2025-03-18T12:23:53.102+01:00 dropped internal Raft message since sending buffer is full (overloaded network) 2025-03-18T12:23:53.195+01:00 dropped internal Raft message since sending buffer is full (overloaded network) 2025-03-18T12:23:53.294+01:00 dropped internal Raft message since sending buffer is full (overloaded network)

From ETCD documentation , we found that its happening because too many client requests creating congestion in network, delaying peer communication.
https://etcd.io/docs/v3.5/tuning/

There are few manual steps given to set traffic priority, But we need some internal solution/WA like some parameter if that set then we wont see any restarts in ETCD

Could you please help with query?

Thanks in advance

[rahulbapumore]

Hi @ahrtr @serathius
Could you please help with this?

[rahulbapumore]

Hi @ahrtr @serathius
Can increasing defaultWriteBufSize for grpc server will solve this issue?
This values is 32 * 1024 inside etcd .