[FEATURE] Queue Stats

[proost]

Summary

Adding a Queue stats to rueidis client. The stats tells a status of queue.

Problem Definition

Background

I’ve already propose ring stats before(#537). Situation changed and It has become worth considering again with different reason.

Primary reason: issue

queue impacts computation resource usage of client and behavior of client. Now, we can tune queue usingRingScaleEachConn option and RUEIDIS_QUEUE_TYPE env.

but tune of queue is not easy. because observability of queue is poor. So we provide stats of queue . So user can tune the queue more predictable way.

Goals

We can monitor how many commands is waiting in queue.

Non-Goals

Now, we don’t design how to integrate stats with rueidisotel

Design

Queue

type queue interface {
	...
	
	WaitingCmdCount() uint32
}

Add WaitingCmdCount method to queue . WaitingCmdCount tells how many commands are waiting in queue. The new method ruturn waits

ring and flowBuffer has a new atomic.Uint32 type field waits . when new commands(or command) are added to queue from PutOne or PutMulti, waits become waits+1. When FinishResultmethod is called waits become waits -1.

Atomic operation is cheap, but makes additional overhead always. So if a user doesn’t want collecting stats overhead, user turn off collecting stats. So ring and flowBuffer has a new bool type field collectStats . collectStats is from constructor. if collectStats is false, do not operate on the waits .

Conn

type conn interface {
	...
	
	WaitingCommandInQueueStats() waitingCommandInQueueStats
}

type waitingCommandInQueueStats struct {
	totalWaitingCommandCount uint64
	queueCount               uint32
	minWaitingCommandCount   uint32
	maxWaitingCommandCount   uint32
}

mux can have multiple muxwire . So we should provide stats on multiple queues. Now we provide 4 metrics. maxWaitingCommandCount and average command count from totalWaitingCommandCount and queueCount tells how to tune queue is better.

Client

type ClientStatsProvider interface {
	Stats() ClientStats
}

type ClientStats struct {
	WaitingCommandsInQueueStats WaitingCommandInQueueStats
}

type WaitingCommandInQueueStats struct {
	TotalWaitingCommandCount   uint64
	AverageWaitingCommandCount float32
	MinWaitingCommandCount     uint32
	MaxWaitingCommandCount     uint32
}

Adding a new interface ClientStatsProvider not to happen breaking change int Client interface. If user wants to get ClientStats, user should type check to the Client.

singleClient, sentinelClient, clusterClient implement `ClientStatsProvider.

sentinelClient aggregates stats from mConn and rConn calling WaitingCommandInQueueStats.

clusterClient aggregates stats from conns calling WaitingCommandInQueueStats.

Overall operation is cheap, but overhead is inevitable. So as I mentioned above, If want to collecting stats, user configures explicitly. So we introduce a new client option.

type ClientOption struct {
	...	
	
	CollectingClientStatsOption CollectingClientStatsOption
}

// CollectingClientStatsOption defines options for enabling the collecting client statistics.
type CollectingClientStatsOption struct {
	// EnableCollectingQueueStats determines whether to enable the collection of queue-related statistics.
	EnableCollectingQueueStats bool
}

Maybe later, More metrics can be added too. So we define a new struct about client stats option.

Alternatives Considered

I’ve consideredwrCounter using stats. pros of wrCounter use no additional computation resource and overhead. But wrCounter increase even though commands are not entered to queue. So wrCounter is not accurate for stats.