RFC: Scheduler scalability – Per-CPU runqueue + double_rq_lock mechanism like Linux

[AndyLiuQian]

Hi Zephyr community,
I have a question and RFC about scheduler scalability under SMP when CONFIG_SCHED_PINMASKONLY is enabled.
Currently, when SCHED_PINMASKONLY is enabled, Zephyr already uses per-CPU runqueues (each CPU has its own runqueue).
However, the scheduler still uses a single global scheduler lock to protect all runqueue operations.
This creates a scalability bottleneck on SMP systems:

1. All CPUs contend on the same global lock even when they operate on their own per-CPU runqueue
2. Cross-CPU migration, interrupts, wakeups, and context switches all block each other

In Linux, this problem is solved by:

1. Per-CPU runqueue (already exists in Zephyr when pinmaskonly enabled)
2. Per-CPU runqueue lock (one lock per CPU, not global)
3. double_rq_lock(): When cross-CPU migration is needed, lock both source and target runqueue locks in CPU order (smallest first) to avoid deadlock

My questions are:

1. Since we already have per-CPU runqueue under SCHED_PINMASKONLY,could we replace the global scheduler lock with per-CPU runqueue locks?

2. For cross-CPU task migration, can we implement a double_rq_lock() mechanism like Linux:

• Lock two runqueues in ascending CPU order to avoid AB-BA deadlock
• Safe for task migration, wakeups, and scheduler IPI

3. Would this design be acceptable in Zephyr?What are the potential blocking issues (memory, real-time, latency, etc)?

I think this would greatly improve SMP scalability without breaking existing functionality,
since per-CPU runqueues are already there.
Looking forward to your thoughts!