Skip to content

Commit 9bb713a

Browse files
kerneltoastbmastbergen
authored andcommitted
crypto: rng - Fix double percpu offset in local_unlock() calls
local_unlock() is supposed to take a percpu pointer, which it then offsets for the current CPU. The problem is that the per-CPU DRBG code passes local_unlock() a pointer that has *already been offset for the current CPU*, so the percpu offset gets added twice. `pcri` is the percpu pointer and `cri` is the offset `pcri` pointer for the current CPU; passing `cri` to local_unlock() results in the current CPU's offset getting added again and thus the resulting pointer is garbage. This went unnoticed at runtime because local_unlock() only dereferences that pointer on PREEMPT_RT or CONFIG_DEBUG_LOCK_ALLOC=y kernels. The pointer is never actually used otherwise. This also went unnoticed at compile time because, for x86_64, __percpu only expands to something on GCC 14+; __percpu is otherwise an empty macro. So there's no pointer type mismatch detected at compile time on older GCC versions. On GCC 14+, the following compile error occurs: ../crypto/rng.c: In function 'lock_default_rng': ../include/linux/percpu-defs.h:221:45: error: initialization from pointer to non-enclosed address space Fix it by using __local_unlock() instead, which eliminates the duplicate per-CPU offset addition. While using local_unlock() with `pcri` also works, using `cri` with __local_unlock() is leaner because it doesn't need to redo the offset addition, saving an instruction on kernels that actually use the pointer. Signed-off-by: Sultan Alsawaf <sultan@ciq.com>
1 parent d4027d1 commit 9bb713a

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

crypto/rng.c

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -338,7 +338,7 @@ lock_default_rng(struct crypto_rng **rng) __acquires(&cri->lock)
338338
*
339339
* Note that this task may be migrated to a different CPU now!
340340
*/
341-
local_unlock(&cri->lock);
341+
__local_unlock(&cri->lock);
342342
rt_mutex_lock(&cri->mlock);
343343
if (!cri->rng) {
344344
struct crypto_rng *new_rng = NULL;
@@ -411,7 +411,7 @@ do { \
411411
if (reseed) \
412412
rt_mutex_unlock(&(cri)->mlock); \
413413
else \
414-
local_unlock(&(cri)->lock); \
414+
__local_unlock(&(cri)->lock); \
415415
} while (0)
416416

417417
static __always_inline void

0 commit comments

Comments
 (0)