tree 6ef9b4b216e87f0f6ce75cf499ce16e37c6455b7
parent 0188171d757596637de6ba02762c94a27518905a
author Alexei Starovoitov <ast@kernel.org> 1743800209 -0700
committer Alexei Starovoitov <ast@kernel.org> 1746069555 -0700

slab: Introduce kmalloc_nolock() and kfree_nolock().

kmalloc_nolock() relies on ability of local_lock to detect the situation
when it's locked.
In !PREEMPT_RT local_lock_is_locked() is true only when NMI happened in
irq saved region that protects _that specific_ per-cpu kmem_cache_cpu.
In that case retry the operation in a different kmalloc bucket.
The second attempt will likely succeed, since this cpu locked
different kmem_cache_cpu.
When lock_local_is_locked() sees locked memcg_stock.stock_lock
fallback to atomic operations.

Similarly, in PREEMPT_RT local_lock_is_locked() returns true when
per-cpu rt_spin_lock is locked by current task. In this case re-entrance
into the same kmalloc bucket is unsafe, and kmalloc_nolock() tries
a different bucket that is most likely is not locked by current
task. Though it may be locked by a different task it's safe to
rt_spin_lock() on it.

Similar to alloc_pages_nolock() the kmalloc_nolock() returns NULL
immediately if called from hard irq or NMI in PREEMPT_RT.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>