qspinlock: Split set_locked() into its own atomic step
The old sp21 label collapsed set_locked(), the wait for node->next, and
arch_mcs_spin_unlock_contended() into a single atomic action. That hid
the contended *,*,0 -> *,*,1 transition: the window where, after the
queue head sets the locked byte but before it hands off the MCS lock, a
concurrent locker can observe the lock held with the tail still
populated and queue behind us.
Move the set_locked() store into sp21 and the wait/handoff into a new
sp22 label so TLC interleaves other CPUs in that window.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
1 file changed