rcu: Make tiny RCU use ksoftirqd to trigger a QS from idle
The commit:
cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup")
fixed an issue where rcutiny would request a quiescent state with
setting TIF_NEED_RESCHED in early boot when init/0 has the PF_IDLE flag
but interrupts aren't enabled yet. An subsequent call to cond_resched()
would then re-enable IRQs too early.
However the call to resched_cpu() can opportunistically be replaced with
raising RCU_SOFTIRQ and forcing ksoftirqd wakeup. If callbacks are
involved within the current grace period, this is what RCU wants and
does currently:
1) Trigger exit from idle and go through the scheduler to call
rcu_note_context_switch() -> rcu_qs()
2) rcu_qs() notes the quiescent state and raises RCU_SOFTIRQ if there
is a callback, waking up ksoftirqd since it isn't called from an
interrupt.
This all can be optimized within a single call to
raise_ksoftirqd_irqsoff(). RCU grace period polling while idle is then
suboptimized but such a usecase can be considered very rare or even
non-existent.
The advantage of this optimization is that it also works if PF_IDLE is
set early because ksoftirqd is only created once IRQs are enabled on
boot and it can't be awaken before its creation. If
raise_ksoftirqd_irqsoff() is called after the first scheduling point
but before kostfirqd is created, nearby voluntary schedule calls are
expected and in the worst case the first launch of ksoftirqd is close
enough during the first initcalls.
Fixes: cff9b2332ab7 ("kernel/sched: Modify initial boot task idle setup")
Cc: Liam R. Howlett <Liam.Howlett@oracle.com>
Cc: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
1 file changed
