rcu: Report quiescent state at IRQ exit to reduce GP latency
When a CPU exits from an IRQ/NMI that interrupted an RCU-idle period,
proactively report a quiescent state before transitioning back to
RCU-idle. This allows grace periods to complete faster by not waiting
for the next FQS scan to detect the dyntick counter change.
Previously, when a CPU was in dyntick-idle state and received an
interrupt, the grace period machinery had to wait for an FQS scan to
notice the CPU's dyntick counter had changed. By calling rcu_qs() at
IRQ exit (via the new rcu_nmi_exit_qs() wrapper), the CPU can
immediately report its quiescent state if one is needed.
The rcu_qs() function has an internal early-return check that avoids
any work if no quiescent state is needed, making this addition
lightweight for the common case.
Benchmark results on 32-CPU VM show:
- Mean latency: unchanged (~6.04ms)
- Tail latency (max): reduced ~14% (11.1ms -> 9.6ms average)
Signed-off-by: Joel Fernandes <joelaf@google.com>
4 files changed
