rcu: Add long-term CPU kicking
Commit d2db185bfee8 ("rcu: Remove short-term CPU kicking") removed
frequent calls to resched_cpu(), which means that the only time
resched_cpu() is invoked is after an RCU CPU stall warning. Although
this is good from an avoid-IPIs perspective, we should try to break
things loose -before- splatting. This commit therefore starts invoking
resched_cpu() for each holdout at each force-quiescent-state interval
that is more than halfway through the stall-warning interval.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index b37699b..a874c95 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -1346,6 +1346,13 @@ static int rcu_implicit_dynticks_qs(struct rcu_data *rdp,
rdp->rsp->jiffies_resched += 5; /* Re-enable beating. */
}
+ /*
+ * If more than halfway to RCU CPU stall-warning time, do
+ * a resched_cpu() to try to loosen things up a bit.
+ */
+ if (jiffies - rdp->rsp->gp_start > rcu_jiffies_till_stall_check() / 2)
+ resched_cpu(rdp->cpu);
+
return 0;
}
