sched,rcu: Make cond_resched() provide RCU quiescent state
There is some confusion as to which of cond_resched() or
cond_resched_rcu_qs() should be added to long in-kernel loops.
This commit therefore eliminates the decision by adding RCU
quiescent states to cond_resched().
Warning: This is a prototype. For example, it does not correctly
handle Tasks RCU. Which is OK for the moment, given that no one
actually uses Tasks RCU yet.
Reported-by: Michal Hocko <mhocko@kernel.org>
Not-yet-signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 348f51b..ccdb606 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -3308,10 +3308,11 @@ static inline int signal_pending_state(long state, struct task_struct *p)
* cond_resched_lock() will drop the spinlock before scheduling,
* cond_resched_softirq() will enable bhs before scheduling.
*/
+void rcu_all_qs(void);
#ifndef CONFIG_PREEMPT
extern int _cond_resched(void);
#else
-static inline int _cond_resched(void) { return 0; }
+static inline int _cond_resched(void) { rcu_all_qs(); return 0; }
#endif
#define cond_resched() ({ \
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 94732d1..40b6908 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4906,6 +4906,7 @@ int __sched _cond_resched(void)
preempt_schedule_common();
return 1;
}
+ rcu_all_qs();
return 0;
}
EXPORT_SYMBOL(_cond_resched);
