linux/kernel/rcu
Paul E. McKenney 127e29815b rcu: Make rcu_barrier() account for offline no-CBs CPUs
Currently, rcu_barrier() ignores offline CPUs,  However, it is possible
for an offline no-CBs CPU to have callbacks queued, and rcu_barrier()
must wait for those callbacks.  This commit therefore makes rcu_barrier()
directly invoke the rcu_barrier_func() with interrupts disabled for such
CPUs.  This requires passing the CPU number into this function so that
it can entrain the rcu_barrier() callback onto the correct CPU's callback
list, given that the code must instead execute on the current CPU.

While in the area, this commit fixes a bug where the first CPU's callback
might have been invoked before rcu_segcblist_entrain() returned, which
would also result in an early wakeup.

Fixes: 5d6742b377 ("rcu/nocb: Use rcu_segcblist for no-CBs CPUs")
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
[ paulmck: Apply optimization feedback from Boqun Feng. ]
Cc: <stable@vger.kernel.org> # 5.5.x
2020-03-21 16:14:25 -07:00
..
Kconfig rcu: Use CONFIG_PREEMPTION where appropriate 2019-12-09 12:37:51 -08:00
Kconfig.debug rcu: Add support for consolidated-RCU reader checking 2019-08-09 11:00:35 -07:00
Makefile rcu: Provide debug symbols and line numbers in KCSAN runs 2020-02-20 15:58:21 -08:00
rcu_segcblist.c rcu: Remove dead code from rcu_segcblist_insert_pend_cbs() 2020-02-20 15:58:23 -08:00
rcu_segcblist.h rcu: Remove kfree_rcu() special casing and lazy-callback handling 2020-01-24 10:24:31 -08:00
rcu.h rcu: Warn on for_each_leaf_node_cpu_mask() from non-leaf 2020-02-20 15:58:21 -08:00
rcuperf.c rcu: Remove kfree_call_rcu_nobatch() 2020-01-24 10:24:31 -08:00
rcutorture.c Merge branches 'doc.2019.12.10a', 'exp.2019.12.09a', 'fixes.2020.01.24a', 'kfree_rcu.2020.01.24a', 'list.2020.01.10a', 'preempt.2020.01.24a' and 'torture.2019.12.09a' into HEAD 2020-01-24 10:37:27 -08:00
srcutiny.c rcu: Use CONFIG_PREEMPTION where appropriate 2019-12-09 12:37:51 -08:00
srcutree.c rcu: Fix typos in file-header comments 2020-02-20 15:58:22 -08:00
sync.c rcu/sync: Simplify the state machine 2019-05-28 09:05:23 -07:00
tiny.c rcu: Remove kfree_rcu() special casing and lazy-callback handling 2020-01-24 10:24:31 -08:00
tree_exp.h rcu: Add WRITE_ONCE to rcu_node ->exp_seq_rq store 2020-02-20 15:58:22 -08:00
tree_plugin.h rcu: Make nocb_gp_wait() double-check unexpected-callback warning 2020-02-20 15:58:23 -08:00
tree_stall.h rcu: Don't flag non-starting GPs before GP kthread is running 2020-02-20 15:58:23 -08:00
tree.c rcu: Make rcu_barrier() account for offline no-CBs CPUs 2020-03-21 16:14:25 -07:00
tree.h Merge branches 'doc.2019.12.10a', 'exp.2019.12.09a', 'fixes.2020.01.24a', 'kfree_rcu.2020.01.24a', 'list.2020.01.10a', 'preempt.2020.01.24a' and 'torture.2019.12.09a' into HEAD 2020-01-24 10:37:27 -08:00
update.c Merge branches 'doc.2019.12.10a', 'exp.2019.12.09a', 'fixes.2020.01.24a', 'kfree_rcu.2020.01.24a', 'list.2020.01.10a', 'preempt.2020.01.24a' and 'torture.2019.12.09a' into HEAD 2020-01-24 10:37:27 -08:00