linux/kernel/rcu
Paul E. McKenney f1fd0e0bb1 rcuscale: Count outstanding callbacks per-task rather than per-CPU
The current rcu_scale_writer() asynchronous grace-period testing uses a
per-CPU counter to track the number of outstanding callbacks.  This is
subject to CPU-imbalance errors when tasks migrate from one CPU to another
between the time that the counter is incremented and the callback is
queued, and additionally in kernels configured such that callbacks can
be invoked on some CPU other than the one that queued it.

This commit therefore arranges for per-task callback counts, thus avoiding
any issues with migration of either tasks or callbacks.

Reported-by: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: "Paul E. McKenney" <paulmck@kernel.org>
Signed-off-by: Neeraj Upadhyay <neeraj.upadhyay@kernel.org>
2024-08-15 00:14:48 +05:30
..
Kconfig rcu: Create NEED_TASKS_RCU to factor out enablement logic 2024-04-15 11:29:48 +02:00
Kconfig.debug rcu: Restrict access to RCU CPU stall notifiers 2023-12-12 02:31:22 +05:30
Makefile rcuperf: Change rcuperf to rcuscale 2020-08-24 18:39:24 -07:00
rcu_segcblist.c rcu: Use rcu_segcblist_segempty() instead of open coding it 2023-10-04 17:33:18 +02:00
rcu_segcblist.h rcu: Throttle callback invocation based on number of ready callbacks 2023-01-03 17:28:34 -08:00
rcu.h rcu: Provide rcu_barrier_cb_is_done() to check rcu_barrier() CBs 2024-08-14 16:49:56 +05:30
rcuscale.c rcuscale: Count outstanding callbacks per-task rather than per-CPU 2024-08-15 00:14:48 +05:30
rcutorture.c rcutorture: Stop testing RCU Tasks Rude asynchronous APIs 2024-08-14 16:46:31 +05:30
refscale.c refscale: Optimize process_durations() 2024-08-14 17:06:01 +05:30
srcutiny.c srcu: Fill out polled grace-period APIs 2024-06-18 10:13:37 -07:00
srcutree.c Merge branches 'doc.2024.06.06a', 'fixes.2024.07.04a', 'mb.2024.06.28a', 'nocb.2024.06.03a', 'rcu-tasks.2024.06.06a', 'rcutorture.2024.06.06a' and 'srcu.2024.06.18a' into HEAD 2024-07-04 13:54:17 -07:00
sync.c rcu: Eliminate lockless accesses to rcu_sync->gp_count 2024-07-04 13:48:57 -07:00
tasks.h rcu/tasks: Add rcu_barrier_tasks*() start time to diagnostics 2024-08-14 17:03:18 +05:30
tiny.c rcu: Make Tiny RCU explicitly disable preemption 2024-04-15 11:29:48 +02:00
tree_exp.h rcu/exp: Remove redundant full memory barrier at the end of GP 2024-06-28 06:44:12 -07:00
tree_nocb.h rcu/nocb: Remove buggy bypass lock contention mitigation 2024-06-03 17:29:15 -07:00
tree_plugin.h Merge branches 'doc.2024.06.06a', 'fixes.2024.07.04a', 'mb.2024.06.28a', 'nocb.2024.06.03a', 'rcu-tasks.2024.06.06a', 'rcutorture.2024.06.06a' and 'srcu.2024.06.18a' into HEAD 2024-07-04 13:54:17 -07:00
tree_stall.h rcu: Remove full memory barrier on RCU stall printout 2024-06-28 06:44:12 -07:00
tree.c rcu: Mark callbacks not currently participating in barrier operation 2024-08-14 17:06:01 +05:30
tree.h Merge branches 'doc.2024.06.06a', 'fixes.2024.07.04a', 'mb.2024.06.28a', 'nocb.2024.06.03a', 'rcu-tasks.2024.06.06a', 'rcutorture.2024.06.06a' and 'srcu.2024.06.18a' into HEAD 2024-07-04 13:54:17 -07:00
update.c rcu-tasks: Make Tasks RCU wait idly for grace-period delays 2024-04-09 15:11:49 +02:00