leandrof/linux
1
0
Fork 0
forked from Minki/linux
Code Issues Pull Requests Packages Projects Releases Wiki Activity

rcu: Allow RCU grace-period cleanup to be preempted

Browse Source 
RCU grace-period cleanup is currently carried out with interrupts
disabled, which can result in excessive latency spikes on large systems
(many hundreds or thousands of CPUs).  This patch therefore makes the
RCU grace-period cleanup be preemptible, including voluntary preemption
points, which should eliminate those latency spikes.  Similar spikes from
forcing of quiescent states will be dealt with similarly by later patches.

Updated to replace uses of spin_lock_irqsave() with spin_lock_irq(), as
suggested by Peter Zijlstra.

Reported-by: Mike Galbraith <mgalbraith@suse.de>
Reported-by: Dimitri Sivanich <sivanich@sgi.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit is contained in:
Paul E. McKenney 2012-06-21 08:19:05 -07:00
parent cabc49c1ff
commit c856bafae7
1 changed files with 5 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
kernel/rcutree.c
View File

@ -1151,7 +1151,7 @@ static int __noreturn rcu_gp_kthread(void *arg)
* completed.
*/
if (*rdp->nxttail[RCU_WAIT_TAIL] == NULL) {
raw_spin_unlock(&rnp->lock); /* irqs remain disabled. */
raw_spin_unlock_irq(&rnp->lock);
/*
* Propagate new ->completed value to rcu_node
@ -1160,14 +1160,13 @@ static int __noreturn rcu_gp_kthread(void *arg)
* to process their callbacks.
*/
rcu_for_each_node_breadth_first(rsp, rnp) {
/* irqs already disabled. */
raw_spin_lock(&rnp->lock);
raw_spin_lock_irq(&rnp->lock);
rnp->completed = rsp->gpnum;
/* irqs remain disabled. */
raw_spin_unlock(&rnp->lock);
raw_spin_unlock_irq(&rnp->lock);
cond_resched();
}
rnp = rcu_get_root(rsp);
raw_spin_lock(&rnp->lock); /* irqs already disabled. */
raw_spin_lock_irq(&rnp->lock);
}
rsp->completed = rsp->gpnum; /* Declare grace period done. */