Paul E. McKenney b8c17e6664 rcu: Maintain special bits at bottom of ->dynticks counter ... Currently, IPIs are used to force other CPUs to invalidate their TLBs in response to a kernel virtual-memory mapping change. This works, but degrades both battery lifetime (for idle CPUs) and real-time response (for nohz_full CPUs), and in addition results in unnecessary IPIs due to the fact that CPUs executing in usermode are unaffected by stale kernel mappings. It would be better to cause a CPU executing in usermode to wait until it is entering kernel mode to do the flush, first to avoid interrupting usemode tasks and second to handle multiple flush requests with a single flush in the case of a long-running user task. This commit therefore reserves a bit at the bottom of the ->dynticks counter, which is checked upon exit from extended quiescent states. If it is set, it is cleared and then a new rcu_eqs_special_exit() macro is invoked, which, if not supplied, is an empty single-pass do-while loop. If this bottom bit is set on -entry- to an extended quiescent state, then a WARN_ON_ONCE() triggers. This bottom bit may be set using a new rcu_eqs_special_set() function, which returns true if the bit was set, or false if the CPU turned out to not be in an extended quiescent state. Please note that this function refuses to set the bit for a non-nohz_full CPU when that CPU is executing in usermode because usermode execution is tracked by RCU as a dyntick-idle extended quiescent state only for nohz_full CPUs. Reported-by: Andy Lutomirski <luto@amacapital.net> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Reviewed-by: Josh Triplett <josh@joshtriplett.org>		2017-04-18 11:19:22 -07:00
..
bpf	Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net	2017-03-04 17:31:39 -08:00
cgroup	scripts/spelling.txt: add "disble(d)" pattern and fix typo instances	2017-03-09 17:01:09 -08:00
configs	config: android-base: enable hardened usercopy and kernel ASLR	2017-02-27 18:43:46 -08:00
debug	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>	2017-03-02 08:42:34 +01:00
events	scripts/spelling.txt: add "disble(d)" pattern and fix typo instances	2017-03-09 17:01:09 -08:00
gcov
irq	sched/headers: Prepare to move the get_task_struct()/put_task_struct() and related APIs from <linux/sched.h> to <linux/sched/task.h>	2017-03-02 08:42:40 +01:00
livepatch	livepatch/module: make TAINT_LIVEPATCH module-specific	2016-08-26 14:42:08 +02:00
locking	Merge branch 'locking-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip	2017-03-07 14:33:11 -08:00
power	Merge branch 'WIP.sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip	2017-03-03 10:16:38 -08:00
printk	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task_stack.h>	2017-03-02 08:42:36 +01:00
rcu	rcu: Maintain special bits at bottom of ->dynticks counter	2017-04-18 11:19:22 -07:00
sched	Power management updates for v4.11-rc2	2017-03-09 16:30:37 -08:00
time	Merge branch 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip	2017-03-07 14:45:22 -08:00
trace	scripts/spelling.txt: add "overide" pattern and fix typo instances	2017-03-09 17:01:09 -08:00
.gitignore
acct.c	sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>	2017-03-02 08:42:39 +01:00
async.c
audit_fsnotify.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs	2016-12-17 18:44:00 -08:00
audit_tree.c	Merge branch 'stable-4.10' of git://git.infradead.org/users/pcmoore/audit	2017-01-05 23:06:06 -08:00
audit_watch.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs	2016-12-17 18:44:00 -08:00
audit.c	Merge branch 'stable-4.11' of git://git.infradead.org/users/pcmoore/audit	2017-02-21 13:25:50 -08:00
audit.h	Merge branch 'stable-4.11' of git://git.infradead.org/users/pcmoore/audit	2017-02-21 13:25:50 -08:00
auditfilter.c	audit: add support for session ID user filter	2016-11-29 15:10:12 -05:00
auditsc.c	Merge branch 'stable-4.11' of git://git.infradead.org/users/pcmoore/audit	2017-02-21 13:25:50 -08:00
backtracetest.c
bounds.c
capability.c	capability: export has_capability	2017-01-12 07:01:56 -07:00
compat.c	Replace <asm/uaccess.h> with <linux/uaccess.h> globally	2016-12-24 11:46:01 -08:00
configs.c	Replace <asm/uaccess.h> with <linux/uaccess.h> globally	2016-12-24 11:46:01 -08:00
context_tracking.c
cpu_pm.c
cpu.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>	2017-03-02 08:42:35 +01:00
crash_dump.c
cred.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/coredump.h>	2017-03-02 08:42:28 +01:00
delayacct.c	sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>	2017-03-02 08:42:39 +01:00
dma.c
elfcore.c
exec_domain.c
exit.c	userfaultfd: non-cooperative: rollback userfaultfd_exit	2017-03-09 17:01:09 -08:00
extable.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next	2017-02-22 10:15:09 -08:00
fork.c	sched/headers, RCU: Move rcu_copy_process() from <linux/sched/task.h> to kernel/fork.c	2017-03-03 01:43:46 +01:00
freezer.c	freezer, oom: check TIF_MEMDIE on the correct task	2016-07-28 16:07:41 -07:00
futex_compat.c	Replace <asm/uaccess.h> with <linux/uaccess.h> globally	2016-12-24 11:46:01 -08:00
futex.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/mm.h>	2017-03-02 08:42:28 +01:00
groups.c	Replace <asm/uaccess.h> with <linux/uaccess.h> globally	2016-12-24 11:46:01 -08:00
hung_task.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>	2017-03-02 08:42:34 +01:00
irq_work.c
jump_label.c	This release has no new tracing features, just clean ups, minor fixes	2017-02-27 13:26:17 -08:00
kallsyms.c	bpf: make jited programs visible in traces	2017-02-17 13:40:05 -05:00
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks	locking/mutex: Allow MUTEX_SPIN_ON_OWNER when DEBUG_MUTEXES	2016-10-25 11:31:51 +02:00
Kconfig.preempt
kcov.c	kcov: make kcov work properly with KASLR enabled	2016-12-20 09:48:47 -08:00
kexec_core.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/pmladek/printk	2017-02-22 17:33:34 -08:00
kexec_file.c	kexec, x86/purgatory: Unbreak it and clean it up	2017-03-10 20:55:09 +01:00
kexec_internal.h	kexec, x86/purgatory: Unbreak it and clean it up	2017-03-10 20:55:09 +01:00
kexec.c	kexec: allow architectures to override boot mapping	2016-08-02 19:35:27 -04:00
kmod.c	sched/headers, vfs/execve: Prepare to move the do_execve*() prototypes from <linux/sched.h> to <linux/binfmts.h>	2017-03-02 08:42:39 +01:00
kprobes.c	powerpc updates for 4.11 part 1.	2017-02-22 10:30:38 -08:00
ksysfs.c	kernel/ksysfs.c: add __ro_after_init to bin_attribute structure	2017-02-24 17:46:56 -08:00
kthread.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>	2017-03-02 08:42:35 +01:00
latencytop.c	sched/headers: Prepare to move sched_info_on() and force_schedstat_enabled() from <linux/sched.h> to <linux/sched/stat.h>	2017-03-02 08:42:39 +01:00
Makefile	cgroup: move cgroup files under kernel/cgroup/	2016-12-27 14:49:05 -05:00
membarrier.c	Fix: Disable sys_membarrier when nohz_full is enabled	2017-01-23 11:32:16 -08:00
memremap.c	mm, devm_memremap_pages: hold device_hotplug lock over mem_hotplug_{begin, done}	2017-02-24 17:46:53 -08:00
module_signing.c
module-internal.h
module.c	Modules updates for v4.11	2017-02-22 17:08:33 -08:00
notifier.c	kernel/notifier.c: simplify expression	2017-02-24 17:46:56 -08:00
nsproxy.c
padata.c	padata: Remove unused but set variables	2016-10-25 11:08:10 +08:00
panic.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>	2017-03-02 08:42:34 +01:00
params.c
pid_namespace.c	sched/headers: Prepare for the reduction of <linux/sched.h>'s signal API dependency	2017-03-02 08:42:37 +01:00
pid.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>	2017-03-02 08:42:35 +01:00
profile.c	sched/headers: Prepare to move sched_info_on() and force_schedstat_enabled() from <linux/sched.h> to <linux/sched/stat.h>	2017-03-02 08:42:39 +01:00
ptrace.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>	2017-03-02 08:42:35 +01:00
range.c
reboot.c
relay.c	lib/vsprintf.c: remove %Z support	2017-02-27 18:43:47 -08:00
resource.c
seccomp.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task_stack.h>	2017-03-02 08:42:36 +01:00
signal.c	sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>	2017-03-02 08:42:39 +01:00
smp.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/idle.h>	2017-03-02 08:42:26 +01:00
smpboot.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>	2017-03-02 08:42:35 +01:00
smpboot.h
softirq.c	softirq: Display IRQ_POLL for irq-poll statistics	2016-10-21 15:45:47 -06:00
stacktrace.c	stacktrace, lockdep: Fix address, newline ugliness	2017-02-08 08:21:31 +01:00
stop_machine.c	locking/core, stop_machine: Yield the CPU during stop machine()	2016-11-16 10:15:09 +01:00
sys_ni.c	move aio compat to fs/aio.c	2016-12-22 22:58:37 -05:00
sys.c	sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>	2017-03-02 08:42:39 +01:00
sysctl_binary.c	sysctl: add KERN_CONT to deprecated_sysctl_warning()	2016-12-14 16:04:07 -08:00
sysctl.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/coredump.h>	2017-03-02 08:42:28 +01:00
task_work.c	task_work: use READ_ONCE/lockless_dereference, avoid pi_lock if !task_works	2016-08-02 19:35:02 -04:00
taskstats.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net	2016-11-15 10:54:36 -05:00
test_kprobes.c
torture.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/clock.h>	2017-03-02 08:42:27 +01:00
tracepoint.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h>	2017-03-02 08:42:35 +01:00
tsacct.c	sched/headers: Prepare to move cputime functionality from <linux/sched.h> into <linux/sched/cputime.h>	2017-03-02 08:42:39 +01:00
ucount.c	ucount: Remove the atomicity from ucount->count	2017-03-06 15:26:37 -06:00
uid16.c	sched/headers: Prepare to remove <linux/cred.h> inclusion from <linux/sched.h>	2017-03-02 08:42:31 +01:00
up.c	smp: Add function to execute a function synchronously on a CPU	2016-09-05 13:52:39 +02:00
user_namespace.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/signal.h>	2017-03-02 08:42:29 +01:00
user-return-notifier.c
user.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/user.h>	2017-03-02 08:42:29 +01:00
utsname_sysctl.c	sched/headers: Remove <linux/rwsem.h> from <linux/sched.h>	2017-03-03 01:45:36 +01:00
utsname.c	sched/headers: Prepare to move the task_lock()/unlock() APIs to <linux/sched/task.h>	2017-03-02 08:42:38 +01:00
watchdog_hld.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>	2017-03-02 08:42:34 +01:00
watchdog.c	sched/headers: Prepare for new header dependencies before moving code to <linux/sched/debug.h>	2017-03-02 08:42:34 +01:00
workqueue_internal.h
workqueue.c	time: Remove CONFIG_TIMER_STATS	2017-02-10 11:15:08 +01:00