linux/kernel
Thomas Gleixner aa3b66f401 tick/sched: Make jiffies update quick check more robust
The quick check in tick_do_update_jiffies64() whether jiffies need to be
updated is not really correct under all circumstances and on all
architectures, especially not on 32bit systems.

The quick check does:

    if (now < READ_ONCE(tick_next_period))
    	return;

and the counterpart in the update is:

    WRITE_ONCE(tick_next_period, next_update_time);

This has two problems:

  1) On weakly ordered architectures there is no guarantee that the stores
     before the WRITE_ONCE() are visible which means that other CPUs can
     operate on a stale jiffies value.

  2) On 32bit the store of tick_next_period which is an u64 is split into
     two 32bit stores. If the first 32bit store advances tick_next_period
     far out and the second 32bit store is delayed (virt, NMI ...) then
     jiffies will become stale until the second 32bit store happens.

Address this by seperating the handling for 32bit and 64bit.

On 64bit problem #1 is addressed by replacing READ_ONCE() / WRITE_ONCE()
with smp_load_acquire() / smp_store_release().

On 32bit problem #2 is addressed by protecting the quick check with the
jiffies sequence counter. The load and stores can be plain because the
sequence count mechanics provides the required barriers already.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://lore.kernel.org/r/87czzpc02w.fsf@nanos.tec.linutronix.de
2020-12-11 23:19:10 +01:00
..
bpf Fixes for 5.10-rc1 from the networking tree: 2020-10-23 12:05:49 -07:00
cgroup kernel/: fix repeated words in comments 2020-10-16 11:11:19 -07:00
configs
debug kdb: Fix pager search for multi-line strings 2020-10-01 14:44:08 +01:00
dma dma-mapping: move more functions to dma-map-ops.h 2020-10-20 10:41:07 +02:00
entry arch-cleanup-2020-10-22 2020-10-23 10:06:38 -07:00
events task_work: cleanup notification modes 2020-10-17 15:05:30 -06:00
gcov gcov: add support for GCC 10.1 2020-09-11 09:33:54 -07:00
irq task_work: cleanup notification modes 2020-10-17 15:05:30 -06:00
kcsan kernel/: fix repeated words in comments 2020-10-16 11:11:19 -07:00
livepatch kernel/: fix repeated words in comments 2020-10-16 11:11:19 -07:00
locking Merge tag 'core-rcu-2020-10-12' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2020-10-18 14:34:50 -07:00
power kernel/: fix repeated words in comments 2020-10-16 11:11:19 -07:00
printk Urgent printk fix for 5.10 2020-10-16 12:52:37 -07:00
rcu Merge tag 'core-rcu-2020-10-12' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2020-10-18 14:34:50 -07:00
sched treewide: Convert macro and uses of __section(foo) to __section("foo") 2020-10-25 14:51:49 -07:00
time tick/sched: Make jiffies update quick check more robust 2020-12-11 23:19:10 +01:00
trace treewide: Convert macro and uses of __section(foo) to __section("foo") 2020-10-25 14:51:49 -07:00
.gitignore
acct.c kernel: acct.c: fix some kernel-doc nits 2020-10-16 11:11:19 -07:00
async.c treewide: Remove uninitialized_var() usage 2020-07-16 12:35:15 -07:00
audit_fsnotify.c fsnotify: create method handle_inode_event() in fsnotify_operations 2020-07-27 23:25:50 +02:00
audit_tree.c \n 2020-08-06 19:29:51 -07:00
audit_watch.c fsnotify: create method handle_inode_event() in fsnotify_operations 2020-07-27 23:25:50 +02:00
audit.c audit: Remove redundant null check 2020-08-26 09:10:39 -04:00
audit.h audit: change unnecessary globals into statics 2020-08-17 20:26:58 -04:00
auditfilter.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
auditsc.c audit/stable-5.9 PR 20200803 2020-08-04 14:20:26 -07:00
backtracetest.c treewide: Replace DECLARE_TASKLET() with DECLARE_TASKLET_OLD() 2020-07-30 11:15:58 -07:00
bounds.c
capability.c LSM: Signal to SafeSetID when setting group IDs 2020-10-13 09:17:34 -07:00
compat.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
configs.c
context_tracking.c context_tracking: Ensure that the critical path cannot be instrumented 2020-06-11 15:14:36 +02:00
cpu_pm.c notifier: Fix broken error handling pattern 2020-09-01 09:58:03 +02:00
cpu.c
crash_core.c kdump: append kernel build-id string to VMCOREINFO 2020-08-12 10:58:01 -07:00
crash_dump.c
cred.c
delayacct.c
dma.c
elfcore.c
exec_domain.c
exit.c pid: move pidfd_get_pid() to pid.c 2020-10-18 09:27:10 -07:00
extable.c
fail_function.c
fork.c kernel/: fix repeated words in comments 2020-10-16 11:11:19 -07:00
freezer.c
futex.c futex: Adjust absolute futex timeouts with per time namespace offset 2020-10-20 17:02:57 +02:00
gen_kheaders.sh
groups.c LSM: Signal to SafeSetID when setting group IDs 2020-10-13 09:17:34 -07:00
hung_task.c
iomem.c
irq_work.c
jump_label.c kernel/: fix repeated words in comments 2020-10-16 11:11:19 -07:00
kallsyms.c treewide: Convert macro and uses of __section(foo) to __section("foo") 2020-10-25 14:51:49 -07:00
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kcov.c kcov: make some symbols static 2020-08-12 10:58:02 -07:00
kexec_core.c kernel/: fix repeated words in comments 2020-10-16 11:11:19 -07:00
kexec_elf.c
kexec_file.c kernel/resource: move and rename IORESOURCE_MEM_DRIVER_MANAGED 2020-10-16 11:11:18 -07:00
kexec_internal.h
kexec.c LSM: Introduce kernel_post_load_data() hook 2020-10-05 13:37:03 +02:00
kheaders.c
kmod.c kmod: remove redundant "be an" in the comment 2020-08-12 10:58:01 -07:00
kprobes.c Updates for tracing and bootconfig: 2020-10-15 15:51:28 -07:00
ksysfs.c
kthread.c kernel/: fix repeated words in comments 2020-10-16 11:11:19 -07:00
latencytop.c
Makefile Kbuild updates for v5.10 2020-10-22 13:13:57 -07:00
module_signature.c
module_signing.c
module-internal.h
module.c Modules updates for v5.10 2020-10-22 13:08:57 -07:00
notifier.c notifier: Fix broken error handling pattern 2020-09-01 09:58:03 +02:00
nsproxy.c nsproxy: support CLONE_NEWTIME with setns() 2020-07-08 11:14:22 +02:00
padata.c padata: fix possible padata_works_lock deadlock 2020-09-04 17:51:55 +10:00
panic.c panic: dump registers on panic_on_warn 2020-10-16 11:11:22 -07:00
params.c moduleparams: Add hexint type parameter 2020-07-28 13:44:53 +02:00
pid_namespace.c kernel/: fix repeated words in comments 2020-10-16 11:11:19 -07:00
pid.c pid: move pidfd_get_pid() to pid.c 2020-10-18 09:27:10 -07:00
profile.c
ptrace.c
range.c kernel.h: split out min()/max() et al. helpers 2020-10-16 11:11:19 -07:00
reboot.c arch: remove unicore32 port 2020-07-01 12:09:13 +03:00
regset.c regset: kill ->get() 2020-07-27 14:31:12 -04:00
relay.c kernel/relay.c: drop unneeded initialization 2020-10-16 11:11:22 -07:00
resource.c kernel/resource: make iomem_resource implicit in release_mem_region_adjustable() 2020-10-16 11:11:18 -07:00
rseq.c
scftorture.c scftorture: Add cond_resched() to test loop 2020-08-24 18:38:38 -07:00
scs.c mm: memcontrol: account kernel stack per node 2020-08-07 11:33:25 -07:00
seccomp.c seccomp: Make duplicate listener detection non-racy 2020-10-08 13:17:47 -07:00
signal.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
smp.c Merge tag 'core-rcu-2020-10-12' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2020-10-18 14:34:50 -07:00
smpboot.c
smpboot.h
softirq.c softirq: Add debug check to __raise_softirq_irqoff() 2020-09-16 15:18:56 +02:00
stackleak.c stackleak: let stack_erasing_sysctl take a kernel pointer buffer 2020-09-19 13:13:39 -07:00
stacktrace.c stacktrace: Remove reliable argument from arch_stack_walk() callback 2020-09-18 14:24:16 +01:00
static_call.c static_call: Fix return type of static_call_init 2020-10-02 21:18:25 +02:00
stop_machine.c
sys_ni.c mm/madvise: introduce process_madvise() syscall: an external memory hinting API 2020-10-18 09:27:10 -07:00
sys.c kernel/sys.c: fix prototype of prctl_get_tid_address() 2020-10-25 11:44:16 -07:00
sysctl-test.c
sysctl.c mm: allow a controlled amount of unfairness in the page lock 2020-09-17 10:26:41 -07:00
task_work.c task_work: cleanup notification modes 2020-10-17 15:05:30 -06:00
taskstats.c taskstats: move specifying netlink policy back to ops 2020-10-02 19:11:12 -07:00
test_kprobes.c
torture.c torture: Dump ftrace at shutdown only if requested 2020-06-29 12:01:45 -07:00
tracepoint.c tracepoint: Fix out of sync data passing by static caller 2020-10-02 21:18:25 +02:00
tsacct.c
ucount.c
uid16.c
uid16.h
umh.c usermodehelper: reset umask to default before executing user process 2020-10-06 10:31:52 -07:00
up.c
user_namespace.c kernel/: fix repeated words in comments 2020-10-16 11:11:19 -07:00
user-return-notifier.c
user.c
usermode_driver.c umd: Stop using split_argv 2020-07-07 11:58:59 -05:00
utsname_sysctl.c
utsname.c
watch_queue.c watch_queue: Limit the number of watches a user can hold 2020-08-17 09:39:18 -07:00
watchdog_hld.c
watchdog.c
workqueue_internal.h
workqueue.c workqueue: fix a kernel-doc warning 2020-10-16 07:28:20 +02:00