linux/kernel
Dietmar Eggemann de40f33e78 sched/deadline: Reduce rq lock contention in dl_add_task_root_domain()
dl_add_task_root_domain() is called during sched domain rebuild:

  rebuild_sched_domains_locked()
    partition_and_rebuild_sched_domains()
      rebuild_root_domains()
         for all top_cpuset descendants:
           update_tasks_root_domain()
             for all tasks of cpuset:
               dl_add_task_root_domain()

Change it so that only the task pi lock is taken to check if the task
has a SCHED_DEADLINE (DL) policy. In case that p is a DL task take the
rq lock as well to be able to safely de-reference root domain's DL
bandwidth structure.

Most of the tasks will have another policy (namely SCHED_NORMAL) and
can now bail without taking the rq lock.

One thing to note here: Even in case that there aren't any DL user
tasks, a slow frequency switching system with cpufreq gov schedutil has
a DL task (sugov) per frequency domain running which participates in DL
bandwidth management.

Signed-off-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Quentin Perret <qperret@google.com>
Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
Reviewed-by: Daniel Bristot de Oliveira <bristot@redhat.com>
Acked-by: Juri Lelli <juri.lelli@redhat.com>
Link: https://lkml.kernel.org/r/20210119083542.19856-1-dietmar.eggemann@arm.com
2021-02-17 14:12:42 +01:00
..
bpf bpf: Fix 32 bit src register truncation on div/mod 2021-02-10 01:32:40 +01:00
cgroup cgroup: fix psi monitor for root cgroup 2021-01-19 11:37:05 -05:00
configs staging: ION: remove some references to CONFIG_ION 2021-01-06 17:39:38 +01:00
debug smp: Cleanup smp_call_function*() 2020-11-24 16:47:49 +01:00
dma dma-mapping: benchmark: use u8 for reserved field in uAPI structure 2021-02-05 12:48:46 +01:00
entry preempt/dynamic: Provide irqentry_exit_cond_resched() static call 2021-02-17 14:12:42 +01:00
events uprobes: (Re)add missing get_uprobe() in __find_uprobe() 2021-02-17 14:12:42 +01:00
gcov init/gcov: allow CONFIG_CONSTRUCTORS on UML to fix module gcov 2021-02-05 11:03:47 -08:00
irq genirq/msi: Activate Multi-MSI early when MSI_FLAG_ACTIVATE_EARLY is set 2021-01-30 01:22:31 +01:00
kcsan kcsan: Fix encoding masks and regain address bit 2020-11-06 17:19:26 -08:00
livepatch livepatch: Use the default ftrace_ops instead of REGS when ARGS is available 2020-11-13 12:15:28 -05:00
locking rbtree, rtmutex: Use rb_add_cached() 2021-02-17 14:07:57 +01:00
power PM: hibernate: flush swap writer after marking 2021-01-25 18:52:30 +01:00
printk Merge branch 'printk-rework' into for-linus 2021-01-25 14:29:35 +01:00
rcu Merge branch 'rcu/urgent' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu 2021-01-04 10:55:19 -08:00
sched sched/deadline: Reduce rq lock contention in dl_add_task_root_domain() 2021-02-17 14:12:42 +01:00
time ntp: Use freezable workqueue for RTC synchronization 2021-02-05 18:03:13 +01:00
trace tracing: Fix buffer overflow in trace event filter 2021-02-12 11:16:17 -08:00
.gitignore
acct.c kernel/acct.c: use #elif instead of #end and #elif 2020-12-15 22:46:15 -08:00
async.c treewide: Remove uninitialized_var() usage 2020-07-16 12:35:15 -07:00
audit_fsnotify.c fsnotify: generalize handle_inode_event() 2020-12-03 14:58:35 +01:00
audit_tree.c fsnotify: generalize handle_inode_event() 2020-12-03 14:58:35 +01:00
audit_watch.c fsnotify: generalize handle_inode_event() 2020-12-03 14:58:35 +01:00
audit.c audit: replace atomic_add_return() 2020-12-02 22:52:16 -05:00
audit.h audit: change unnecessary globals into statics 2020-08-17 20:26:58 -04:00
auditfilter.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
auditsc.c audit/stable-5.11 PR 20201214 2020-12-16 10:54:03 -08:00
backtracetest.c treewide: Replace DECLARE_TASKLET() with DECLARE_TASKLET_OLD() 2020-07-30 11:15:58 -07:00
bounds.c
capability.c LSM: Signal to SafeSetID when setting group IDs 2020-10-13 09:17:34 -07:00
compat.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
configs.c
context_tracking.c
cpu_pm.c notifier: Fix broken error handling pattern 2020-09-01 09:58:03 +02:00
cpu.c Scheduler updates: 2020-12-14 18:29:11 -08:00
crash_core.c kdump: append uts_namespace.name offset to VMCOREINFO 2020-12-15 22:46:18 -08:00
crash_dump.c
cred.c
delayacct.c
dma.c
exec_domain.c
exit.c kernel/io_uring: cancel io_uring before task works 2020-12-30 19:36:54 -07:00
extable.c
fail_function.c fault-injection: handle EI_ETYPE_TRUE 2020-12-15 22:46:19 -08:00
fork.c Kernel: fork.c: Fix coding style: Do not use {} around single-line statements 2021-01-11 12:55:01 +01:00
freezer.c
futex.c futex: Handle faults correctly for PI futexes 2021-01-26 15:11:00 +01:00
gen_kheaders.sh
groups.c LSM: Signal to SafeSetID when setting group IDs 2020-10-13 09:17:34 -07:00
hung_task.c kernel/hung_task.c: make type annotations consistent 2020-11-02 12:14:19 -08:00
iomem.c
irq_work.c irq_work: Optimize irq_work_single() 2020-11-24 16:47:49 +01:00
jump_label.c jump_label: Fix usage in module __init 2020-12-18 16:53:12 +01:00
kallsyms.c treewide: Convert macro and uses of __section(foo) to __section("foo") 2020-10-25 14:51:49 -07:00
kcmp.c Merge branch 'exec-update-lock-for-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2020-12-15 19:36:48 -08:00
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt preempt: Introduce CONFIG_PREEMPT_DYNAMIC 2021-02-17 14:12:24 +01:00
kcov.c kernel: make kcov_common_handle consider the current context 2020-11-02 18:00:20 -08:00
kexec_core.c kernel: kexec: remove the lock operation of system_transition_mutex 2021-01-25 18:40:37 +01:00
kexec_elf.c
kexec_file.c crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
kexec_internal.h
kexec.c LSM: Introduce kernel_post_load_data() hook 2020-10-05 13:37:03 +02:00
kheaders.c
kmod.c kmod: remove redundant "be an" in the comment 2020-08-12 10:58:01 -07:00
kprobes.c kretprobe: Avoid re-registration of the same kretprobe earlier 2021-01-29 17:29:16 -05:00
ksysfs.c
kthread.c - Correct the marking of kthreads which are supposed to run on a specific, 2021-01-24 10:09:20 -08:00
latencytop.c
Makefile kcov: don't instrument with UBSAN 2020-12-15 22:46:19 -08:00
module_signature.c
module_signing.c
module-internal.h
module.c Modules updates for v5.11 2020-12-17 13:01:31 -08:00
notifier.c notifier: Fix broken error handling pattern 2020-09-01 09:58:03 +02:00
nsproxy.c fixes-v5.11 2020-12-14 16:40:27 -08:00
padata.c padata: fix possible padata_works_lock deadlock 2020-09-04 17:51:55 +10:00
panic.c panic: don't dump stack twice on warn 2020-11-14 11:26:04 -08:00
params.c Modules updates for v5.11 2020-12-17 13:01:31 -08:00
pid_namespace.c fixes-v5.11 2020-12-14 16:40:27 -08:00
pid.c Merge branch 'exec-update-lock-for-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2020-12-15 19:36:48 -08:00
profile.c
ptrace.c Merge branch 'akpm' (patches from Andrew) 2020-12-15 12:53:37 -08:00
range.c kernel.h: split out min()/max() et al. helpers 2020-10-16 11:11:19 -07:00
reboot.c reboot: hide from sysfs not applicable settings 2020-12-15 22:46:19 -08:00
regset.c regset: kill ->get() 2020-07-27 14:31:12 -04:00
relay.c relay: allow the use of const callback structs 2020-12-15 22:46:18 -08:00
resource_kunit.c resource: provide meaningful MODULE_LICENSE() in test suite 2020-11-25 18:52:35 +01:00
resource.c kernel/resource.c: fix kernel-doc markups 2020-12-15 22:46:18 -08:00
rseq.c
scftorture.c scftorture: Add full-test stutter capability 2020-11-06 17:13:56 -08:00
scs.c scs: switch to vmapped shadow stacks 2020-12-01 10:30:28 +00:00
seccomp.c seccomp updates for v5.11-rc1 2020-12-16 11:30:10 -08:00
signal.c for-linus-2021-01-24 2021-01-24 09:35:28 -08:00
smp.c smp: Process pending softirqs in flush_smp_call_function_from_idle() 2021-02-17 14:12:42 +01:00
smpboot.c kthread: Extract KTHREAD_IS_PER_CPU 2021-01-22 15:09:42 +01:00
smpboot.h
softirq.c Misc fixes/updates: 2020-12-27 09:06:10 -08:00
stackleak.c stackleak: let stack_erasing_sysctl take a kernel pointer buffer 2020-09-19 13:13:39 -07:00
stacktrace.c stacktrace: Remove reliable argument from arch_stack_walk() callback 2020-09-18 14:24:16 +01:00
static_call.c static_call: Allow module use without exposing static_call_key 2021-02-17 14:12:42 +01:00
stop_machine.c Merge branch 'linus' into sched/core, to resolve semantic conflict 2020-11-27 11:10:50 +01:00
sys_ni.c epoll: wire up syscall epoll_pwait2 2020-12-19 11:18:38 -08:00
sys.c kernel: Implement selective syscall userspace redirection 2020-12-02 15:07:56 +01:00
sysctl-test.c
sysctl.c rcu: Panic after fixed number of stalls 2020-11-19 19:37:16 -08:00
task_work.c task_work: remove legacy TWA_SIGNAL path 2020-12-12 09:17:38 -07:00
taskstats.c treewide: rename nla_strlcpy to nla_strscpy. 2020-11-16 08:08:54 -08:00
test_kprobes.c
torture.c rcutorture: Make stutter_wait() caller restore priority 2020-11-06 17:13:54 -08:00
tracepoint.c tracepoints: Migrate to use SYSCALL_WORK flag 2020-11-16 21:53:15 +01:00
tsacct.c
ucount.c
uid16.c
uid16.h
umh.c usermodehelper: reset umask to default before executing user process 2020-10-06 10:31:52 -07:00
up.c
user_namespace.c fixes-v5.11 2020-12-14 16:40:27 -08:00
user-return-notifier.c
user.c user: Use generic ns_common::count 2020-08-19 14:14:12 +02:00
usermode_driver.c umd: Stop using split_argv 2020-07-07 11:58:59 -05:00
utsname_sysctl.c
utsname.c uts: Use generic ns_common::count 2020-08-19 14:13:20 +02:00
watch_queue.c watch_queue: Limit the number of watches a user can hold 2020-08-17 09:39:18 -07:00
watchdog_hld.c
watchdog.c kernel/watchdog: fix watchdog_allowed_mask not used warning 2020-11-14 11:26:03 -08:00
workqueue_internal.h
workqueue.c workqueue: Restrict affinity change to rescuer 2021-01-22 15:09:43 +01:00