Wangyang Guo 85f0d8e39a workqueue: Reduce expensive locks for unbound workqueue ... For unbound workqueue, pwqs usually map to just a few pools. Most of the time, pwqs will be linked sequentially to wq->pwqs list by cpu index. Usually, consecutive CPUs have the same workqueue attribute (e.g. belong to the same NUMA node). This makes pwqs with the same pool cluster together in the pwq list. Only do lock/unlock if the pool has changed in flush_workqueue_prep_pwqs(). This reduces the number of expensive lock operations. The performance data shows this change boosts FIO by 65x in some cases when multiple concurrent threads write to xfs mount points with fsync. FIO Benchmark Details - FIO version: v3.35 - FIO Options: ioengine=libaio,iodepth=64,norandommap=1,rw=write, size=128M,bs=4k,fsync=1 - FIO Job Configs: 64 jobs in total writing to 4 mount points (ramdisks formatted as xfs file system). - Kernel Codebase: v6.12-rc5 - Test Platform: Xeon 8380 (2 sockets) Reviewed-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Wangyang Guo <wangyang.guo@intel.com> Reviewed-by: Lai Jiangshan <jiangshanlai@gmail.com> Signed-off-by: Tejun Heo <tj@kernel.org>		2024-11-15 06:43:39 -10:00
..
bpf	[tree-wide] finally take no_llseek out	2024-09-27 08:18:43 -07:00
cgroup	struct fd layout change (and conversion to accessor helpers)	2024-09-23 09:35:36 -07:00
configs	tinyconfig: remove unnecessary 'is not set' for choice blocks	2024-09-01 20:34:38 +09:00
debug	kdb: Get rid of redundant kdb_curr_task()	2024-06-21 15:49:29 +01:00
dma	dma-mapping: report unlimited DMA addressing in IOMMU DMA path	2024-09-23 08:38:56 +02:00
entry	treewide: context_tracking: Rename CONTEXT_* into CT_STATE_*	2024-07-29 07:33:10 +05:30
events	[tree-wide] finally take no_llseek out	2024-09-27 08:18:43 -07:00
futex	fault-inject: improve build for CONFIG_FAULT_INJECTION=n	2024-09-01 20:43:33 -07:00
gcov	gcov: add support for GCC 14	2024-06-15 10:43:06 -07:00
irq	pci-v6.12-changes	2024-09-23 12:47:06 -07:00
kcsan	kcsan: Use min() to fix Coccinelle warning	2024-08-01 16:40:44 -07:00
livepatch	livepatch: Replace snprintf() with sysfs_emit()	2024-07-02 16:56:18 +02:00
locking	Locking changes for v6.12:	2024-09-29 08:51:30 -07:00
module	Modules changes for v6.12-rc1	2024-09-28 09:06:15 -07:00
power	[tree-wide] finally take no_llseek out	2024-09-27 08:18:43 -07:00
printk	drm next for 6.12-rc1	2024-09-19 10:18:15 +02:00
rcu	slab updates for 6.12	2024-09-18 08:53:53 +02:00
sched	sched_ext: Provide a sysfs enable_seq counter	2024-09-23 06:53:02 -10:00
time	[tree-wide] finally take no_llseek out	2024-09-27 08:18:43 -07:00
trace	[tree-wide] finally take no_llseek out	2024-09-27 08:18:43 -07:00
.gitignore
acct.c	kernel misc: Remove the now superfluous sentinel elements from ctl_table array	2024-04-24 09:43:53 +02:00
async.c	async: Use a dedicated unbound workqueue with raised min_active	2024-02-09 11:13:59 -10:00
audit_fsnotify.c
audit_tree.c	fsnotify: create a wrapper fsnotify_find_inode_mark()	2024-04-04 16:24:16 +02:00
audit_watch.c	fsnotify: create a wrapper fsnotify_find_inode_mark()	2024-04-04 16:24:16 +02:00
audit.c	audit: Make use of str_enabled_disabled() helper	2024-09-03 16:35:16 -04:00
audit.h
auditfilter.c	audit: use task_tgid_nr() instead of task_pid_nr()	2024-08-28 16:48:28 -04:00
auditsc.c	audit: use task_tgid_nr() instead of task_pid_nr()	2024-08-28 16:48:28 -04:00
backtracetest.c	backtracetest: add MODULE_DESCRIPTION()	2024-06-24 22:24:55 -07:00
bounds.c	bounds: Use the right number of bits for power-of-two CONFIG_NR_CPUS	2024-04-29 08:29:29 -07:00
capability.c
cfi.c
compat.c
configs.c
context_tracking.c	context_tracking, rcu: Rename rcu_dyntick trace event into rcu_watching	2024-08-15 21:30:43 +05:30
cpu_pm.c
cpu.c	Updates for timers and timekeeping:	2024-09-17 07:25:37 +02:00
crash_core.c	Document/kexec: generalize crash hotplug description	2024-09-01 20:43:37 -07:00
crash_reserve.c	crash: fix crash memory reserve exceed system memory bug	2024-09-01 20:43:30 -07:00
cred.c	cred: Use KMEM_CACHE() instead of kmem_cache_create()	2024-02-23 17:33:31 -05:00
delayacct.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
dma.c
elfcorehdr.c	crash: remove dependency of FA_DUMP on CRASH_DUMP	2024-02-23 17:48:22 -08:00
exec_domain.c
exit.c	ALong with the usual shower of singleton patches, notable patch series in	2024-09-21 07:29:05 -07:00
exit.h
extable.c
fail_function.c
fork.c	memblock: updates for 6.12-rc1	2024-09-25 11:35:19 -07:00
freezer.c	sched,freezer: Mark TASK_FROZEN special	2024-08-17 11:06:44 +02:00
gen_kheaders.sh	kheaders: use command -v to test for existence of cpio	2024-05-30 01:13:20 +09:00
groups.c
hung_task.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
iomem.c
irq_work.c
jump_label.c	jump_label: Fix static_key_slow_dec() yet again	2024-09-10 11:57:27 +02:00
kallsyms_internal.h	kallsyms: get rid of code for absolute kallsyms	2024-07-20 16:33:21 +09:00
kallsyms_selftest.c	kallsyms: Match symbols exactly with CONFIG_LTO_CLANG	2024-08-15 09:33:35 -07:00
kallsyms_selftest.h
kallsyms.c	kallsyms: Match symbols exactly with CONFIG_LTO_CLANG	2024-08-15 09:33:35 -07:00
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.kexec	crash: clean up kdump related config items	2024-02-23 17:48:22 -08:00
Kconfig.locks
Kconfig.preempt	sched_ext: Build fix on !CONFIG_STACKTRACE[_SUPPORT]	2024-08-01 07:08:01 -10:00
kcov.c	Updates for KCOV instrumentation on x86:	2024-09-17 12:40:34 +02:00
kexec_core.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
kexec_elf.c
kexec_file.c	kexec_file: fix elfcorehdr digest exclusion when CONFIG_CRASH_HOTPLUG=y	2024-09-01 17:59:01 -07:00
kexec_internal.h	kexec: use atomic_try_cmpxchg_acquire() in kexec_trylock()	2024-09-01 20:43:23 -07:00
kexec.c	crash: add a new kexec flag for hotplug support	2024-04-23 14:59:01 +10:00
kheaders.c
kprobes.c	kprobes: Fix to check symbol prefixes correctly	2024-08-05 14:04:03 +09:00
ksyms_common.c
ksysfs.c	profiling: remove prof_cpu_mask	2024-07-29 10:45:54 -07:00
kthread.c	kthread: Fix task state in kthread worker if being frozen	2024-09-10 09:51:14 +02:00
latencytop.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
Makefile	mm: move kernel/numa.c to mm/	2024-09-03 21:15:26 -07:00
module_signature.c
notifier.c
nsproxy.c	introduce fd_file(), convert all accessors to it.	2024-08-12 22:00:43 -04:00
padata.c	This update includes the following changes:	2024-09-16 06:28:28 +02:00
panic.c	drm next for 6.12-rc1	2024-09-19 10:18:15 +02:00
params.c	params: Fix multi-line comment style	2023-12-01 09:51:44 -08:00
pid_namespace.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
pid_sysctl.h	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
pid.c	introduce fd_file(), convert all accessors to it.	2024-08-12 22:00:43 -04:00
profile.c	profiling: remove profile=sleep support	2024-08-04 13:36:28 -07:00
ptrace.c	ptrace_attach: shift send(SIGSTOP) into ptrace_set_stopped()	2024-02-22 15:38:52 -08:00
range.c
reboot.c	kernel misc: Remove the now superfluous sentinel elements from ctl_table array	2024-04-24 09:43:53 +02:00
regset.c	regset: use kvzalloc() for regset_get_alloc()	2024-04-25 21:07:03 -07:00
relay.c	[tree-wide] finally take no_llseek out	2024-09-27 08:18:43 -07:00
resource_kunit.c	resource, kunit: add test case for region_intersects()	2024-09-17 01:07:00 -07:00
resource.c	ALong with the usual shower of singleton patches, notable patch series in	2024-09-21 07:29:05 -07:00
rseq.c
scftorture.c	scftorture: Make torture_type static	2024-05-30 15:31:51 -07:00
scs.c
seccomp.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
signal.c	Revert "binfmt_elf, coredump: Log the reason of the failed core dumps"	2024-09-26 11:39:02 -07:00
smp.c	smp: print only local CPU info when sched_clock goes backward	2024-08-15 00:06:48 +05:30
smpboot.c
smpboot.h
softirq.c	softirq: Remove unused 'action' parameter from action callback	2024-08-20 17:13:40 +02:00
stackleak.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
stacktrace.c	stacktrace: fix kernel-doc typo	2023-12-29 12:22:29 -08:00
static_call_inline.c	static_call: Replace pointless WARN_ON() in static_call_module_notify()	2024-09-06 16:29:22 +02:00
static_call.c
stop_machine.c	rcu: Rename rcu_momentary_dyntick_idle() into rcu_momentary_eqs()	2024-08-15 21:30:42 +05:30
sys_ni.c	Probes updates for v6.11:	2024-07-18 12:19:20 -07:00
sys.c	struct fd layout change (and conversion to accessor helpers)	2024-09-23 09:35:36 -07:00
sysctl-test.c	sysctl: Add module description to sysctl-testing	2024-06-03 15:20:37 +02:00
sysctl.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
task_work.c	task_work: make TWA_NMI_CURRENT handling conditional on IRQ_WORK	2024-07-29 12:05:06 -07:00
taskstats.c	introduce fd_file(), convert all accessors to it.	2024-08-12 22:00:43 -04:00
torture.c	torture: Add MODULE_DESCRIPTION()	2024-05-30 15:31:38 -07:00
tracepoint.c	tracepoint: Support iterating tracepoints in a loading module	2024-09-25 23:23:44 +09:00
tsacct.c	tsacct: replace strncpy() with strscpy()	2024-07-12 16:39:53 -07:00
ucount.c	sysctl changes for v6.10-rc1	2024-05-17 17:31:24 -07:00
uid16.c
uid16.h
umh.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
up.c
user_namespace.c	user_namespace: use kmemdup_array() instead of kmemdup() for multiple allocation	2024-09-09 16:47:42 -07:00
user-return-notifier.c
user.c	uidgid: make sure we fit into one cacheline	2024-09-12 12:16:09 +02:00
usermode_driver.c
utsname_sysctl.c	sysctl: treewide: constify the ctl_table argument of proc_handlers	2024-07-24 20:59:29 +02:00
utsname.c
vhost_task.c	vhost_task: Handle SIGKILL by flushing work and exiting	2024-05-22 08:31:15 -04:00
vmcore_info.c	mm: support only one page_type per page	2024-09-03 21:15:43 -07:00
watch_queue.c	introduce fd_file(), convert all accessors to it.	2024-08-12 22:00:43 -04:00
watchdog_buddy.c
watchdog_perf.c	watchdog/perf: properly initialize the turbo mode timestamp and rearm counter	2024-07-17 21:11:34 -07:00
watchdog.c	watchdog: handle the ENODEV failure case of lockup_detector_delay_init() separately	2024-09-01 20:43:32 -07:00
workqueue_internal.h
workqueue.c	workqueue: Reduce expensive locks for unbound workqueue	2024-11-15 06:43:39 -10:00