Alexei Starovoitov e21aa34178 bpf: Fix fexit trampoline. ... The fexit/fmod_ret programs can be attached to kernel functions that can sleep. The synchronize_rcu_tasks() will not wait for such tasks to complete. In such case the trampoline image will be freed and when the task wakes up the return IP will point to freed memory causing the crash. Solve this by adding percpu_ref_get/put for the duration of trampoline and separate trampoline vs its image life times. The "half page" optimization has to be removed, since first_half->second_half->first_half transition cannot be guaranteed to complete in deterministic time. Every trampoline update becomes a new image. The image with fmod_ret or fexit progs will be freed via percpu_ref_kill and call_rcu_tasks. Together they will wait for the original function and trampoline asm to complete. The trampoline is patched from nop to jmp to skip fexit progs. They are freed independently from the trampoline. The image with fentry progs only will be freed via call_rcu_tasks_trace+call_rcu_tasks which will wait for both sleepable and non-sleepable progs to complete. Fixes: fec56f5890 ("bpf: Introduce BPF trampoline") Reported-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Paul E. McKenney <paulmck@kernel.org> # for RCU Link: https://lore.kernel.org/bpf/20210316210007.38949-1-alexei.starovoitov@gmail.com		2021-03-18 00:22:51 +01:00
..
bpf	bpf: Fix fexit trampoline.	2021-03-18 00:22:51 +01:00
cgroup	idmapped-mounts-v5.12	2021-02-23 13:39:45 -08:00
configs	staging: ION: remove some references to CONFIG_ION	2021-01-06 17:39:38 +01:00
debug	kgdb: fix to kill breakpoints on initmem after boot	2021-02-26 09:41:05 -08:00
dma	Merge branch 'stable/for-linus-5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/konrad/swiotlb	2021-02-26 13:59:32 -08:00
entry	entry: Explicitly flush pending rcuog wakeup before last rescheduling point	2021-02-17 14:12:43 +01:00
events	kernel: delete repeated words in comments	2021-02-26 09:41:03 -08:00
gcov	init/gcov: allow CONFIG_CONSTRUCTORS on UML to fix module gcov	2021-02-05 11:03:47 -08:00
irq	Driver core / debugfs update for 5.12-rc1	2021-02-24 10:13:55 -08:00
kcsan	kcsan: Rewrite kcsan_prandom_u32_max() without prandom_u32_state()	2021-01-04 14:39:07 -08:00
livepatch	kallsyms: refactor {,module_}kallsyms_on_each_symbol	2021-02-08 12:22:08 +01:00
locking	kernel: delete repeated words in comments	2021-02-26 09:41:03 -08:00
power	Merge branches 'powercap' and 'pm-misc'	2021-02-15 18:50:01 +01:00
printk	Merge branch 'printk-rework' into for-linus	2021-02-22 13:43:55 +01:00
rcu	Scheduler updates for v5.12:	2021-02-21 12:35:04 -08:00
sched	kernel: delete repeated words in comments	2021-02-26 09:41:03 -08:00
time	These are the latest RCU updates for v5.12:	2021-02-21 12:04:41 -08:00
trace	ftrace: Fix modify_ftrace_direct.	2021-03-17 00:43:12 +01:00
.gitignore
acct.c	kernel/acct.c: use #elif instead of #end and #elif	2020-12-15 22:46:15 -08:00
async.c	treewide: Remove uninitialized_var() usage	2020-07-16 12:35:15 -07:00
audit_fsnotify.c	audit_alloc_mark(): don't open-code ERR_CAST()	2021-02-23 10:25:27 -05:00
audit_tree.c	fsnotify: generalize handle_inode_event()	2020-12-03 14:58:35 +01:00
audit_watch.c	fsnotify: generalize handle_inode_event()	2020-12-03 14:58:35 +01:00
audit.c	audit: Remove leftover reference to the audit_tasklet	2021-01-15 11:58:10 -05:00
audit.h	audit: change unnecessary globals into statics	2020-08-17 20:26:58 -04:00
auditfilter.c	treewide: Use fallthrough pseudo-keyword	2020-08-23 17:36:59 -05:00
auditsc.c	idmapped-mounts-v5.12	2021-02-23 13:39:45 -08:00
backtracetest.c	treewide: Replace DECLARE_TASKLET() with DECLARE_TASKLET_OLD()	2020-07-30 11:15:58 -07:00
bounds.c
capability.c	capability: handle idmapped mounts	2021-01-24 14:27:16 +01:00
compat.c	treewide: Use fallthrough pseudo-keyword	2020-08-23 17:36:59 -05:00
configs.c
context_tracking.c
cpu_pm.c	notifier: Fix broken error handling pattern	2020-09-01 09:58:03 +02:00
cpu.c	cpu/hotplug: Add lockdep_is_cpus_held()	2021-01-06 16:24:59 -08:00
crash_core.c	kdump: append uts_namespace.name offset to VMCOREINFO	2020-12-15 22:46:18 -08:00
crash_dump.c
cred.c
delayacct.c
dma.c
exec_domain.c
exit.c	kernel/io_uring: cancel io_uring before task works	2020-12-30 19:36:54 -07:00
extable.c
fail_function.c	fault-injection: handle EI_ETYPE_TRUE	2020-12-15 22:46:19 -08:00
fork.c	kernel: provide create_io_thread() helper	2021-03-04 15:45:03 -07:00
freezer.c
futex.c	Merge branch 'linus' into locking/core, to pick up upstream fixes	2021-02-12 12:54:58 +01:00
gen_kheaders.sh
groups.c	groups: simplify struct group_info allocation	2021-02-26 09:41:03 -08:00
hung_task.c	kernel/hung_task.c: make type annotations consistent	2020-11-02 12:14:19 -08:00
iomem.c
irq_work.c	irq_work: Optimize irq_work_single()	2020-11-24 16:47:49 +01:00
jump_label.c	jump_label: Fix usage in module __init	2020-12-18 16:53:12 +01:00
kallsyms.c	kallsyms: only build {,module_}kallsyms_on_each_symbol when required	2021-02-08 12:24:04 +01:00
kcmp.c	Merge branch 'exec-update-lock-for-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace	2020-12-15 19:36:48 -08:00
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt	preempt: Introduce CONFIG_PREEMPT_DYNAMIC	2021-02-17 14:12:24 +01:00
kcov.c	kernel: make kcov_common_handle consider the current context	2020-11-02 18:00:20 -08:00
kexec_core.c	Merge branch 'work.elf-compat' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs	2021-02-21 09:29:23 -08:00
kexec_elf.c
kexec_file.c	ima: Free IMA measurement buffer after kexec syscall	2021-02-10 15:49:38 -05:00
kexec_internal.h	kexec: move machine_kexec_post_load() to public interface	2021-02-22 12:33:26 +00:00
kexec.c	LSM: Introduce kernel_post_load_data() hook	2020-10-05 13:37:03 +02:00
kheaders.c
kmod.c	kmod: remove redundant "be an" in the comment	2020-08-12 10:58:01 -07:00
kprobes.c	kprobes: Fix to delay the kprobes jump optimization	2021-02-19 14:57:12 -05:00
ksysfs.c
kthread.c	- Correct the marking of kthreads which are supposed to run on a specific,	2021-01-24 10:09:20 -08:00
latencytop.c
Makefile	kcmp: Support selection of SYS_kcmp without CHECKPOINT_RESTORE	2021-02-16 09:59:41 +01:00
module_signature.c	module: harden ELF info handling	2021-01-19 10:24:45 +01:00
module_signing.c	module: harden ELF info handling	2021-01-19 10:24:45 +01:00
module-internal.h
module.c	module: potential uninitialized return in module_kallsyms_on_each_symbol()	2021-02-10 16:57:04 +01:00
notifier.c	notifier: Fix broken error handling pattern	2020-09-01 09:58:03 +02:00
nsproxy.c	fixes-v5.11	2020-12-14 16:40:27 -08:00
padata.c	padata: fix possible padata_works_lock deadlock	2020-09-04 17:51:55 +10:00
panic.c	panic: don't dump stack twice on warn	2020-11-14 11:26:04 -08:00
params.c	Modules updates for v5.11	2020-12-17 13:01:31 -08:00
pid_namespace.c	fixes-v5.11	2020-12-14 16:40:27 -08:00
pid.c	Merge branch 'exec-update-lock-for-v5.11' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace	2020-12-15 19:36:48 -08:00
profile.c
ptrace.c	kernel: treat PF_IO_WORKER like PF_KTHREAD for ptrace/signals	2021-02-21 17:25:22 -07:00
range.c	kernel.h: split out min()/max() et al. helpers	2020-10-16 11:11:19 -07:00
reboot.c	reboot: hide from sysfs not applicable settings	2020-12-15 22:46:19 -08:00
regset.c	regset: kill ->get()	2020-07-27 14:31:12 -04:00
relay.c	relay: allow the use of const callback structs	2020-12-15 22:46:18 -08:00
resource_kunit.c	resource: provide meaningful MODULE_LICENSE() in test suite	2020-11-25 18:52:35 +01:00
resource.c	resource: Move devmem revoke code to resource framework	2021-01-12 14:26:31 +01:00
rseq.c
scftorture.c	scftorture: Add debug output for wrong-CPU warning	2021-01-04 13:53:41 -08:00
scs.c	scs: switch to vmapped shadow stacks	2020-12-01 10:30:28 +00:00
seccomp.c	seccomp: Improve performace by optimizing rmb()	2021-02-10 12:40:11 -08:00
signal.c	kernel: treat PF_IO_WORKER like PF_KTHREAD for ptrace/signals	2021-02-21 17:25:22 -07:00
smp.c	smp: Process pending softirqs in flush_smp_call_function_from_idle()	2021-02-17 14:12:42 +01:00
smpboot.c	kthread: Extract KTHREAD_IS_PER_CPU	2021-01-22 15:09:42 +01:00
smpboot.h
softirq.c	softirq: Move do_softirq_own_stack() to generic asm header	2021-02-10 23:34:16 +01:00
stackleak.c	stackleak: let stack_erasing_sysctl take a kernel pointer buffer	2020-09-19 13:13:39 -07:00
stacktrace.c	stacktrace: Remove reliable argument from arch_stack_walk() callback	2020-09-18 14:24:16 +01:00
static_call.c	static_call: Allow module use without exposing static_call_key	2021-02-17 14:12:42 +01:00
stop_machine.c	Merge branch 'linus' into sched/core, to resolve semantic conflict	2020-11-27 11:10:50 +01:00
sys_ni.c	epoll: wire up syscall epoll_pwait2	2020-12-19 11:18:38 -08:00
sys.c	Kbuild updates for v5.12	2021-02-25 10:17:31 -08:00
sysctl-test.c
sysctl.c	sysctl.c: fix underflow value setting risk in vm_table	2021-02-26 09:41:03 -08:00
task_work.c	task_work: remove legacy TWA_SIGNAL path	2020-12-12 09:17:38 -07:00
taskstats.c	treewide: rename nla_strlcpy to nla_strscpy.	2020-11-16 08:08:54 -08:00
test_kprobes.c
torture.c	torture: Maintain torture-specific set of CPUs-online books	2021-01-06 17:17:22 -08:00
tracepoint.c	tracepoints: Code clean up	2021-02-09 12:27:29 -05:00
tsacct.c
ucount.c
uid16.c
uid16.h
umh.c	usermodehelper: reset umask to default before executing user process	2020-10-06 10:31:52 -07:00
up.c
user_namespace.c	fixes-v5.11	2020-12-14 16:40:27 -08:00
user-return-notifier.c
user.c	user: Use generic ns_common::count	2020-08-19 14:14:12 +02:00
usermode_driver.c	umd: Stop using split_argv	2020-07-07 11:58:59 -05:00
utsname_sysctl.c
utsname.c	uts: Use generic ns_common::count	2020-08-19 14:13:20 +02:00
watch_queue.c	watch_queue: rectify kernel-doc for init_watch()	2021-01-26 11:16:34 +00:00
watchdog_hld.c
watchdog.c	kernel/watchdog: fix watchdog_allowed_mask not used warning	2020-11-14 11:26:03 -08:00
workqueue_internal.h
workqueue.c	Merge branch 'for-5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq	2021-02-22 17:06:54 -08:00