Chuyi Zhou dfab99df14 bpf: teach the verifier to enforce css_iter and task_iter in RCU CS ... css_iter and task_iter should be used in rcu section. Specifically, in sleepable progs explicit bpf_rcu_read_lock() is needed before use these iters. In normal bpf progs that have implicit rcu_read_lock(), it's OK to use them directly. This patch adds a new a KF flag KF_RCU_PROTECTED for bpf_iter_task_new and bpf_iter_css_new. It means the kfunc should be used in RCU CS. We check whether we are in rcu cs before we want to invoke this kfunc. If the rcu protection is guaranteed, we would let st->type = PTR_TO_STACK | MEM_RCU. Once user do rcu_unlock during the iteration, state MEM_RCU of regs would be cleared. is_iter_reg_valid_init() will reject if reg->type is UNTRUSTED. It is worth noting that currently, bpf_rcu_read_unlock does not clear the state of the STACK_ITER reg, since bpf_for_each_spilled_reg only considers STACK_SPILL. This patch also let bpf_for_each_spilled_reg search STACK_ITER. Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20231018061746.111364-6-zhouchuyi@bytedance.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>		2023-10-19 17:02:46 -07:00
..
preload	bpf: make preloaded map iterators to display map elements count	2023-07-06 12:42:25 -07:00
arraymap.c	bpf: return long from bpf_map_ops funcs	2023-03-22 15:11:30 -07:00
bloom_filter.c	bpf: Centralize permissions checks for all BPF map types	2023-06-19 14:04:04 +02:00
bpf_cgrp_storage.c	bpf: Teach verifier that certain helpers accept NULL pointer.	2023-04-04 16:57:16 -07:00
bpf_inode_storage.c	Networking changes for 6.4.	2023-04-26 16:07:23 -07:00
bpf_iter.c	bpf: Don't explicitly emit BTF for struct btf_iter_num	2023-10-13 15:48:58 -07:00
bpf_local_storage.c	bpf: bpf_sk_storage: Fix the missing uncharge in sk_omem_alloc	2023-09-06 11:08:14 +02:00
bpf_lru_list.c	bpf: Address KCSAN report on bpf_lru_list	2023-05-12 12:01:03 -07:00
bpf_lru_list.h	bpf: lru: Remove unused declaration bpf_lru_promote()	2023-08-08 17:21:42 -07:00
bpf_lsm.c	bpf: Fix the kernel crash caused by bpf_setsockopt().	2023-01-26 23:26:40 -08:00
bpf_struct_ops_types.h	bpf: Add dummy BPF STRUCT_OPS for test purpose	2021-11-01 14:10:00 -07:00
bpf_struct_ops.c	bpf: Charge modmem for struct_ops trampoline	2023-09-14 15:30:45 -07:00
bpf_task_storage.c	bpf: Teach verifier that certain helpers accept NULL pointer.	2023-04-04 16:57:16 -07:00
btf.c	bpf: Add bpf_sock_addr_set_sun_path() to allow writing unix sockaddr from bpf	2023-10-11 16:29:25 -07:00
cgroup_iter.c	bpf: Introduce css open-coded iterator kfuncs	2023-10-19 17:02:46 -07:00
cgroup.c	bpf: Implement cgroup sockaddr hooks for unix sockets	2023-10-11 17:27:47 -07:00
core.c	Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next	2023-09-17 15:12:06 +01:00
cpumap.c	net, bpf: Add a warning if NAPI cb missed xdp_do_flush().	2023-10-17 15:02:03 +02:00
cpumask.c	bpf: Convert bpf_cpumask to bpf_mem_cache_free_rcu.	2023-07-12 23:45:23 +02:00
devmap.c	net, bpf: Add a warning if NAPI cb missed xdp_do_flush().	2023-10-17 15:02:03 +02:00
disasm.c	bpf: change bpf_alu_sign_string and bpf_movsx_string to static	2023-08-04 16:15:50 -07:00
disasm.h	bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause	2021-09-02 14:49:23 +02:00
dispatcher.c	bpf: Synchronize dispatcher update with bpf_dispatcher_xdp_func	2022-12-14 12:02:14 -08:00
hashtab.c	bpf: populate the per-cpu insertions/deletions counters for hashmaps	2023-07-06 12:42:25 -07:00
helpers.c	bpf: teach the verifier to enforce css_iter and task_iter in RCU CS	2023-10-19 17:02:46 -07:00
inode.c	bpf: convert to ctime accessor functions	2023-07-24 10:30:07 +02:00
Kconfig	bpf: Add fd-based tcx multi-prog infra with link support	2023-07-19 10:07:27 -07:00
link_iter.c	bpf: Add bpf_link iterator	2022-05-10 11:20:45 -07:00
local_storage.c	cgroup changes for v6.4-rc1	2023-04-29 10:05:22 -07:00
log.c	bpf: drop unnecessary user-triggerable WARN_ONCE in verifierl log	2023-05-16 22:34:50 -07:00
lpm_trie.c	bpf: Centralize permissions checks for all BPF map types	2023-06-19 14:04:04 +02:00
Makefile	bpf: Add fd-based tcx multi-prog infra with link support	2023-07-19 10:07:27 -07:00
map_in_map.c	bpf: Fix elem_size not being set for inner maps	2023-06-02 16:22:12 -07:00
map_in_map.h
map_iter.c	bpf: allow any program to use the bpf_map_sum_elem_count kfunc	2023-07-19 09:48:53 -07:00
memalloc.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net	2023-10-05 13:16:47 -07:00
mmap_unlock_work.h	bpf: Introduce helper bpf_find_vma	2021-11-07 11:54:51 -08:00
mprog.c	bpf: Handle bpf_mprog_query with NULL entry	2023-10-06 17:11:20 -07:00
net_namespace.c	net: Add includes masked by netdevice.h including uapi/bpf.h	2021-12-29 20:03:05 -08:00
offload.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net	2023-09-21 21:49:45 +02:00
percpu_freelist.c	bpf: Initialize same number of free nodes for each pcpu_freelist	2022-11-11 12:05:14 -08:00
percpu_freelist.h
prog_iter.c
queue_stack_maps.c	bpf: Avoid deadlock when using queue and stack maps from NMI	2023-09-11 19:04:49 -07:00
reuseport_array.c	bpf: Centralize permissions checks for all BPF map types	2023-06-19 14:04:04 +02:00
ringbuf.c	bpf: Remove unnecessary ring buffer size check	2023-07-05 14:09:45 +02:00
stackmap.c	bpf: Annotate struct bpf_stack_map with __counted_by	2023-10-06 23:44:35 +02:00
syscall.c	bpf-next-for-netdev	2023-10-16 21:05:33 -07:00
sysfs_btf.c
task_iter.c	bpf: Introduce task open coded iterator kfuncs	2023-10-19 17:02:46 -07:00
tcx.c	bpf: Handle bpf_mprog_query with NULL entry	2023-10-06 17:11:20 -07:00
tnum.c
trampoline.c	bpf, x64: Fix tailcall infinite loop	2023-09-12 13:06:12 -07:00
verifier.c	bpf: teach the verifier to enforce css_iter and task_iter in RCU CS	2023-10-19 17:02:46 -07:00