linux/kernel/bpf
Stanislav Fomichev 055eb95533 bpf: Move rcu lock management out of BPF_PROG_RUN routines
Commit 7d08c2c911 ("bpf: Refactor BPF_PROG_RUN_ARRAY family of macros
into functions") switched a bunch of BPF_PROG_RUN macros to inline
routines. This changed the semantic a bit. Due to arguments expansion
of macros, it used to be:

	rcu_read_lock();
	array = rcu_dereference(cgrp->bpf.effective[atype]);
	...

Now, with with inline routines, we have:
	array_rcu = rcu_dereference(cgrp->bpf.effective[atype]);
	/* array_rcu can be kfree'd here */
	rcu_read_lock();
	array = rcu_dereference(array_rcu);

I'm assuming in practice rcu subsystem isn't fast enough to trigger
this but let's use rcu API properly.

Also, rename to lower caps to not confuse with macros. Additionally,
drop and expand BPF_PROG_CGROUP_INET_EGRESS_RUN_ARRAY.

See [1] for more context.

  [1] https://lore.kernel.org/bpf/CAKH8qBs60fOinFdxiiQikK_q0EcVxGvNTQoWvHLEUGbgcj1UYg@mail.gmail.com/T/#u

v2
- keep rcu locks inside by passing cgroup_bpf

Fixes: 7d08c2c911 ("bpf: Refactor BPF_PROG_RUN_ARRAY family of macros into functions")
Signed-off-by: Stanislav Fomichev <sdf@google.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Link: https://lore.kernel.org/bpf/20220414161233.170780-1-sdf@google.com
2022-04-19 09:45:47 -07:00
..
preload bpf: Remove redundant slash 2022-03-07 22:19:32 -08:00
arraymap.c bpf: generalise tail call map compatibility check 2022-01-21 14:14:03 -08:00
bloom_filter.c bpf: Add missing map_get_next_key method to bloom filter map. 2021-12-29 09:38:31 -08:00
bpf_inode_storage.c bpf: Enable non-atomic allocations in local storage 2022-03-20 18:55:05 -07:00
bpf_iter.c bpf: Remove redundant assignment to meta.seq in __task_seq_show() 2022-04-11 21:14:34 +02:00
bpf_local_storage.c bpf: Enable non-atomic allocations in local storage 2022-03-20 18:55:05 -07:00
bpf_lru_list.c bpf_lru_list: Read double-checked variable once without lock 2021-02-10 15:54:26 -08:00
bpf_lru_list.h bpf: Fix a typo "inacitve" -> "inactive" 2020-04-06 21:54:10 +02:00
bpf_lsm.c bpf-lsm: Make bpf_lsm_kernel_read_file() as sleepable 2022-03-10 18:57:55 -08:00
bpf_struct_ops_types.h bpf: Add dummy BPF STRUCT_OPS for test purpose 2021-11-01 14:10:00 -07:00
bpf_struct_ops.c bpf: Remove unnecessary type castings 2022-04-14 01:02:24 +02:00
bpf_task_storage.c bpf: Enable non-atomic allocations in local storage 2022-03-20 18:55:05 -07:00
btf.c bpf: Fix maximum permitted number of arguments check 2022-03-28 19:08:17 -07:00
cgroup.c bpf: Move rcu lock management out of BPF_PROG_RUN routines 2022-04-19 09:45:47 -07:00
core.c bpf: Fix bpf_prog_pack when PMU_SIZE is not defined 2022-03-21 13:53:45 -07:00
cpumap.c bpf: generalise tail call map compatibility check 2022-01-21 14:14:03 -08:00
devmap.c bpf: generalise tail call map compatibility check 2022-01-21 14:14:03 -08:00
disasm.c bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause 2021-09-02 14:49:23 +02:00
disasm.h bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause 2021-09-02 14:49:23 +02:00
dispatcher.c bpf: Remove bpf_image tree 2020-03-13 12:49:52 -07:00
hashtab.c bpf: Remove unnecessary type castings 2022-04-14 01:02:24 +02:00
helpers.c bpf: Replace strncpy() with strscpy() 2022-03-07 22:04:33 -08:00
inode.c bpf: Convert bpf_preload.ko to use light skeleton. 2022-02-10 23:31:51 +01:00
Kconfig bpf: Add "live packet" mode for XDP in BPF_PROG_RUN 2022-03-09 14:19:22 -08:00
local_storage.c bpf: Cleanup comments 2022-02-23 15:17:51 -08:00
lpm_trie.c bpf: Fix typo in a comment in bpf lpm_trie. 2021-12-30 18:42:34 -08:00
Makefile bpf: Prepare relo_core.c for kernel duty. 2021-12-02 11:18:34 -08:00
map_in_map.c bpf: Remember BTF of inner maps. 2021-07-15 22:31:10 +02:00
map_in_map.h bpf: Add map_meta_equal map ops 2020-08-28 15:41:30 +02:00
map_iter.c bpf: Introduce MEM_RDONLY flag 2021-12-18 13:27:41 -08:00
mmap_unlock_work.h bpf: Introduce helper bpf_find_vma 2021-11-07 11:54:51 -08:00
net_namespace.c net: Add includes masked by netdevice.h including uapi/bpf.h 2021-12-29 20:03:05 -08:00
offload.c bpf, offload: Replace bitwise AND by logical AND in bpf_prog_offload_info_fill 2020-02-17 16:53:49 +01:00
percpu_freelist.c bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI 2020-10-06 00:04:11 +02:00
percpu_freelist.h bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI 2020-10-06 00:04:11 +02:00
prog_iter.c bpf: Refactor bpf_iter_reg to have separate seq_info member 2020-07-25 20:16:32 -07:00
queue_stack_maps.c bpf: Eliminate rlimit-based memory accounting for queue_stack_maps maps 2020-12-02 18:32:46 -08:00
reuseport_array.c bpf: Cleanup comments 2022-02-23 15:17:51 -08:00
ringbuf.c bpf: Use VM_MAP instead of VM_ALLOC for ringbuf 2022-02-02 23:15:24 -08:00
stackmap.c bpf: Fix excessive memory allocation in stack_map_alloc() 2022-04-09 00:28:21 +02:00
syscall.c bpf: Move BPF sysctls from kernel/sysctl.c to BPF core 2022-04-13 21:36:56 +02:00
sysfs_btf.c bpf: Load and verify kernel module BTFs 2020-11-10 15:25:53 -08:00
task_iter.c bpf: Remove redundant assignment to meta.seq in __task_seq_show() 2022-04-11 21:14:34 +02:00
tnum.c bpf, tnums: Provably sound, faster, and more precise algorithm for tnum_mul 2021-06-01 13:34:15 +02:00
trampoline.c Add support for Intel CET-IBT, available since Tigerlake (11th gen), which is a 2022-03-27 10:17:23 -07:00
verifier.c bpf: Reject writes for PTR_TO_MAP_KEY in check_helper_mem_access 2022-04-06 10:32:12 -07:00