linux/kernel/bpf
Yonghong Song 36d8bdf75a bpf: Add alloc/xchg/direct_access support for local percpu kptr
Add two new kfunc's, bpf_percpu_obj_new_impl() and
bpf_percpu_obj_drop_impl(), to allocate a percpu obj.
Two functions are very similar to bpf_obj_new_impl()
and bpf_obj_drop_impl(). The major difference is related
to percpu handling.

    bpf_rcu_read_lock()
    struct val_t __percpu_kptr *v = map_val->percpu_data;
    ...
    bpf_rcu_read_unlock()

For a percpu data map_val like above 'v', the reg->type
is set as
	PTR_TO_BTF_ID | MEM_PERCPU | MEM_RCU
if inside rcu critical section.

MEM_RCU marking here is similar to NON_OWN_REF as 'v'
is not a owning reference. But NON_OWN_REF is
trusted and typically inside the spinlock while
MEM_RCU is under rcu read lock. RCU is preferred here
since percpu data structures mean potential concurrent
access into its contents.

Also, bpf_percpu_obj_new_impl() is restricted such that
no pointers or special fields are allowed. Therefore,
the bpf_list_head and bpf_rb_root will not be supported
in this patch set to avoid potential memory leak issue
due to racing between bpf_obj_free_fields() and another
bpf_kptr_xchg() moving an allocated object to
bpf_list_head and bpf_rb_root.

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20230827152744.1996739-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-09-08 08:42:17 -07:00
..
preload bpf: make preloaded map iterators to display map elements count 2023-07-06 12:42:25 -07:00
arraymap.c bpf: return long from bpf_map_ops funcs 2023-03-22 15:11:30 -07:00
bloom_filter.c bpf: Centralize permissions checks for all BPF map types 2023-06-19 14:04:04 +02:00
bpf_cgrp_storage.c bpf: Teach verifier that certain helpers accept NULL pointer. 2023-04-04 16:57:16 -07:00
bpf_inode_storage.c Networking changes for 6.4. 2023-04-26 16:07:23 -07:00
bpf_iter.c bpf: implement numbers iterator 2023-03-08 16:19:51 -08:00
bpf_local_storage.c bpf: bpf_sk_storage: Fix the missing uncharge in sk_omem_alloc 2023-09-06 11:08:14 +02:00
bpf_lru_list.c bpf: Address KCSAN report on bpf_lru_list 2023-05-12 12:01:03 -07:00
bpf_lru_list.h bpf: lru: Remove unused declaration bpf_lru_promote() 2023-08-08 17:21:42 -07:00
bpf_lsm.c bpf: Fix the kernel crash caused by bpf_setsockopt(). 2023-01-26 23:26:40 -08:00
bpf_struct_ops_types.h bpf: Add dummy BPF STRUCT_OPS for test purpose 2021-11-01 14:10:00 -07:00
bpf_struct_ops.c bpf: Support default .validate() and .update() behavior for struct_ops links 2023-08-14 22:23:39 -07:00
bpf_task_storage.c bpf: Teach verifier that certain helpers accept NULL pointer. 2023-04-04 16:57:16 -07:00
btf.c bpf: Add BPF_KPTR_PERCPU as a field type 2023-09-08 08:42:17 -07:00
cgroup_iter.c cgroup: bpf: use cgroup_lock()/cgroup_unlock() wrappers 2023-03-17 12:07:13 -10:00
cgroup.c bpf-next-for-netdev 2023-05-16 19:50:05 -07:00
core.c bpf: Add support for non-fix-size percpu mem allocation 2023-09-08 08:42:17 -07:00
cpumap.c bpf, cpumask: Clean up bpf_cpu_map_entry directly in cpu_map_free 2023-08-21 15:21:16 -07:00
cpumask.c bpf: Convert bpf_cpumask to bpf_mem_cache_free_rcu. 2023-07-12 23:45:23 +02:00
devmap.c bpf, devmap: Remove unused dtab field from bpf_dtab_netdev 2023-07-31 18:26:08 -07:00
disasm.c bpf: change bpf_alu_sign_string and bpf_movsx_string to static 2023-08-04 16:15:50 -07:00
disasm.h bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause 2021-09-02 14:49:23 +02:00
dispatcher.c bpf: Synchronize dispatcher update with bpf_dispatcher_xdp_func 2022-12-14 12:02:14 -08:00
hashtab.c bpf: populate the per-cpu insertions/deletions counters for hashmaps 2023-07-06 12:42:25 -07:00
helpers.c bpf: Add alloc/xchg/direct_access support for local percpu kptr 2023-09-08 08:42:17 -07:00
inode.c bpf: convert to ctime accessor functions 2023-07-24 10:30:07 +02:00
Kconfig bpf: Add fd-based tcx multi-prog infra with link support 2023-07-19 10:07:27 -07:00
link_iter.c bpf: Add bpf_link iterator 2022-05-10 11:20:45 -07:00
local_storage.c cgroup changes for v6.4-rc1 2023-04-29 10:05:22 -07:00
log.c bpf: drop unnecessary user-triggerable WARN_ONCE in verifierl log 2023-05-16 22:34:50 -07:00
lpm_trie.c bpf: Centralize permissions checks for all BPF map types 2023-06-19 14:04:04 +02:00
Makefile bpf: Add fd-based tcx multi-prog infra with link support 2023-07-19 10:07:27 -07:00
map_in_map.c bpf: Fix elem_size not being set for inner maps 2023-06-02 16:22:12 -07:00
map_in_map.h
map_iter.c bpf: allow any program to use the bpf_map_sum_elem_count kfunc 2023-07-19 09:48:53 -07:00
memalloc.c bpf: Add support for non-fix-size percpu mem allocation 2023-09-08 08:42:17 -07:00
mmap_unlock_work.h bpf: Introduce helper bpf_find_vma 2021-11-07 11:54:51 -08:00
mprog.c bpf: Fix mprog detachment for empty mprog entry 2023-08-04 09:35:39 -07:00
net_namespace.c net: Add includes masked by netdevice.h including uapi/bpf.h 2021-12-29 20:03:05 -08:00
offload.c net: invert the netdevice.h vs xdp.h dependency 2023-08-03 08:38:07 -07:00
percpu_freelist.c bpf: Initialize same number of free nodes for each pcpu_freelist 2022-11-11 12:05:14 -08:00
percpu_freelist.h
prog_iter.c
queue_stack_maps.c bpf: Centralize permissions checks for all BPF map types 2023-06-19 14:04:04 +02:00
reuseport_array.c bpf: Centralize permissions checks for all BPF map types 2023-06-19 14:04:04 +02:00
ringbuf.c bpf: Remove unnecessary ring buffer size check 2023-07-05 14:09:45 +02:00
stackmap.c bpf: Centralize permissions checks for all BPF map types 2023-06-19 14:04:04 +02:00
syscall.c bpf: Add BPF_KPTR_PERCPU as a field type 2023-09-08 08:42:17 -07:00
sysfs_btf.c
task_iter.c bpf: keep a reference to the mm, in case the task is dead. 2022-12-28 14:11:48 -08:00
tcx.c tcx: Fix splat during dev unregister 2023-07-31 14:44:02 -07:00
tnum.c bpf, tnums: Provably sound, faster, and more precise algorithm for tnum_mul 2021-06-01 13:34:15 +02:00
trampoline.c bpf: Assign bpf_tramp_run_ctx::saved_run_ctx before recursion check. 2023-09-06 10:44:28 +02:00
verifier.c bpf: Add alloc/xchg/direct_access support for local percpu kptr 2023-09-08 08:42:17 -07:00