linux/kernel/bpf
Cong Wang 53f523f305 bpf: Clear percpu pointers in bpf_prog_clone_free()
Similar to bpf_prog_realloc(), bpf_prog_clone_create() also copies
the percpu pointers, but the clone still shares them with the original
prog, so we have to clear these two percpu pointers in
bpf_prog_clone_free(). Otherwise we would get a double free:

 BUG: kernel NULL pointer dereference, address: 0000000000000000
 #PF: supervisor read access in kernel mode
 #PF: error_code(0x0000) - not-present page
 PGD 0 P4D 0
 Oops: 0000 [#1] SMP PTI
 CPU: 13 PID: 8140 Comm: kworker/13:247 Kdump: loaded Tainted: G                W    OE
  5.11.0-rc4.bm.1-amd64+ #1
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
 test_bpf: #1 TXA
 Workqueue: events bpf_prog_free_deferred
 RIP: 0010:percpu_ref_get_many.constprop.97+0x42/0xf0
 Code: [...]
 RSP: 0018:ffffa6bce1f9bda0 EFLAGS: 00010002
 RAX: 0000000000000001 RBX: 0000000000000000 RCX: 00000000021dfc7b
 RDX: ffffffffae2eeb90 RSI: 867f92637e338da5 RDI: 0000000000000046
 RBP: ffffa6bce1f9bda8 R08: 0000000000000000 R09: 0000000000000001
 R10: 0000000000000046 R11: 0000000000000000 R12: 0000000000000280
 R13: 0000000000000000 R14: 0000000000000000 R15: ffff9b5f3ffdedc0
 FS:    0000000000000000(0000) GS:ffff9b5f2fb40000(0000) knlGS:0000000000000000
 CS:    0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 CR2: 0000000000000000 CR3: 000000027c36c002 CR4: 00000000003706e0
 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
 Call Trace:
    refill_obj_stock+0x5e/0xd0
    free_percpu+0xee/0x550
    __bpf_prog_free+0x4d/0x60
    process_one_work+0x26a/0x590
    worker_thread+0x3c/0x390
    ? process_one_work+0x590/0x590
    kthread+0x130/0x150
    ? kthread_park+0x80/0x80
    ret_from_fork+0x1f/0x30

This bug is 100% reproducible with test_kmod.sh.

Fixes: 700d4796ef ("bpf: Optimize program stats")
Fixes: ca06f55b90 ("bpf: Add per-program recursion prevention mechanism")
Reported-by: Jiang Wang <jiang.wang@bytedance.com>
Signed-off-by: Cong Wang <cong.wang@bytedance.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20210218001647.71631-1-xiyou.wangcong@gmail.com
2021-02-22 18:08:35 +01:00
..
preload Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2021-02-02 14:21:31 -08:00
arraymap.c bpf: Eliminate rlimit-based memory accounting for arraymap maps 2020-12-02 18:32:46 -08:00
bpf_inode_storage.c bpf, inode_storage: Put file handler if no storage was found 2021-01-22 23:19:24 +01:00
bpf_iter.c bpf: Fix an unitialized value in bpf_iter 2021-02-12 13:33:50 -08:00
bpf_local_storage.c bpf: Eliminate rlimit-based memory accounting for bpf local storage maps 2020-12-02 18:32:47 -08:00
bpf_lru_list.c bpf_lru_list: Read double-checked variable once without lock 2021-02-10 15:54:26 -08:00
bpf_lru_list.h bpf: Fix a typo "inacitve" -> "inactive" 2020-04-06 21:54:10 +02:00
bpf_lsm.c bpf: Drop disabled LSM hooks from the sleepable set 2021-01-26 17:08:50 +01:00
bpf_struct_ops_types.h bpf: tcp: Support tcp_congestion_ops in bpf 2020-01-09 08:46:18 -08:00
bpf_struct_ops.c bpf: Eliminate rlimit-based memory accounting for bpf_struct_ops maps 2020-12-02 18:32:46 -08:00
bpf_task_storage.c bpf: Local storage helpers should check nullness of owner ptr passed 2021-01-12 16:07:56 +01:00
btf.c bpf: Support pointers in global func args 2021-02-12 17:37:23 -08:00
cgroup.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2021-02-16 13:14:06 -08:00
core.c bpf: Clear percpu pointers in bpf_prog_clone_free() 2021-02-22 18:08:35 +01:00
cpumap.c net, xdp: Introduce __xdp_build_skb_from_frame utility routine 2021-01-20 14:10:35 -08:00
devmap.c bpf, devmap: Use GFP_KERNEL for xdp bulk queue allocation 2021-02-13 00:11:26 +01:00
disasm.c bpf: Change 'BPF_ADD' to 'BPF_AND' in print_bpf_insn() 2021-01-27 22:23:46 +01:00
disasm.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 295 2019-06-05 17:36:38 +02:00
dispatcher.c bpf: Remove bpf_image tree 2020-03-13 12:49:52 -07:00
hashtab.c bpf: Allows per-cpu maps and map-in-map in sleepable programs 2021-02-11 16:19:26 +01:00
helpers.c bpf: Simplify cases in bpf_base_func_proto 2021-01-29 02:20:28 +01:00
inode.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2020-09-22 16:45:34 -07:00
local_storage.c bpf: Eliminate rlimit-based memory accounting for cgroup storage maps 2020-12-02 18:32:46 -08:00
lpm_trie.c bpf: Eliminate rlimit-based memory accounting for lpm_trie maps 2020-12-02 18:32:46 -08:00
Makefile Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2020-11-14 09:13:41 -08:00
map_in_map.c bpf: Relax max_entries check for most of the inner map types 2020-08-28 15:41:30 +02:00
map_in_map.h bpf: Add map_meta_equal map ops 2020-08-28 15:41:30 +02:00
map_iter.c bpf: Implement link_query callbacks in map element iterators 2020-08-21 14:01:39 -07:00
net_namespace.c bpf: Add support for forced LINK_DETACH command 2020-08-01 20:38:28 -07:00
offload.c bpf, offload: Replace bitwise AND by logical AND in bpf_prog_offload_info_fill 2020-02-17 16:53:49 +01:00
percpu_freelist.c bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI 2020-10-06 00:04:11 +02:00
percpu_freelist.h bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI 2020-10-06 00:04:11 +02:00
prog_iter.c bpf: Refactor bpf_iter_reg to have separate seq_info member 2020-07-25 20:16:32 -07:00
queue_stack_maps.c bpf: Eliminate rlimit-based memory accounting for queue_stack_maps maps 2020-12-02 18:32:46 -08:00
reuseport_array.c bpf: Eliminate rlimit-based memory accounting for reuseport_array maps 2020-12-02 18:32:47 -08:00
ringbuf.c bpf: Eliminate rlimit-based memory accounting for bpf ringbuffer 2020-12-02 18:32:47 -08:00
stackmap.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2021-02-10 13:30:12 -08:00
syscall.c bpf: Count the number of times recursion was prevented 2021-02-11 16:19:20 +01:00
sysfs_btf.c bpf: Load and verify kernel module BTFs 2020-11-10 15:25:53 -08:00
task_iter.c bpf: Introduce task_vma bpf_iter 2021-02-12 12:56:53 -08:00
tnum.c bpf: Verifier, do explicit ALU32 bounds tracking 2020-03-30 14:59:53 -07:00
trampoline.c bpf: Count the number of times recursion was prevented 2021-02-11 16:19:20 +01:00
verifier.c bpf: Fix a warning message in mark_ptr_not_null_reg() 2021-02-22 18:03:11 +01:00