linux/kernel/bpf
Hao Luo ceac059ed4 bpf: Cache the last valid build_id
For binaries that are statically linked, consecutive stack frames are
likely to be in the same VMA and therefore have the same build id.

On a real-world workload, we observed that 66% of CPU cycles in
__bpf_get_stackid() were spent on build_id_parse() and find_vma().

As an optimization for this case, we can cache the previous frame's
VMA, if the new frame has the same VMA as the previous one, reuse the
previous one's build id.

We are holding the MM locks as reader across the entire loop, so we
don't need to worry about VMA going away.

Tested through "stacktrace_build_id" and "stacktrace_build_id_nmi" in
test_progs.

Suggested-by: Greg Thelen <gthelen@google.com>
Signed-off-by: Hao Luo <haoluo@google.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Song Liu <songliubraving@fb.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Link: https://lore.kernel.org/bpf/20220224000531.1265030-1-haoluo@google.com
2022-02-28 18:10:28 +01:00
..
preload bpf: Fix issue with bpf preload module taking over stdout/stdin of kernel. 2022-02-25 12:48:35 -08:00
arraymap.c bpf: generalise tail call map compatibility check 2022-01-21 14:14:03 -08:00
bloom_filter.c bpf: Add missing map_get_next_key method to bloom filter map. 2021-12-29 09:38:31 -08:00
bpf_inode_storage.c bpf: Allow bpf_local_storage to be used by sleepable programs 2021-12-29 17:54:40 -08:00
bpf_iter.c bpf: Add support for bpf iterator programs to use sleepable helpers 2022-01-24 19:55:40 -08:00
bpf_local_storage.c bpf: Cleanup comments 2022-02-23 15:17:51 -08:00
bpf_lru_list.c bpf_lru_list: Read double-checked variable once without lock 2021-02-10 15:54:26 -08:00
bpf_lru_list.h bpf: Fix a typo "inacitve" -> "inactive" 2020-04-06 21:54:10 +02:00
bpf_lsm.c bpf: Fix renaming task_getsecid_subj->current_getsecid_subj. 2022-01-24 20:20:51 -08:00
bpf_struct_ops_types.h bpf: Add dummy BPF STRUCT_OPS for test purpose 2021-11-01 14:10:00 -07:00
bpf_struct_ops.c bpf: Rename btf_member accessors. 2021-12-02 11:18:34 -08:00
bpf_task_storage.c bpf: Allow bpf_local_storage to be used by sleepable programs 2021-12-29 17:54:40 -08:00
btf.c bpf: Add config to allow loading modules with BTF mismatches 2022-02-28 14:17:10 +01:00
cgroup.c bpf: Cleanup comments 2022-02-23 15:17:51 -08:00
core.c bpf: bpf_prog_pack: Set proper size before freeing ro_header 2022-02-17 13:15:36 -08:00
cpumap.c bpf: generalise tail call map compatibility check 2022-01-21 14:14:03 -08:00
devmap.c bpf: generalise tail call map compatibility check 2022-01-21 14:14:03 -08:00
disasm.c bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause 2021-09-02 14:49:23 +02:00
disasm.h bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause 2021-09-02 14:49:23 +02:00
dispatcher.c bpf: Remove bpf_image tree 2020-03-13 12:49:52 -07:00
hashtab.c bpf: Cleanup comments 2022-02-23 15:17:51 -08:00
helpers.c bpf: Cleanup comments 2022-02-23 15:17:51 -08:00
inode.c bpf: Convert bpf_preload.ko to use light skeleton. 2022-02-10 23:31:51 +01:00
Kconfig bpf: Disallow unprivileged bpf by default 2021-11-01 17:06:47 +01:00
local_storage.c bpf: Cleanup comments 2022-02-23 15:17:51 -08:00
lpm_trie.c bpf: Fix typo in a comment in bpf lpm_trie. 2021-12-30 18:42:34 -08:00
Makefile bpf: Prepare relo_core.c for kernel duty. 2021-12-02 11:18:34 -08:00
map_in_map.c bpf: Remember BTF of inner maps. 2021-07-15 22:31:10 +02:00
map_in_map.h bpf: Add map_meta_equal map ops 2020-08-28 15:41:30 +02:00
map_iter.c bpf: Introduce MEM_RDONLY flag 2021-12-18 13:27:41 -08:00
mmap_unlock_work.h bpf: Introduce helper bpf_find_vma 2021-11-07 11:54:51 -08:00
net_namespace.c net: Add includes masked by netdevice.h including uapi/bpf.h 2021-12-29 20:03:05 -08:00
offload.c bpf, offload: Replace bitwise AND by logical AND in bpf_prog_offload_info_fill 2020-02-17 16:53:49 +01:00
percpu_freelist.c bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI 2020-10-06 00:04:11 +02:00
percpu_freelist.h bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI 2020-10-06 00:04:11 +02:00
prog_iter.c bpf: Refactor bpf_iter_reg to have separate seq_info member 2020-07-25 20:16:32 -07:00
queue_stack_maps.c bpf: Eliminate rlimit-based memory accounting for queue_stack_maps maps 2020-12-02 18:32:46 -08:00
reuseport_array.c bpf: Cleanup comments 2022-02-23 15:17:51 -08:00
ringbuf.c bpf: Use VM_MAP instead of VM_ALLOC for ringbuf 2022-02-02 23:15:24 -08:00
stackmap.c bpf: Cache the last valid build_id 2022-02-28 18:10:28 +01:00
syscall.c bpf: Cleanup comments 2022-02-23 15:17:51 -08:00
sysfs_btf.c bpf: Load and verify kernel module BTFs 2020-11-10 15:25:53 -08:00
task_iter.c bpf: Introduce btf_tracing_ids 2021-11-12 10:19:09 -08:00
tnum.c bpf, tnums: Provably sound, faster, and more precise algorithm for tnum_mul 2021-06-01 13:34:15 +02:00
trampoline.c bpf: Cleanup comments 2022-02-23 15:17:51 -08:00
verifier.c bpf: Reject kfunc calls that overflow insn->imm 2022-02-15 10:05:11 -08:00