linux/kernel/bpf
Alexei Starovoitov bfc03c15be bpf: Remove usage of kmem_cache from bpf_mem_cache.
For bpf_mem_cache based hash maps the following stress test:
for (i = 1; i <= 512; i <<= 1)
  for (j = 1; j <= 1 << 18; j <<= 1)
    fd = bpf_map_create(BPF_MAP_TYPE_HASH, NULL, i, j, 2, 0);
creates many kmem_cache-s that are not mergeable in debug kernels
and consume unnecessary amount of memory.
Turned out bpf_mem_cache's free_list logic does batching well,
so usage of kmem_cache for fixes size allocations doesn't bring
any performance benefits vs normal kmalloc.
Hence get rid of kmem_cache in bpf_mem_cache.
That saves memory, speeds up map create/destroy operations,
while maintains hash map update/delete performance.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20220902211058.60789-16-alexei.starovoitov@gmail.com
2022-09-05 15:33:07 +02:00
..
preload bpf: iterators: Build and use lightweight bootstrap version of bpftool 2022-07-15 12:01:30 -07:00
arraymap.c bpf: Acquire map uref in .init_seq_private for array map iterator 2022-08-10 10:12:47 -07:00
bloom_filter.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
bpf_inode_storage.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
bpf_iter.c bpf: Initialize the bpf_run_ctx in bpf_iter_run_prog() 2022-08-18 17:06:13 -07:00
bpf_local_storage.c bpf: Use this_cpu_{inc|dec|inc_return} for bpf_task_storage_busy 2022-09-01 12:16:12 -07:00
bpf_lru_list.c bpf_lru_list: Read double-checked variable once without lock 2021-02-10 15:54:26 -08:00
bpf_lru_list.h printk: stop including cache.h from printk.h 2022-05-13 07:20:07 -07:00
bpf_lsm.c bpf: Only add BTF IDs for socket security hooks when CONFIG_SECURITY_NETWORK is on 2022-09-01 16:21:14 -07:00
bpf_struct_ops_types.h bpf: Add dummy BPF STRUCT_OPS for test purpose 2021-11-01 14:10:00 -07:00
bpf_struct_ops.c bpf: Remove is_valid_bpf_tramp_flags() 2022-07-11 21:04:58 +02:00
bpf_task_storage.c bpf: Use this_cpu_{inc|dec|inc_return} for bpf_task_storage_busy 2022-09-01 12:16:12 -07:00
btf.c bpf: Always return corresponding btf_type in __get_type_size() 2022-08-09 10:30:45 -07:00
cgroup_iter.c bpf: Remove useless else if 2022-09-01 14:04:07 -07:00
cgroup.c bpf: expose bpf_strtol and bpf_strtoul to all program types 2022-08-23 16:08:21 -07:00
core.c bpf: Add BPF-helper for accessing CLOCK_TAI 2022-08-09 09:47:13 -07:00
cpumap.c bpf: Use bpf_map_area_alloc consistently on bpf map creation 2022-08-10 11:50:43 -07:00
devmap.c bpf: Use bpf_map_area_alloc consistently on bpf map creation 2022-08-10 11:50:43 -07:00
disasm.c bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause 2021-09-02 14:49:23 +02:00
disasm.h bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause 2021-09-02 14:49:23 +02:00
dispatcher.c bpf: Remove bpf_image tree 2020-03-13 12:49:52 -07:00
hashtab.c bpf: Convert percpu hash map to per-cpu bpf_mem_alloc. 2022-09-05 15:33:06 +02:00
helpers.c bpf: Move bpf_loop and bpf_for_each_map_elem under CAP_BPF 2022-08-23 16:21:59 -07:00
inode.c bpf: Convert bpf_preload.ko to use light skeleton. 2022-02-10 23:31:51 +01:00
Kconfig rcu: Make the TASKS_RCU Kconfig option be selected 2022-04-20 16:52:58 -07:00
link_iter.c bpf: Add bpf_link iterator 2022-05-10 11:20:45 -07:00
local_storage.c bpf: Use bpf_map_area_alloc consistently on bpf map creation 2022-08-10 11:50:43 -07:00
lpm_trie.c bpf: Use bpf_map_area_alloc consistently on bpf map creation 2022-08-10 11:50:43 -07:00
Makefile bpf: Introduce any context BPF specific memory allocator. 2022-09-05 15:33:05 +02:00
map_in_map.c bpf: Allow storing unreferenced kptr in map 2022-04-25 17:31:35 -07:00
map_in_map.h bpf: Add map_meta_equal map ops 2020-08-28 15:41:30 +02:00
map_iter.c bpf: Introduce MEM_RDONLY flag 2021-12-18 13:27:41 -08:00
memalloc.c bpf: Remove usage of kmem_cache from bpf_mem_cache. 2022-09-05 15:33:07 +02:00
mmap_unlock_work.h bpf: Introduce helper bpf_find_vma 2021-11-07 11:54:51 -08:00
net_namespace.c net: Add includes masked by netdevice.h including uapi/bpf.h 2021-12-29 20:03:05 -08:00
offload.c bpf: Use bpf_map_area_alloc consistently on bpf map creation 2022-08-10 11:50:43 -07:00
percpu_freelist.c bpf: avoid grabbing spin_locks of all cpus when no free elems 2022-06-11 14:25:35 -07:00
percpu_freelist.h bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI 2020-10-06 00:04:11 +02:00
prog_iter.c bpf: Refactor bpf_iter_reg to have separate seq_info member 2020-07-25 20:16:32 -07:00
queue_stack_maps.c bpf: Remove unneeded memset in queue_stack_map creation 2022-08-10 11:48:22 -07:00
reuseport_array.c net: refactor bpf_sk_reuseport_detach() 2022-08-10 21:48:04 -07:00
ringbuf.c bpf: Use bpf_map_area_alloc consistently on bpf map creation 2022-08-10 11:50:43 -07:00
stackmap.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
syscall.c bpf: Batch call_rcu callbacks instead of SLAB_TYPESAFE_BY_RCU. 2022-09-05 15:33:06 +02:00
sysfs_btf.c bpf: Load and verify kernel module BTFs 2020-11-10 15:25:53 -08:00
task_iter.c bpf: Remove redundant assignment to meta.seq in __task_seq_show() 2022-04-11 21:14:34 +02:00
tnum.c bpf, tnums: Provably sound, faster, and more precise algorithm for tnum_mul 2021-06-01 13:34:15 +02:00
trampoline.c bpf: Use this_cpu_{inc_return|dec} for prog->active 2022-09-01 12:16:18 -07:00
verifier.c bpf: Remove prealloc-only restriction for sleepable bpf programs. 2022-09-05 15:33:06 +02:00