forked from Minki/linux
54a9c3a42d
This patch use head->first in pcpu_freelist_head to check freelist
having free or not. If having, grab spin_lock, or check next cpu's
freelist.
Before patch: hash_map performance
./map_perf_test 1
0:hash_map_perf pre-alloc 1043397 events per sec
...
The average of the test results is around 1050000 events per sec.
hash_map the worst: no free
./run_bench_bpf_hashmap_full_update.sh
Setting up benchmark 'bpf-hashmap-ful-update'...
Benchmark 'bpf-hashmap-ful-update' started.
1:hash_map_full_perf 15687 events per sec
...
The average of the test results is around 16000 events per sec.
ftrace trace:
0) | htab_map_update_elem() {
0) | __pcpu_freelist_pop() {
0) | _raw_spin_lock()
0) | _raw_spin_unlock()
0) | ...
0) + 25.188 us | }
0) + 28.439 us | }
The test machine is 16C, trying to get spin_lock 17 times, in addition
to 16c, there is an extralist.
after patch: hash_map performance
./map_perf_test 1
0:hash_map_perf pre-alloc 1053298 events per sec
...
The average of the test results is around 1050000 events per sec.
hash_map worst: no free
./run_bench_bpf_hashmap_full_update.sh
Setting up benchmark 'bpf-hashmap-ful-update'...
Benchmark 'bpf-hashmap-ful-update' started.
1:hash_map_full_perf 555830 events per sec
...
The average of the test results is around 550000 events per sec.
ftrace trace:
0) | htab_map_update_elem() {
0) | alloc_htab_elem() {
0) 0.586 us | __pcpu_freelist_pop();
0) 0.945 us | }
0) 8.669 us | }
It can be seen that after adding this patch, the map performance is
almost not degraded, and when free=0, first check head->first instead of
directly acquiring spin_lock.
Co-developed-by: Chengming Zhou <zhouchengming@bytedance.com>
Signed-off-by: Chengming Zhou <zhouchengming@bytedance.com>
Signed-off-by: Feng Zhou <zhoufeng.zf@bytedance.com>
Link: https://lore.kernel.org/r/20220610023308.93798-2-zhoufeng.zf@bytedance.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
|
||
|---|---|---|
| .. | ||
| preload | ||
| arraymap.c | ||
| bloom_filter.c | ||
| bpf_inode_storage.c | ||
| bpf_iter.c | ||
| bpf_local_storage.c | ||
| bpf_lru_list.c | ||
| bpf_lru_list.h | ||
| bpf_lsm.c | ||
| bpf_struct_ops_types.h | ||
| bpf_struct_ops.c | ||
| bpf_task_storage.c | ||
| btf.c | ||
| cgroup.c | ||
| core.c | ||
| cpumap.c | ||
| devmap.c | ||
| disasm.c | ||
| disasm.h | ||
| dispatcher.c | ||
| hashtab.c | ||
| helpers.c | ||
| inode.c | ||
| Kconfig | ||
| link_iter.c | ||
| local_storage.c | ||
| lpm_trie.c | ||
| Makefile | ||
| map_in_map.c | ||
| map_in_map.h | ||
| map_iter.c | ||
| mmap_unlock_work.h | ||
| net_namespace.c | ||
| offload.c | ||
| percpu_freelist.c | ||
| percpu_freelist.h | ||
| prog_iter.c | ||
| queue_stack_maps.c | ||
| reuseport_array.c | ||
| ringbuf.c | ||
| stackmap.c | ||
| syscall.c | ||
| sysfs_btf.c | ||
| task_iter.c | ||
| tnum.c | ||
| trampoline.c | ||
| verifier.c | ||