linux/tools/lib/bpf
Alan Maguire f3c51fe02c libbpf: Btf dedup identical struct test needs check for nested structs/arrays
When examining module BTF, it is common to see core kernel structures
such as sk_buff, net_device duplicated in the module.  After adding
debug messaging to BTF it turned out that much of the problem
was down to the identical struct test failing during deduplication;
sometimes the compiler adds identical structs.  However
it turns out sometimes that type ids of identical struct members
can also differ, even when the containing structs are still identical.

To take an example, for struct sk_buff, debug messaging revealed
that the identical struct matching was failing for the anon
struct "headers"; specifically for the first field:

__u8       __pkt_type_offset[0]; /*   128     0 */

Looking at the code in BTF deduplication, we have code that guards
against the possibility of identical struct definitions, down to
type ids, and identical array definitions.  However in this case
we have a struct which is being defined twice but does not have
identical type ids since each duplicate struct has separate type
ids for the above array member.   A similar problem (though not
observed) could occur for struct-in-struct.

The solution is to make the "identical struct" test check members
not just for matching ids, but to also check if they in turn are
identical structs or arrays.

The results of doing this are quite dramatic (for some modules
at least); I see the number of type ids drop from around 10000
to just over 1000 in one module for example.

For testing use latest pahole or apply [1], otherwise dedups
can fail for the reasons described there.

Also fix return type of btf_dedup_identical_arrays() as
suggested by Andrii to match boolean return type used
elsewhere.

Fixes: efdd3eb801 ("libbpf: Accommodate DWARF/compiler bug with duplicated structs")
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/1666622309-22289-1-git-send-email-alan.maguire@oracle.com

[1] https://lore.kernel.org/bpf/1666364523-9648-1-git-send-email-alan.maguire
2022-10-25 16:16:14 -07:00
..
.gitignore libbpf: Make libbpf_version.h non-auto-generated 2021-09-13 15:36:47 -07:00
bpf_core_read.h libbpf: add bpf_core_type_matches() helper macro 2022-07-05 21:15:19 -07:00
bpf_endian.h libbpf: Make bpf_endian co-exist with vmlinux.h 2020-07-01 09:06:12 +02:00
bpf_gen_internal.h Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2021-12-10 15:56:13 -08:00
bpf_helpers.h libbpf: Clean up legacy bpf maps declaration in bpf_helpers 2022-09-16 22:56:09 +02:00
bpf_prog_linfo.c libbpf: Streamline error reporting for high-level APIs 2021-05-25 17:32:35 -07:00
bpf_tracing.h libbpf: Improve BPF_PROG2 macro code quality and description 2022-09-21 17:05:31 -07:00
bpf.c libbpf: Introduce bpf_link_get_fd_by_id_opts() 2022-10-10 16:49:20 -07:00
bpf.h libbpf: Introduce bpf_link_get_fd_by_id_opts() 2022-10-10 16:49:20 -07:00
btf_dump.c libbpf: Fix use-after-free in btf_dump_name_dups 2022-10-13 10:53:03 -07:00
btf.c libbpf: Btf dedup identical struct test needs check for nested structs/arrays 2022-10-25 16:16:14 -07:00
btf.h libbpf: Don't require full struct enum64 in UAPI headers 2022-09-27 20:45:17 +02:00
Build libbpf: move xsk.{c,h} into selftests/bpf 2022-06-28 13:13:32 -07:00
gen_loader.c libbpf: fix an snprintf() overflow check 2022-07-19 10:47:31 -07:00
hashmap.c libbpf: Use IS_ERR_OR_NULL() in hashmap__free() 2022-01-12 17:01:36 -08:00
hashmap.h libbpf, hashmap: Fix undefined behavior in hash_bits 2020-11-02 23:33:51 +01:00
libbpf_common.h libbpf: cleanup LIBBPF_DEPRECATED_SINCE supporting macros for v0.x 2022-06-28 13:13:33 -07:00
libbpf_errno.c libbpf: Streamline error reporting for high-level APIs 2021-05-25 17:32:35 -07:00
libbpf_internal.h libbpf: Making bpf_prog_load() ignore name if kernel doesn't support 2022-08-15 14:58:20 -07:00
libbpf_legacy.h libbpf: Clean up deprecated and legacy aliases 2022-08-17 22:42:56 +02:00
libbpf_probes.c bpf: Add libbpf logic for user-space ring buffer 2022-09-21 16:25:03 -07:00
libbpf_version.h bpf: Add libbpf logic for user-space ring buffer 2022-09-21 16:25:03 -07:00
libbpf.c libbpf: only add BPF_F_MMAPABLE flag for data maps with global vars 2022-10-19 16:40:45 -07:00
libbpf.h libbpf: restore memory layout of bpf_object_open_opts 2022-09-23 16:19:37 -07:00
libbpf.map libbpf: Introduce bpf_link_get_fd_by_id_opts() 2022-10-10 16:49:20 -07:00
libbpf.pc.template libbpf: Add zlib as a dependency in pkg-config template 2019-12-16 14:55:29 -08:00
linker.c libbpf: Add enum64 support for bpf linking 2022-06-07 10:20:43 -07:00
Makefile libbpf: move xsk.{c,h} into selftests/bpf 2022-06-28 13:13:32 -07:00
netlink.c libbpf: Streamline bpf_attr and perf_event_attr initialization 2022-08-17 22:42:10 +02:00
nlattr.c libbpf: Fix overrun in netlink attribute iteration 2022-09-30 15:16:22 -07:00
nlattr.h libbpf: Switch to void * casting in netlink helpers 2021-06-22 17:04:02 +02:00
relo_core.c bpf: Correctly propagate errors up from bpf_core_composites_match 2022-07-08 15:31:43 -07:00
relo_core.h bpf, libbpf: Add type match support 2022-07-05 21:14:25 -07:00
ringbuf.c bpf: Add libbpf logic for user-space ring buffer 2022-09-21 16:25:03 -07:00
skel_internal.h libbpf: add map_get_fd_by_id and map_delete_elem in light skeleton 2022-08-25 18:52:29 -07:00
str_error.c libbpf: Poison kernel-only integer types 2020-01-10 10:38:00 -08:00
str_error.h libbpf: relicense libbpf as LGPL-2.1 OR BSD-2-Clause 2018-10-08 10:09:48 +02:00
strset.c libbpf: Fix memory leak in strset 2021-10-01 22:54:38 +02:00
strset.h libbpf: Extract internal set-of-strings datastructure APIs 2021-03-18 16:14:22 -07:00
usdt.bpf.h libbpf: Ensure functions with always_inline attribute are inline 2022-08-04 14:43:41 -07:00
usdt.c libbpf: Avoid allocating reg_name with sscanf in parse_usdt_arg() 2022-10-21 14:28:14 -07:00