leandrof/linux
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'BPF static linking'

Browse Source 
Andrii Nakryiko says:

====================

This patch set adds new libbpf APIs and their bpftool integration that allows
to perform static linking of BPF object files. Currently no extern resolution
across object files is performed. This is going to be the focus of the follow
up patches. But, given amount of code and logic necessary to perform just
basic functionality of linking together mostly independent BPF object files,
it was decided to land basic BPF linker code and logic first and extend it
afterwards.

The motivation for BPF static linking is to provide the functionality that is
naturally assumed for user-space development process: ability to structure
application's code without artificial restrictions of having all the code and
data (variables and maps) inside a single source code file.

This enables better engineering practices of splitting code into
well-encapsulated parts. It provides ability to hide internal state from other
parts of the code base through static variables and maps. It is also a first
steps towards having generic reusable BPF libraries.

Please see individual patches (mostly #6 and #7) for more details. Patch #10
passes all test_progs' individual BPF .o files through BPF static linker,
which is supposed to be a no-op operation, so is essentially validating that
BPF static linker doesn't produce corrupted ELF object files. Patch #11 adds
Makefile infra to be able to specify multi-file BPF object files and adds the
first multi-file test to validate correctness.

v3->v4:
  - fix Makefile copy/paste error of diff'ing invalid object files (Alexei);
  - fix uninitialized obj_name variable that could lead to bogus object names
    being used during skeleton generation (kernel-patches CI);
v2->v3:
  - added F(F(F(X))) = F(F(X)) test for all linked BPF object files (Alexei);
  - used reallocarray() more consistently in few places (Alexei);
  - improved bash completions for `gen object` (Quentin);
  - dropped .bpfo extension, but had to add optional `name OBJECT_FILE`
    parameter (path #8) to `gen skeleton` command to specify desired object
    name during skeleton generation;
  - fixed bug of merging DATASECS of special "license" and "version" sections.
    Linker currently strictly validates that all versions and licenses matches
    exactly and keeps only ELF symbols and BTF DATASEC from the very first
    object file with license/version. For all other object files, we ignore
    ELF symbols, but weren't ignoring DATASECs, which caused further problems
    of not being able to find a corresponding ELF symbol, if variable name
    differs between two files (which we test deliberately in multi-file
    linking selftest). The fix is to ignore BTF DATASECS;
v1->v2:
  - extracted `struct strset` to manage unique set of strings both for BTF and
    ELF SYMTAB (patch #4, refactors btf and btf_dedup logic as well) (Alexei);
  - fixed bugs in bpftool gen command; renamed it to `gen object`, added BASH
    completions and extended/updated man page (Quentin).
====================

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
This commit is contained in:
Alexei Starovoitov 2021-03-18 16:14:23 -07:00
commit 6d8b271682
18 changed files with 2808 additions and 426 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

78
tools/bpf/bpftool/Documentation/bpftool-gen.rst
View File

@ -14,16 +14,37 @@ SYNOPSIS
*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] }
*COMMAND* := { **skeleton** | **help** }
*COMMAND* := { **object** | **skeleton** | **help** }
GEN COMMANDS
=============
| **bpftool** **gen skeleton** *FILE*
| **bpftool** **gen object** *OUTPUT_FILE* *INPUT_FILE* [*INPUT_FILE*...]
| **bpftool** **gen skeleton** *FILE* [**name** *OBJECT_NAME*]
| **bpftool** **gen help**
DESCRIPTION
===========
**bpftool gen object** *OUTPUT_FILE* *INPUT_FILE* [*INPUT_FILE*...]
Statically link (combine) together one or more *INPUT_FILE*'s
into a single resulting *OUTPUT_FILE*. All the files involved
are BPF ELF object files.
The rules of BPF static linking are mostly the same as for
user-space object files, but in addition to combining data
and instruction sections, .BTF and .BTF.ext (if present in
any of the input files) data are combined together. .BTF
data is deduplicated, so all the common types across
*INPUT_FILE*'s will only be represented once in the resulting
BTF information.
BPF static linking allows to partition BPF source code into
individually compiled files that are then linked into
a single resulting BPF object file, which can be used to
generated BPF skeleton (with **gen skeleton** command) or
passed directly into **libbpf** (using **bpf_object__open()**
family of APIs).
**bpftool gen skeleton** *FILE*
Generate BPF skeleton C header file for a given *FILE*.
@ -75,10 +96,13 @@ DESCRIPTION
specific maps, programs, etc.
As part of skeleton, few custom functions are generated.
Each of them is prefixed with object name, derived from
object file name. I.e., if BPF object file name is
**example.o**, BPF object name will be **example**. The
following custom functions are provided in such case:
Each of them is prefixed with object name. Object name can
either be derived from object file name, i.e., if BPF object
file name is **example.o**, BPF object name will be
**example**. Object name can be also specified explicitly
through **name** *OBJECT_NAME* parameter. The following
custom functions are provided (assuming **example** as
the object name):
- **example__open** and **example__open_opts**.
These functions are used to instantiate skeleton. It
@ -130,26 +154,19 @@ OPTIONS
EXAMPLES
========
**$ cat example.c**
**$ cat example1.bpf.c**
::
#include <stdbool.h>
#include <linux/ptrace.h>
#include <linux/bpf.h>
#include "bpf_helpers.h"
#include <bpf/bpf_helpers.h>
const volatile int param1 = 42;
bool global_flag = true;
struct { int x; } data = {};
struct {
__uint(type, BPF_MAP_TYPE_HASH);
__uint(max_entries, 128);
__type(key, int);
__type(value, long);
} my_map SEC(".maps");
SEC("raw_tp/sys_enter")
int handle_sys_enter(struct pt_regs *ctx)
{
@ -161,6 +178,21 @@ EXAMPLES
return 0;
}
**$ cat example2.bpf.c**
::
#include <linux/ptrace.h>
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
struct {
__uint(type, BPF_MAP_TYPE_HASH);
__uint(max_entries, 128);
__type(key, int);
__type(value, long);
} my_map SEC(".maps");
SEC("raw_tp/sys_exit")
int handle_sys_exit(struct pt_regs *ctx)
{
@ -170,9 +202,17 @@ EXAMPLES
}
This is example BPF application with two BPF programs and a mix of BPF maps
and global variables.
and global variables. Source code is split across two source code files.
**$ bpftool gen skeleton example.o**
**$ clang -target bpf -g example1.bpf.c -o example1.bpf.o**
**$ clang -target bpf -g example2.bpf.c -o example2.bpf.o**
**$ bpftool gen object example.bpf.o example1.bpf.o example2.bpf.o**
This set of commands compiles *example1.bpf.c* and *example2.bpf.c*
individually and then statically links respective object files into the final
BPF ELF object file *example.bpf.o*.
**$ bpftool gen skeleton example.bpf.o name example | tee example.skel.h**
::
@ -227,7 +267,7 @@ and global variables.
#endif /* __EXAMPLE_SKEL_H__ */
**$ cat example_user.c**
**$ cat example.c**
::
@ -270,7 +310,7 @@ and global variables.
return err;
}
**# ./example_user**
**# ./example**
::

17
tools/bpf/bpftool/bash-completion/bpftool
View File

@ -981,12 +981,25 @@ _bpftool()
;;
gen)
case $command in
skeleton)
object)
_filedir
return 0
;;
skeleton)
case $prev in
$command)
_filedir
return 0
;;
*)
_bpftool_once_attr 'name'
return 0
;;
esac
;;
*)
[[ $prev == $object ]] && \
COMPREPLY=( $( compgen -W 'skeleton help' -- "$cur" ) )
COMPREPLY=( $( compgen -W 'object skeleton help' -- "$cur" ) )
;;
esac
;;

72
tools/bpf/bpftool/gen.c
View File

@ -273,7 +273,7 @@ static int do_skeleton(int argc, char **argv)
char header_guard[MAX_OBJ_NAME_LEN + sizeof("__SKEL_H__")];
size_t i, map_cnt = 0, prog_cnt = 0, file_sz, mmap_sz;
DECLARE_LIBBPF_OPTS(bpf_object_open_opts, opts);
char obj_name[MAX_OBJ_NAME_LEN], *obj_data;
char obj_name[MAX_OBJ_NAME_LEN] = "", *obj_data;
struct bpf_object *obj = NULL;
const char *file, *ident;
struct bpf_program *prog;
@ -288,6 +288,28 @@ static int do_skeleton(int argc, char **argv)
}
file = GET_ARG();
while (argc) {
if (!REQ_ARGS(2))
return -1;
if (is_prefix(*argv, "name")) {
NEXT_ARG();
if (obj_name[0] != '\0') {
p_err("object name already specified");
return -1;
}
strncpy(obj_name, *argv, MAX_OBJ_NAME_LEN - 1);
obj_name[MAX_OBJ_NAME_LEN - 1] = '\0';
} else {
p_err("unknown arg %s", *argv);
return -1;
}
NEXT_ARG();
}
if (argc) {
p_err("extra unknown arguments");
return -1;
@ -310,7 +332,8 @@ static int do_skeleton(int argc, char **argv)
p_err("failed to mmap() %s: %s", file, strerror(errno));
goto out;
}
get_obj_name(obj_name, file);
if (obj_name[0] == '\0')
get_obj_name(obj_name, file);
opts.object_name = obj_name;
obj = bpf_object__open_mem(obj_data, file_sz, &opts);
if (IS_ERR(obj)) {
@ -591,6 +614,47 @@ out:
return err;
}
static int do_object(int argc, char **argv)
{
struct bpf_linker *linker;
const char *output_file, *file;
int err = 0;
if (!REQ_ARGS(2)) {
usage();
return -1;
}
output_file = GET_ARG();
linker = bpf_linker__new(output_file, NULL);
if (!linker) {
p_err("failed to create BPF linker instance");
return -1;
}
while (argc) {
file = GET_ARG();
err = bpf_linker__add_file(linker, file);
if (err) {
p_err("failed to link '%s': %s (%d)", file, strerror(err), err);
goto out;
}
}
err = bpf_linker__finalize(linker);
if (err) {
p_err("failed to finalize ELF file: %s (%d)", strerror(err), err);
goto out;
}
err = 0;
out:
bpf_linker__free(linker);
return err;
}
static int do_help(int argc, char **argv)
{
if (json_output) {
@ -599,7 +663,8 @@ static int do_help(int argc, char **argv)
}
fprintf(stderr,
"Usage: %1$s %2$s skeleton FILE\n"
"Usage: %1$s %2$s object OUTPUT_FILE INPUT_FILE [INPUT_FILE...]\n"
" %1$s %2$s skeleton FILE [name OBJECT_NAME]\n"
" %1$s %2$s help\n"
"\n"
" " HELP_SPEC_OPTIONS "\n"
@ -610,6 +675,7 @@ static int do_help(int argc, char **argv)
}
static const struct cmd cmds[] = {
{ "object", do_object },
{ "skeleton", do_skeleton },
{ "help", do_help },
{ 0 }

2
tools/lib/bpf/Build
View File

@ -1,3 +1,3 @@
libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \
netlink.o bpf_prog_linfo.o libbpf_probes.o xsk.o hashmap.o \
btf_dump.o ringbuf.o
btf_dump.o ringbuf.o strset.o linker.o

718
tools/lib/bpf/btf.c
View File

@ -21,6 +21,7 @@
#include "libbpf.h"
#include "libbpf_internal.h"
#include "hashmap.h"
#include "strset.h"
#define BTF_MAX_NR_TYPES 0x7fffffffU
#define BTF_MAX_STR_OFFSET 0x7fffffffU
@ -67,7 +68,7 @@ struct btf {
* | | |
* hdr | |
* types_data----+ |
* strs_data------------------+
* strset__data(strs_set)-----+
*
* +----------+---------+-----------+
* | Header | Types | Strings |
@ -105,20 +106,15 @@ struct btf {
*/
int start_str_off;
/* only one of strs_data or strs_set can be non-NULL, depending on
* whether BTF is in a modifiable state (strs_set is used) or not
* (strs_data points inside raw_data)
*/
void *strs_data;
size_t strs_data_cap; /* used size stored in hdr->str_len */
/* lookup index for each unique string in strings section */
struct hashmap *strs_hash;
/* a set of unique strings */
struct strset *strs_set;
/* whether strings are already deduplicated */
bool strs_deduped;
/* extra indirection layer to make strings hashmap work with stable
* string offsets and ability to transparently choose between
* btf->strs_data or btf_dedup->strs_data as a source of strings.
* This is used for BTF strings dedup to transfer deduplicated strings
* data back to struct btf without re-building strings index.
*/
void **strs_data_ptr;
/* BTF object FD, if loaded into kernel */
int fd;
@ -142,8 +138,8 @@ static inline __u64 ptr_to_u64(const void *ptr)
* On success, memory pointer to the beginning of unused memory is returned.
* On error, NULL is returned.
*/
void *btf_add_mem(void **data, size_t *cap_cnt, size_t elem_sz,
size_t cur_cnt, size_t max_cnt, size_t add_cnt)
void *libbpf_add_mem(void **data, size_t *cap_cnt, size_t elem_sz,
size_t cur_cnt, size_t max_cnt, size_t add_cnt)
{
size_t new_cnt;
void *new_data;
@ -179,14 +175,14 @@ void *btf_add_mem(void **data, size_t *cap_cnt, size_t elem_sz,
/* Ensure given dynamically allocated memory region has enough allocated space
* to accommodate *need_cnt* elements of size *elem_sz* bytes each
*/
int btf_ensure_mem(void **data, size_t *cap_cnt, size_t elem_sz, size_t need_cnt)
int libbpf_ensure_mem(void **data, size_t *cap_cnt, size_t elem_sz, size_t need_cnt)
{
void *p;
if (need_cnt <= *cap_cnt)
return 0;
p = btf_add_mem(data, cap_cnt, elem_sz, *cap_cnt, SIZE_MAX, need_cnt - *cap_cnt);
p = libbpf_add_mem(data, cap_cnt, elem_sz, *cap_cnt, SIZE_MAX, need_cnt - *cap_cnt);
if (!p)
return -ENOMEM;
@ -197,8 +193,8 @@ static int btf_add_type_idx_entry(struct btf *btf, __u32 type_off)
{
__u32 *p;
p = btf_add_mem((void **)&btf->type_offs, &btf->type_offs_cap, sizeof(__u32),
btf->nr_types, BTF_MAX_NR_TYPES, 1);
p = libbpf_add_mem((void **)&btf->type_offs, &btf->type_offs_cap, sizeof(__u32),
btf->nr_types, BTF_MAX_NR_TYPES, 1);
if (!p)
return -ENOMEM;
@ -435,7 +431,7 @@ const struct btf *btf__base_btf(const struct btf *btf)
}
/* internal helper returning non-const pointer to a type */
static struct btf_type *btf_type_by_id(struct btf *btf, __u32 type_id)
struct btf_type *btf_type_by_id(struct btf *btf, __u32 type_id)
{
if (type_id == 0)
return &btf_void;
@ -738,7 +734,7 @@ void btf__free(struct btf *btf)
*/
free(btf->hdr);
free(btf->types_data);
free(btf->strs_data);
strset__free(btf->strs_set);
}
free(btf->raw_data);
free(btf->raw_data_swapped);
@ -1246,6 +1242,11 @@ void btf__set_fd(struct btf *btf, int fd)
btf->fd = fd;
}
static const void *btf_strs_data(const struct btf *btf)
{
return btf->strs_data ? btf->strs_data : strset__data(btf->strs_set);
}
static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endian)
{
struct btf_header *hdr = btf->hdr;
@ -1286,7 +1287,7 @@ static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endi
}
p += hdr->type_len;
memcpy(p, btf->strs_data, hdr->str_len);
memcpy(p, btf_strs_data(btf), hdr->str_len);
p += hdr->str_len;
*size = data_sz;
@ -1320,7 +1321,7 @@ const char *btf__str_by_offset(const struct btf *btf, __u32 offset)
if (offset < btf->start_str_off)
return btf__str_by_offset(btf->base_btf, offset);
else if (offset - btf->start_str_off < btf->hdr->str_len)
return btf->strs_data + (offset - btf->start_str_off);
return btf_strs_data(btf) + (offset - btf->start_str_off);
else
return NULL;
}
@ -1474,25 +1475,6 @@ int btf__get_map_kv_tids(const struct btf *btf, const char *map_name,
return 0;
}
static size_t strs_hash_fn(const void *key, void *ctx)
{
const struct btf *btf = ctx;
const char *strs = *btf->strs_data_ptr;
const char *str = strs + (long)key;
return str_hash(str);
}
static bool strs_hash_equal_fn(const void *key1, const void *key2, void *ctx)
{
const struct btf *btf = ctx;
const char *strs = *btf->strs_data_ptr;
const char *str1 = strs + (long)key1;
const char *str2 = strs + (long)key2;
return strcmp(str1, str2) == 0;
}
static void btf_invalidate_raw_data(struct btf *btf)
{
if (btf->raw_data) {
@ -1511,10 +1493,9 @@ static void btf_invalidate_raw_data(struct btf *btf)
*/
static int btf_ensure_modifiable(struct btf *btf)
{
void *hdr, *types, *strs, *strs_end, *s;
struct hashmap *hash = NULL;
long off;
int err;
void *hdr, *types;
struct strset *set = NULL;
int err = -ENOMEM;
if (btf_is_modifiable(btf)) {
/* any BTF modification invalidates raw_data */
@ -1525,44 +1506,25 @@ static int btf_ensure_modifiable(struct btf *btf)
/* split raw data into three memory regions */
hdr = malloc(btf->hdr->hdr_len);
types = malloc(btf->hdr->type_len);
strs = malloc(btf->hdr->str_len);
if (!hdr || !types || !strs)
if (!hdr || !types)
goto err_out;
memcpy(hdr, btf->hdr, btf->hdr->hdr_len);
memcpy(types, btf->types_data, btf->hdr->type_len);
memcpy(strs, btf->strs_data, btf->hdr->str_len);
/* make hashmap below use btf->strs_data as a source of strings */
btf->strs_data_ptr = &btf->strs_data;
/* build lookup index for all strings */
hash = hashmap__new(strs_hash_fn, strs_hash_equal_fn, btf);
if (IS_ERR(hash)) {
err = PTR_ERR(hash);
hash = NULL;
set = strset__new(BTF_MAX_STR_OFFSET, btf->strs_data, btf->hdr->str_len);
if (IS_ERR(set)) {
err = PTR_ERR(set);
goto err_out;
}
strs_end = strs + btf->hdr->str_len;
for (off = 0, s = strs; s < strs_end; off += strlen(s) + 1, s = strs + off) {
/* hashmap__add() returns EEXIST if string with the same
* content already is in the hash map
*/
err = hashmap__add(hash, (void *)off, (void *)off);
if (err == -EEXIST)
continue; /* duplicate */
if (err)
goto err_out;
}
/* only when everything was successful, update internal state */
btf->hdr = hdr;
btf->types_data = types;
btf->types_data_cap = btf->hdr->type_len;
btf->strs_data = strs;
btf->strs_data_cap = btf->hdr->str_len;
btf->strs_hash = hash;
btf->strs_data = NULL;
btf->strs_set = set;
/* if BTF was created from scratch, all strings are guaranteed to be
* unique and deduplicated
*/
@ -1577,17 +1539,10 @@ static int btf_ensure_modifiable(struct btf *btf)
return 0;
err_out:
hashmap__free(hash);
strset__free(set);
free(hdr);
free(types);
free(strs);
return -ENOMEM;
}
static void *btf_add_str_mem(struct btf *btf, size_t add_sz)
{
return btf_add_mem(&btf->strs_data, &btf->strs_data_cap, 1,
btf->hdr->str_len, BTF_MAX_STR_OFFSET, add_sz);
return err;
}
/* Find an offset in BTF string section that corresponds to a given string *s*.
@ -1598,34 +1553,23 @@ static void *btf_add_str_mem(struct btf *btf, size_t add_sz)
*/
int btf__find_str(struct btf *btf, const char *s)
{
long old_off, new_off, len;
void *p;
int off;
if (btf->base_btf) {
int ret;
ret = btf__find_str(btf->base_btf, s);
if (ret != -ENOENT)
return ret;
off = btf__find_str(btf->base_btf, s);
if (off != -ENOENT)
return off;
}
/* BTF needs to be in a modifiable state to build string lookup index */
if (btf_ensure_modifiable(btf))
return -ENOMEM;
/* see btf__add_str() for why we do this */
len = strlen(s) + 1;
p = btf_add_str_mem(btf, len);
if (!p)
return -ENOMEM;
off = strset__find_str(btf->strs_set, s);
if (off < 0)
return off;
new_off = btf->hdr->str_len;
memcpy(p, s, len);
if (hashmap__find(btf->strs_hash, (void *)new_off, (void **)&old_off))
return btf->start_str_off + old_off;
return -ENOENT;
return btf->start_str_off + off;
}
/* Add a string s to the BTF string section.
@ -1635,56 +1579,30 @@ int btf__find_str(struct btf *btf, const char *s)
*/
int btf__add_str(struct btf *btf, const char *s)
{
long old_off, new_off, len;
void *p;
int err;
int off;
if (btf->base_btf) {
int ret;
ret = btf__find_str(btf->base_btf, s);
if (ret != -ENOENT)
return ret;
off = btf__find_str(btf->base_btf, s);
if (off != -ENOENT)
return off;
}
if (btf_ensure_modifiable(btf))
return -ENOMEM;
/* Hashmap keys are always offsets within btf->strs_data, so to even
* look up some string from the "outside", we need to first append it
* at the end, so that it can be addressed with an offset. Luckily,
* until btf->hdr->str_len is incremented, that string is just a piece
* of garbage for the rest of BTF code, so no harm, no foul. On the
* other hand, if the string is unique, it's already appended and
* ready to be used, only a simple btf->hdr->str_len increment away.
*/
len = strlen(s) + 1;
p = btf_add_str_mem(btf, len);
if (!p)
return -ENOMEM;
off = strset__add_str(btf->strs_set, s);
if (off < 0)
return off;
new_off = btf->hdr->str_len;
memcpy(p, s, len);
btf->hdr->str_len = strset__data_size(btf->strs_set);
/* Now attempt to add the string, but only if the string with the same
* contents doesn't exist already (HASHMAP_ADD strategy). If such
* string exists, we'll get its offset in old_off (that's old_key).
*/
err = hashmap__insert(btf->strs_hash, (void *)new_off, (void *)new_off,
HASHMAP_ADD, (const void **)&old_off, NULL);
if (err == -EEXIST)
return btf->start_str_off + old_off; /* duplicated string, return existing offset */
if (err)
return err;
btf->hdr->str_len += len; /* new unique string, adjust data length */
return btf->start_str_off + new_off;
return btf->start_str_off + off;
}
static void *btf_add_type_mem(struct btf *btf, size_t add_sz)
{
return btf_add_mem(&btf->types_data, &btf->types_data_cap, 1,
btf->hdr->type_len, UINT_MAX, add_sz);
return libbpf_add_mem(&btf->types_data, &btf->types_data_cap, 1,
btf->hdr->type_len, UINT_MAX, add_sz);
}
static __u32 btf_type_info(int kind, int vlen, int kflag)
@ -1711,6 +1629,54 @@ static int btf_commit_type(struct btf *btf, int data_sz)
return btf->start_id + btf->nr_types - 1;
}
struct btf_pipe {
const struct btf *src;
struct btf *dst;
};
static int btf_rewrite_str(__u32 *str_off, void *ctx)
{
struct btf_pipe *p = ctx;
int off;
if (!*str_off) /* nothing to do for empty strings */
return 0;
off = btf__add_str(p->dst, btf__str_by_offset(p->src, *str_off));
if (off < 0)
return off;
*str_off = off;
return 0;
}
int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_type *src_type)
{
struct btf_pipe p = { .src = src_btf, .dst = btf };
struct btf_type *t;
int sz, err;
sz = btf_type_size(src_type);
if (sz < 0)
return sz;
/* deconstruct BTF, if necessary, and invalidate raw_data */
if (btf_ensure_modifiable(btf))
return -ENOMEM;
t = btf_add_type_mem(btf, sz);
if (!t)
return -ENOMEM;
memcpy(t, src_type, sz);
err = btf_type_visit_str_offs(t, btf_rewrite_str, &p);
if (err)
return err;
return btf_commit_type(btf, sz);
}
/*
* Append new BTF_KIND_INT type with:
* - *name* - non-empty, non-NULL type name;
@ -3016,10 +2982,7 @@ struct btf_dedup {
/* Various option modifying behavior of algorithm */
struct btf_dedup_opts opts;
/* temporary strings deduplication state */
void *strs_data;
size_t strs_cap;
size_t strs_len;
struct hashmap* strs_hash;
struct strset *strs_set;
};
static long hash_combine(long h, long value)
@ -3155,95 +3118,28 @@ done:
return d;
}
typedef int (*str_off_fn_t)(__u32 *str_off_ptr, void *ctx);
/*
* Iterate over all possible places in .BTF and .BTF.ext that can reference
* string and pass pointer to it to a provided callback `fn`.
*/
static int btf_for_each_str_off(struct btf_dedup *d, str_off_fn_t fn, void *ctx)
static int btf_for_each_str_off(struct btf_dedup *d, str_off_visit_fn fn, void *ctx)
{
void *line_data_cur, *line_data_end;
int i, j, r, rec_size;
struct btf_type *t;
int i, r;
for (i = 0; i < d->btf->nr_types; i++) {
t = btf_type_by_id(d->btf, d->btf->start_id + i);
r = fn(&t->name_off, ctx);
struct btf_type *t = btf_type_by_id(d->btf, d->btf->start_id + i);
r = btf_type_visit_str_offs(t, fn, ctx);
if (r)
return r;
switch (btf_kind(t)) {
case BTF_KIND_STRUCT:
case BTF_KIND_UNION: {
struct btf_member *m = btf_members(t);
__u16 vlen = btf_vlen(t);
for (j = 0; j < vlen; j++) {
r = fn(&m->name_off, ctx);
if (r)
return r;
m++;
}
break;
}
case BTF_KIND_ENUM: {
struct btf_enum *m = btf_enum(t);
__u16 vlen = btf_vlen(t);
for (j = 0; j < vlen; j++) {
r = fn(&m->name_off, ctx);
if (r)
return r;
m++;
}
break;
}
case BTF_KIND_FUNC_PROTO: {
struct btf_param *m = btf_params(t);
__u16 vlen = btf_vlen(t);
for (j = 0; j < vlen; j++) {
r = fn(&m->name_off, ctx);
if (r)
return r;
m++;
}
break;
}
default:
break;
}
}
if (!d->btf_ext)
return 0;
line_data_cur = d->btf_ext->line_info.info;
line_data_end = d->btf_ext->line_info.info + d->btf_ext->line_info.len;
rec_size = d->btf_ext->line_info.rec_size;
while (line_data_cur < line_data_end) {
struct btf_ext_info_sec *sec = line_data_cur;
struct bpf_line_info_min *line_info;
__u32 num_info = sec->num_info;
r = fn(&sec->sec_name_off, ctx);
if (r)
return r;
line_data_cur += sizeof(struct btf_ext_info_sec);
for (i = 0; i < num_info; i++) {
line_info = line_data_cur;
r = fn(&line_info->file_name_off, ctx);
if (r)
return r;
r = fn(&line_info->line_off, ctx);
if (r)
return r;
line_data_cur += rec_size;
}
}
r = btf_ext_visit_str_offs(d->btf_ext, fn, ctx);
if (r)
return r;
return 0;
}
@ -3252,10 +3148,8 @@ static int strs_dedup_remap_str_off(__u32 *str_off_ptr, void *ctx)
{
struct btf_dedup *d = ctx;
__u32 str_off = *str_off_ptr;
long old_off, new_off, len;
const char *s;
void *p;
int err;
int off, err;
/* don't touch empty string or string in main BTF */
if (str_off == 0 || str_off < d->btf->start_str_off)
@ -3272,29 +3166,11 @@ static int strs_dedup_remap_str_off(__u32 *str_off_ptr, void *ctx)
return err;
}
len = strlen(s) + 1;
off = strset__add_str(d->strs_set, s);
if (off < 0)
return off;
new_off = d->strs_len;
p = btf_add_mem(&d->strs_data, &d->strs_cap, 1, new_off, BTF_MAX_STR_OFFSET, len);
if (!p)
return -ENOMEM;
memcpy(p, s, len);
/* Now attempt to add the string, but only if the string with the same
* contents doesn't exist already (HASHMAP_ADD strategy). If such
* string exists, we'll get its offset in old_off (that's old_key).
*/
err = hashmap__insert(d->strs_hash, (void *)new_off, (void *)new_off,
HASHMAP_ADD, (const void **)&old_off, NULL);
if (err == -EEXIST) {
*str_off_ptr = d->btf->start_str_off + old_off;
} else if (err) {
return err;
} else {
*str_off_ptr = d->btf->start_str_off + new_off;
d->strs_len += len;
}
*str_off_ptr = d->btf->start_str_off + off;
return 0;
}
@ -3311,39 +3187,23 @@ static int strs_dedup_remap_str_off(__u32 *str_off_ptr, void *ctx)
*/
static int btf_dedup_strings(struct btf_dedup *d)
{
char *s;
int err;
if (d->btf->strs_deduped)
return 0;
/* temporarily switch to use btf_dedup's strs_data for strings for hash
* functions; later we'll just transfer hashmap to struct btf as is,
* along the strs_data
*/
d->btf->strs_data_ptr = &d->strs_data;
d->strs_hash = hashmap__new(strs_hash_fn, strs_hash_equal_fn, d->btf);
if (IS_ERR(d->strs_hash)) {
err = PTR_ERR(d->strs_hash);
d->strs_hash = NULL;
d->strs_set = strset__new(BTF_MAX_STR_OFFSET, NULL, 0);
if (IS_ERR(d->strs_set)) {
err = PTR_ERR(d->strs_set);
goto err_out;
}
if (!d->btf->base_btf) {
s = btf_add_mem(&d->strs_data, &d->strs_cap, 1, d->strs_len, BTF_MAX_STR_OFFSET, 1);
if (!s)
return -ENOMEM;
/* initial empty string */
s[0] = 0;
d->strs_len = 1;
/* insert empty string; we won't be looking it up during strings
* dedup, but it's good to have it for generic BTF string lookups
*/
err = hashmap__insert(d->strs_hash, (void *)0, (void *)0,
HASHMAP_ADD, NULL, NULL);
if (err)
err = strset__add_str(d->strs_set, "");
if (err < 0)
goto err_out;
}
@ -3353,28 +3213,16 @@ static int btf_dedup_strings(struct btf_dedup *d)
goto err_out;
/* replace BTF string data and hash with deduped ones */
free(d->btf->strs_data);
hashmap__free(d->btf->strs_hash);
d->btf->strs_data = d->strs_data;
d->btf->strs_data_cap = d->strs_cap;
d->btf->hdr->str_len = d->strs_len;
d->btf->strs_hash = d->strs_hash;
/* now point strs_data_ptr back to btf->strs_data */
d->btf->strs_data_ptr = &d->btf->strs_data;
d->strs_data = d->strs_hash = NULL;
d->strs_len = d->strs_cap = 0;
strset__free(d->btf->strs_set);
d->btf->hdr->str_len = strset__data_size(d->strs_set);
d->btf->strs_set = d->strs_set;
d->strs_set = NULL;
d->btf->strs_deduped = true;
return 0;
err_out:
free(d->strs_data);
hashmap__free(d->strs_hash);
d->strs_data = d->strs_hash = NULL;
d->strs_len = d->strs_cap = 0;
/* restore strings pointer for existing d->btf->strs_hash back */
d->btf->strs_data_ptr = &d->strs_data;
strset__free(d->strs_set);
d->strs_set = NULL;
return err;
}
@ -4498,15 +4346,18 @@ static int btf_dedup_compact_types(struct btf_dedup *d)
* then mapping it to a deduplicated type ID, stored in btf_dedup->hypot_map,
* which is populated during compaction phase.
*/
static int btf_dedup_remap_type_id(struct btf_dedup *d, __u32 type_id)
static int btf_dedup_remap_type_id(__u32 *type_id, void *ctx)
{
struct btf_dedup *d = ctx;
__u32 resolved_type_id, new_type_id;
resolved_type_id = resolve_type_id(d, type_id);
resolved_type_id = resolve_type_id(d, *type_id);
new_type_id = d->hypot_map[resolved_type_id];
if (new_type_id > BTF_MAX_NR_TYPES)
return -EINVAL;
return new_type_id;
*type_id = new_type_id;
return 0;
}
/*
@ -4519,109 +4370,25 @@ static int btf_dedup_remap_type_id(struct btf_dedup *d, __u32 type_id)
* referenced from any BTF type (e.g., struct fields, func proto args, etc) to
* their final deduped type IDs.
*/
static int btf_dedup_remap_type(struct btf_dedup *d, __u32 type_id)
{
struct btf_type *t = btf_type_by_id(d->btf, type_id);
int i, r;
switch (btf_kind(t)) {
case BTF_KIND_INT:
case BTF_KIND_ENUM:
case BTF_KIND_FLOAT:
break;
case BTF_KIND_FWD:
case BTF_KIND_CONST:
case BTF_KIND_VOLATILE:
case BTF_KIND_RESTRICT:
case BTF_KIND_PTR:
case BTF_KIND_TYPEDEF:
case BTF_KIND_FUNC:
case BTF_KIND_VAR:
r = btf_dedup_remap_type_id(d, t->type);
if (r < 0)
return r;
t->type = r;
break;
case BTF_KIND_ARRAY: {
struct btf_array *arr_info = btf_array(t);
r = btf_dedup_remap_type_id(d, arr_info->type);
if (r < 0)
return r;
arr_info->type = r;
r = btf_dedup_remap_type_id(d, arr_info->index_type);
if (r < 0)
return r;
arr_info->index_type = r;
break;
}
case BTF_KIND_STRUCT:
case BTF_KIND_UNION: {
struct btf_member *member = btf_members(t);
__u16 vlen = btf_vlen(t);
for (i = 0; i < vlen; i++) {
r = btf_dedup_remap_type_id(d, member->type);
if (r < 0)
return r;
member->type = r;
member++;
}
break;
}
case BTF_KIND_FUNC_PROTO: {
struct btf_param *param = btf_params(t);
__u16 vlen = btf_vlen(t);
r = btf_dedup_remap_type_id(d, t->type);
if (r < 0)
return r;
t->type = r;
for (i = 0; i < vlen; i++) {
r = btf_dedup_remap_type_id(d, param->type);
if (r < 0)
return r;
param->type = r;
param++;
}
break;
}
case BTF_KIND_DATASEC: {
struct btf_var_secinfo *var = btf_var_secinfos(t);
__u16 vlen = btf_vlen(t);
for (i = 0; i < vlen; i++) {
r = btf_dedup_remap_type_id(d, var->type);
if (r < 0)
return r;
var->type = r;
var++;
}
break;
}
default:
return -EINVAL;
}
return 0;
}
static int btf_dedup_remap_types(struct btf_dedup *d)
{
int i, r;
for (i = 0; i < d->btf->nr_types; i++) {
r = btf_dedup_remap_type(d, d->btf->start_id + i);
if (r < 0)
struct btf_type *t = btf_type_by_id(d->btf, d->btf->start_id + i);
r = btf_type_visit_type_ids(t, btf_dedup_remap_type_id, d);
if (r)
return r;
}
if (!d->btf_ext)
return 0;
r = btf_ext_visit_type_ids(d->btf_ext, btf_dedup_remap_type_id, d);
if (r)
return r;
return 0;
}
@ -4675,3 +4442,200 @@ struct btf *libbpf_find_kernel_btf(void)
pr_warn("failed to find valid kernel BTF\n");
return ERR_PTR(-ESRCH);
}
int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx)
{
int i, n, err;
switch (btf_kind(t)) {
case BTF_KIND_INT:
case BTF_KIND_FLOAT:
case BTF_KIND_ENUM:
return 0;
case BTF_KIND_FWD:
case BTF_KIND_CONST:
case BTF_KIND_VOLATILE:
case BTF_KIND_RESTRICT:
case BTF_KIND_PTR:
case BTF_KIND_TYPEDEF:
case BTF_KIND_FUNC:
case BTF_KIND_VAR:
return visit(&t->type, ctx);
case BTF_KIND_ARRAY: {
struct btf_array *a = btf_array(t);
err = visit(&a->type, ctx);
err = err ?: visit(&a->index_type, ctx);
return err;
}
case BTF_KIND_STRUCT:
case BTF_KIND_UNION: {
struct btf_member *m = btf_members(t);
for (i = 0, n = btf_vlen(t); i < n; i++, m++) {
err = visit(&m->type, ctx);
if (err)
return err;
}
return 0;
}
case BTF_KIND_FUNC_PROTO: {
struct btf_param *m = btf_params(t);
err = visit(&t->type, ctx);
if (err)
return err;
for (i = 0, n = btf_vlen(t); i < n; i++, m++) {
err = visit(&m->type, ctx);
if (err)
return err;
}
return 0;
}
case BTF_KIND_DATASEC: {
struct btf_var_secinfo *m = btf_var_secinfos(t);
for (i = 0, n = btf_vlen(t); i < n; i++, m++) {
err = visit(&m->type, ctx);
if (err)
return err;
}
return 0;
}
default:
return -EINVAL;
}
}
int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx)
{
int i, n, err;
err = visit(&t->name_off, ctx);
if (err)
return err;
switch (btf_kind(t)) {
case BTF_KIND_STRUCT:
case BTF_KIND_UNION: {
struct btf_member *m = btf_members(t);
for (i = 0, n = btf_vlen(t); i < n; i++, m++) {
err = visit(&m->name_off, ctx);
if (err)
return err;
}
break;
}
case BTF_KIND_ENUM: {
struct btf_enum *m = btf_enum(t);
for (i = 0, n = btf_vlen(t); i < n; i++, m++) {
err = visit(&m->name_off, ctx);
if (err)
return err;
}
break;
}
case BTF_KIND_FUNC_PROTO: {
struct btf_param *m = btf_params(t);
for (i = 0, n = btf_vlen(t); i < n; i++, m++) {
err = visit(&m->name_off, ctx);
if (err)
return err;
}
break;
}
default:
break;
}
return 0;
}
int btf_ext_visit_type_ids(struct btf_ext *btf_ext, type_id_visit_fn visit, void *ctx)
{
const struct btf_ext_info *seg;
struct btf_ext_info_sec *sec;
int i, err;
seg = &btf_ext->func_info;
for_each_btf_ext_sec(seg, sec) {
struct bpf_func_info_min *rec;
for_each_btf_ext_rec(seg, sec, i, rec) {
err = visit(&rec->type_id, ctx);
if (err < 0)
return err;
}
}
seg = &btf_ext->core_relo_info;
for_each_btf_ext_sec(seg, sec) {
struct bpf_core_relo *rec;
for_each_btf_ext_rec(seg, sec, i, rec) {
err = visit(&rec->type_id, ctx);
if (err < 0)
return err;
}
}
return 0;
}
int btf_ext_visit_str_offs(struct btf_ext *btf_ext, str_off_visit_fn visit, void *ctx)
{
const struct btf_ext_info *seg;
struct btf_ext_info_sec *sec;
int i, err;
seg = &btf_ext->func_info;
for_each_btf_ext_sec(seg, sec) {
err = visit(&sec->sec_name_off, ctx);
if (err)
return err;
}
seg = &btf_ext->line_info;
for_each_btf_ext_sec(seg, sec) {
struct bpf_line_info_min *rec;
err = visit(&sec->sec_name_off, ctx);
if (err)
return err;
for_each_btf_ext_rec(seg, sec, i, rec) {
err = visit(&rec->file_name_off, ctx);
if (err)
return err;
err = visit(&rec->line_off, ctx);
if (err)
return err;
}
}
seg = &btf_ext->core_relo_info;
for_each_btf_ext_sec(seg, sec) {
struct bpf_core_relo *rec;
err = visit(&sec->sec_name_off, ctx);
if (err)
return err;
for_each_btf_ext_rec(seg, sec, i, rec) {
err = visit(&rec->access_str_off, ctx);
if (err)
return err;
}
}
return 0;
}

2
tools/lib/bpf/btf.h
View File

@ -93,6 +93,8 @@ LIBBPF_API struct btf *libbpf_find_kernel_btf(void);
LIBBPF_API int btf__find_str(struct btf *btf, const char *s);
LIBBPF_API int btf__add_str(struct btf *btf, const char *s);
LIBBPF_API int btf__add_type(struct btf *btf, const struct btf *src_btf,
const struct btf_type *src_type);
LIBBPF_API int btf__add_int(struct btf *btf, const char *name, size_t byte_sz, int encoding);
LIBBPF_API int btf__add_float(struct btf *btf, const char *name, size_t byte_sz);

8
tools/lib/bpf/btf_dump.c
View File

@ -166,11 +166,11 @@ static int btf_dump_resize(struct btf_dump *d)
if (last_id <= d->last_id)
return 0;
if (btf_ensure_mem((void **)&d->type_states, &d->type_states_cap,
sizeof(*d->type_states), last_id + 1))
if (libbpf_ensure_mem((void **)&d->type_states, &d->type_states_cap,
sizeof(*d->type_states), last_id + 1))
return -ENOMEM;
if (btf_ensure_mem((void **)&d->cached_names, &d->cached_names_cap,
sizeof(*d->cached_names), last_id + 1))
if (libbpf_ensure_mem((void **)&d->cached_names, &d->cached_names_cap,
sizeof(*d->cached_names), last_id + 1))
return -ENOMEM;
if (d->last_id == 0) {

15
tools/lib/bpf/libbpf.c
View File

@ -55,10 +55,6 @@
#include "libbpf_internal.h"
#include "hashmap.h"
#ifndef EM_BPF
#define EM_BPF 247
#endif
#ifndef BPF_FS_MAGIC
#define BPF_FS_MAGIC 0xcafe4a11
#endif
@ -1134,11 +1130,6 @@ static void bpf_object__elf_finish(struct bpf_object *obj)
obj->efile.obj_buf_sz = 0;
}
/* if libelf is old and doesn't support mmap(), fall back to read() */
#ifndef ELF_C_READ_MMAP
#define ELF_C_READ_MMAP ELF_C_READ
#endif
static int bpf_object__elf_init(struct bpf_object *obj)
{
int err = 0;
@ -2807,7 +2798,7 @@ static bool ignore_elf_section(GElf_Shdr *hdr, const char *name)
return true;
/* ignore .llvm_addrsig section as well */
if (hdr->sh_type == 0x6FFF4C03 /* SHT_LLVM_ADDRSIG */)
if (hdr->sh_type == SHT_LLVM_ADDRSIG)
return true;
/* no subprograms will lead to an empty .text section, ignore it */
@ -4867,8 +4858,8 @@ static int load_module_btfs(struct bpf_object *obj)
goto err_out;
}
err = btf_ensure_mem((void **)&obj->btf_modules, &obj->btf_module_cap,
sizeof(*obj->btf_modules), obj->btf_module_cnt + 1);
err = libbpf_ensure_mem((void **)&obj->btf_modules, &obj->btf_module_cap,
sizeof(*obj->btf_modules), obj->btf_module_cnt + 1);
if (err)
goto err_out;

13
tools/lib/bpf/libbpf.h
View File

@ -760,6 +760,19 @@ enum libbpf_tristate {
TRI_MODULE = 2,
};
struct bpf_linker_opts {
/* size of this struct, for forward/backward compatiblity */
size_t sz;
};
#define bpf_linker_opts__last_field sz
struct bpf_linker;
LIBBPF_API struct bpf_linker *bpf_linker__new(const char *filename, struct bpf_linker_opts *opts);
LIBBPF_API int bpf_linker__add_file(struct bpf_linker *linker, const char *filename);
LIBBPF_API int bpf_linker__finalize(struct bpf_linker *linker);
LIBBPF_API void bpf_linker__free(struct bpf_linker *linker);
#ifdef __cplusplus
} /* extern "C" */
#endif

5
tools/lib/bpf/libbpf.map
View File

@ -354,4 +354,9 @@ LIBBPF_0.3.0 {
LIBBPF_0.4.0 {
global:
btf__add_float;
btf__add_type;
bpf_linker__add_file;
bpf_linker__finalize;
bpf_linker__free;
bpf_linker__new;
} LIBBPF_0.3.0;

38
tools/lib/bpf/libbpf_internal.h
View File

@ -20,6 +20,26 @@
#include "libbpf.h"
#ifndef EM_BPF
#define EM_BPF 247
#endif
#ifndef R_BPF_64_64
#define R_BPF_64_64 1
#endif
#ifndef R_BPF_64_32
#define R_BPF_64_32 10
#endif
#ifndef SHT_LLVM_ADDRSIG
#define SHT_LLVM_ADDRSIG 0x6FFF4C03
#endif
/* if libelf is old and doesn't support mmap(), fall back to read() */
#ifndef ELF_C_READ_MMAP
#define ELF_C_READ_MMAP ELF_C_READ
#endif
#define BTF_INFO_ENC(kind, kind_flag, vlen) \
((!!(kind_flag) << 31) | ((kind) << 24) | ((vlen) & BTF_MAX_VLEN))
#define BTF_TYPE_ENC(name, info, size_or_type) (name), (info), (size_or_type)
@ -107,9 +127,14 @@ static inline void *libbpf_reallocarray(void *ptr, size_t nmemb, size_t size)
return realloc(ptr, total);
}
void *btf_add_mem(void **data, size_t *cap_cnt, size_t elem_sz,
size_t cur_cnt, size_t max_cnt, size_t add_cnt);
int btf_ensure_mem(void **data, size_t *cap_cnt, size_t elem_sz, size_t need_cnt);
struct btf;
struct btf_type;
struct btf_type *btf_type_by_id(struct btf *btf, __u32 type_id);
void *libbpf_add_mem(void **data, size_t *cap_cnt, size_t elem_sz,
size_t cur_cnt, size_t max_cnt, size_t add_cnt);
int libbpf_ensure_mem(void **data, size_t *cap_cnt, size_t elem_sz, size_t need_cnt);
static inline bool libbpf_validate_opts(const char *opts,
size_t opts_sz, size_t user_sz,
@ -351,4 +376,11 @@ struct bpf_core_relo {
enum bpf_core_relo_kind kind;
};
typedef int (*type_id_visit_fn)(__u32 *type_id, void *ctx);
typedef int (*str_off_visit_fn)(__u32 *str_off, void *ctx);
int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx);
int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx);
int btf_ext_visit_type_ids(struct btf_ext *btf_ext, type_id_visit_fn visit, void *ctx);
int btf_ext_visit_str_offs(struct btf_ext *btf_ext, str_off_visit_fn visit, void *ctx);
#endif /* __LIBBPF_LIBBPF_INTERNAL_H */

1941
tools/lib/bpf/linker.c Normal file
View File

File diff suppressed because it is too large Load Diff

176
tools/lib/bpf/strset.c Normal file
View File

@ -0,0 +1,176 @@
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
/* Copyright (c) 2021 Facebook */
#include <stdint.h>
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#include <linux/err.h>
#include "hashmap.h"
#include "libbpf_internal.h"
#include "strset.h"
struct strset {
void *strs_data;
size_t strs_data_len;
size_t strs_data_cap;
size_t strs_data_max_len;
/* lookup index for each unique string in strings set */
struct hashmap *strs_hash;
};
static size_t strset_hash_fn(const void *key, void *ctx)
{
const struct strset *s = ctx;
const char *str = s->strs_data + (long)key;
return str_hash(str);
}
static bool strset_equal_fn(const void *key1, const void *key2, void *ctx)
{
const struct strset *s = ctx;
const char *str1 = s->strs_data + (long)key1;
const char *str2 = s->strs_data + (long)key2;
return strcmp(str1, str2) == 0;
}
struct strset *strset__new(size_t max_data_sz, const char *init_data, size_t init_data_sz)
{
struct strset *set = calloc(1, sizeof(*set));
struct hashmap *hash;
int err = -ENOMEM;
if (!set)
return ERR_PTR(-ENOMEM);
hash = hashmap__new(strset_hash_fn, strset_equal_fn, set);
if (IS_ERR(hash))
goto err_out;
set->strs_data_max_len = max_data_sz;
set->strs_hash = hash;
if (init_data) {
long off;
set->strs_data = malloc(init_data_sz);
if (!set->strs_data)
goto err_out;
memcpy(set->strs_data, init_data, init_data_sz);
set->strs_data_len = init_data_sz;
set->strs_data_cap = init_data_sz;
for (off = 0; off < set->strs_data_len; off += strlen(set->strs_data + off) + 1) {
/* hashmap__add() returns EEXIST if string with the same
* content already is in the hash map
*/
err = hashmap__add(hash, (void *)off, (void *)off);
if (err == -EEXIST)
continue; /* duplicate */
if (err)
goto err_out;
}
}
return set;
err_out:
strset__free(set);
return ERR_PTR(err);
}
void strset__free(struct strset *set)
{
if (IS_ERR_OR_NULL(set))
return;
hashmap__free(set->strs_hash);
free(set->strs_data);
}
size_t strset__data_size(const struct strset *set)
{
return set->strs_data_len;
}
const char *strset__data(const struct strset *set)
{
return set->strs_data;
}
static void *strset_add_str_mem(struct strset *set, size_t add_sz)
{
return libbpf_add_mem(&set->strs_data, &set->strs_data_cap, 1,
set->strs_data_len, set->strs_data_max_len, add_sz);
}
/* Find string offset that corresponds to a given string *s*.
* Returns:
* - >0 offset into string data, if string is found;
* - -ENOENT, if string is not in the string data;
* - <0, on any other error.
*/
int strset__find_str(struct strset *set, const char *s)
{
long old_off, new_off, len;
void *p;
/* see strset__add_str() for why we do this */
len = strlen(s) + 1;
p = strset_add_str_mem(set, len);
if (!p)
return -ENOMEM;
new_off = set->strs_data_len;
memcpy(p, s, len);
if (hashmap__find(set->strs_hash, (void *)new_off, (void **)&old_off))
return old_off;
return -ENOENT;
}
/* Add a string s to the string data. If the string already exists, return its
* offset within string data.
* Returns:
* - > 0 offset into string data, on success;
* - < 0, on error.
*/
int strset__add_str(struct strset *set, const char *s)
{
long old_off, new_off, len;
void *p;
int err;
/* Hashmap keys are always offsets within set->strs_data, so to even
* look up some string from the "outside", we need to first append it
* at the end, so that it can be addressed with an offset. Luckily,
* until set->strs_data_len is incremented, that string is just a piece
* of garbage for the rest of the code, so no harm, no foul. On the
* other hand, if the string is unique, it's already appended and
* ready to be used, only a simple set->strs_data_len increment away.
*/
len = strlen(s) + 1;
p = strset_add_str_mem(set, len);
if (!p)
return -ENOMEM;
new_off = set->strs_data_len;
memcpy(p, s, len);
/* Now attempt to add the string, but only if the string with the same
* contents doesn't exist already (HASHMAP_ADD strategy). If such
* string exists, we'll get its offset in old_off (that's old_key).
*/
err = hashmap__insert(set->strs_hash, (void *)new_off, (void *)new_off,
HASHMAP_ADD, (const void **)&old_off, NULL);
if (err == -EEXIST)
return old_off; /* duplicated string, return existing offset */
if (err)
return err;
set->strs_data_len += len; /* new unique string, adjust data length */
return new_off;
}

21
tools/lib/bpf/strset.h Normal file
View File

@ -0,0 +1,21 @@
/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
/* Copyright (c) 2021 Facebook */
#ifndef __LIBBPF_STRSET_H
#define __LIBBPF_STRSET_H
#include <stdbool.h>
#include <stddef.h>
struct strset;
struct strset *strset__new(size_t max_data_sz, const char *init_data, size_t init_data_sz);
void strset__free(struct strset *set);
const char *strset__data(const struct strset *set);
size_t strset__data_size(const struct strset *set);
int strset__find_str(struct strset *set, const char *s);
int strset__add_str(struct strset *set, const char *s);
#endif /* __LIBBPF_STRSET_H */

27
tools/testing/selftests/bpf/Makefile
View File

@ -232,7 +232,7 @@ $(HOST_BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \
DESTDIR=$(HOST_SCRATCH_DIR)/ prefix= all install_headers
endif
$(INCLUDE_DIR)/vmlinux.h: $(VMLINUX_BTF) | $(BPFTOOL) $(INCLUDE_DIR)
$(INCLUDE_DIR)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL) | $(INCLUDE_DIR)
ifeq ($(VMLINUX_H),)
$(call msg,GEN,,$@)
$(Q)$(BPFTOOL) btf dump file $(VMLINUX_BTF) format c > $@
@ -303,6 +303,10 @@ endef
SKEL_BLACKLIST := btf__% test_pinning_invalid.c test_sk_assign.c
LINKED_SKELS := test_static_linked.skel.h
test_static_linked.skel.h-deps := test_static_linked1.o test_static_linked2.o
# Set up extra TRUNNER_XXX "temporary" variables in the environment (relies on
# $eval()) and pass control to DEFINE_TEST_RUNNER_RULES.
# Parameters:
@ -323,6 +327,7 @@ TRUNNER_BPF_OBJS := $$(patsubst %.c,$$(TRUNNER_OUTPUT)/%.o, $$(TRUNNER_BPF_SRCS)
TRUNNER_BPF_SKELS := $$(patsubst %.c,$$(TRUNNER_OUTPUT)/%.skel.h, \
$$(filter-out $(SKEL_BLACKLIST), \
$$(TRUNNER_BPF_SRCS)))
TRUNNER_BPF_SKELS_LINKED := $$(addprefix $$(TRUNNER_OUTPUT)/,$(LINKED_SKELS))
TEST_GEN_FILES += $$(TRUNNER_BPF_OBJS)
# Evaluate rules now with extra TRUNNER_XXX variables above already defined
@ -355,11 +360,22 @@ $(TRUNNER_BPF_OBJS): $(TRUNNER_OUTPUT)/%.o: \
$$(call $(TRUNNER_BPF_BUILD_RULE),$$<,$$@, \
$(TRUNNER_BPF_CFLAGS))
$(TRUNNER_BPF_SKELS): $(TRUNNER_OUTPUT)/%.skel.h: \
$(TRUNNER_OUTPUT)/%.o \
| $(BPFTOOL) $(TRUNNER_OUTPUT)
$(TRUNNER_BPF_SKELS): %.skel.h: %.o $(BPFTOOL) | $(TRUNNER_OUTPUT)
$$(call msg,GEN-SKEL,$(TRUNNER_BINARY),$$@)
$(Q)$$(BPFTOOL) gen skeleton $$< > $$@
$(Q)$$(BPFTOOL) gen object $$(<:.o=.linked1.o) $$<
$(Q)$$(BPFTOOL) gen object $$(<:.o=.linked2.o) $$(<:.o=.linked1.o)
$(Q)$$(BPFTOOL) gen object $$(<:.o=.linked3.o) $$(<:.o=.linked2.o)
$(Q)diff $$(<:.o=.linked2.o) $$(<:.o=.linked3.o)
$(Q)$$(BPFTOOL) gen skeleton $$(<:.o=.linked3.o) name $$(notdir $$(<:.o=)) > $$@
$(TRUNNER_BPF_SKELS_LINKED): $(TRUNNER_BPF_OBJS) $(BPFTOOL) | $(TRUNNER_OUTPUT)
$$(call msg,LINK-BPF,$(TRUNNER_BINARY),$$(@:.skel.h=.o))
$(Q)$$(BPFTOOL) gen object $$(@:.skel.h=.linked1.o) $$(addprefix $(TRUNNER_OUTPUT)/,$$($$(@F)-deps))
$(Q)$$(BPFTOOL) gen object $$(@:.skel.h=.linked2.o) $$(@:.skel.h=.linked1.o)
$(Q)$$(BPFTOOL) gen object $$(@:.skel.h=.linked3.o) $$(@:.skel.h=.linked2.o)
$(Q)diff $$(@:.skel.h=.linked2.o) $$(@:.skel.h=.linked3.o)
$$(call msg,GEN-SKEL,$(TRUNNER_BINARY),$$@)
$(Q)$$(BPFTOOL) gen skeleton $$(@:.skel.h=.linked3.o) name $$(notdir $$(@:.skel.h=)) > $$@
endif
# ensure we set up tests.h header generation rule just once
@ -381,6 +397,7 @@ $(TRUNNER_TEST_OBJS): $(TRUNNER_OUTPUT)/%.test.o: \
$(TRUNNER_EXTRA_HDRS) \
$(TRUNNER_BPF_OBJS) \
$(TRUNNER_BPF_SKELS) \
$(TRUNNER_BPF_SKELS_LINKED) \
$$(BPFOBJ) | $(TRUNNER_OUTPUT)
$$(call msg,TEST-OBJ,$(TRUNNER_BINARY),$$@)
$(Q)cd $$(@D) && $$(CC) -I. $$(CFLAGS) -c $(CURDIR)/$$< $$(LDLIBS) -o $$(@F)

40
tools/testing/selftests/bpf/prog_tests/static_linked.c Normal file
View File

@ -0,0 +1,40 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2019 Facebook */
#include <test_progs.h>
#include "test_static_linked.skel.h"
void test_static_linked(void)
{
int err;
struct test_static_linked* skel;
skel = test_static_linked__open();
if (!ASSERT_OK_PTR(skel, "skel_open"))
return;
skel->rodata->rovar1 = 1;
skel->bss->static_var1 = 2;
skel->bss->static_var11 = 3;
skel->rodata->rovar2 = 4;
skel->bss->static_var2 = 5;
skel->bss->static_var22 = 6;
err = test_static_linked__load(skel);
if (!ASSERT_OK(err, "skel_load"))
goto cleanup;
err = test_static_linked__attach(skel);
if (!ASSERT_OK(err, "skel_attach"))
goto cleanup;
/* trigger */
usleep(1);
ASSERT_EQ(skel->bss->var1, 1 * 2 + 2 + 3, "var1");
ASSERT_EQ(skel->bss->var2, 4 * 3 + 5 + 6, "var2");
cleanup:
test_static_linked__destroy(skel);
}

30
tools/testing/selftests/bpf/progs/test_static_linked1.c Normal file
View File

@ -0,0 +1,30 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2021 Facebook */
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
/* 8-byte aligned .bss */
static volatile long static_var1;
static volatile int static_var11;
int var1 = 0;
/* 4-byte aligned .rodata */
const volatile int rovar1;
/* same "subprog" name in both files */
static __noinline int subprog(int x)
{
/* but different formula */
return x * 2;
}
SEC("raw_tp/sys_enter")
int handler1(const void *ctx)
{
var1 = subprog(rovar1) + static_var1 + static_var11;
return 0;
}
char LICENSE[] SEC("license") = "GPL";
int VERSION SEC("version") = 1;

31
tools/testing/selftests/bpf/progs/test_static_linked2.c Normal file
View File

@ -0,0 +1,31 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2021 Facebook */
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
/* 4-byte aligned .bss */
static volatile int static_var2;
static volatile int static_var22;
int var2 = 0;
/* 8-byte aligned .rodata */
const volatile long rovar2;
/* same "subprog" name in both files */
static __noinline int subprog(int x)
{
/* but different formula */
return x * 3;
}
SEC("raw_tp/sys_enter")
int handler2(const void *ctx)
{
var2 = subprog(rovar2) + static_var2 + static_var22;
return 0;
}
/* different name and/or type of the variable doesn't matter */
char _license[] SEC("license") = "GPL";
int _version SEC("version") = 1;