linux/kernel/bpf
Roman Gushchin b936ca643a bpf: rework memlock-based memory accounting for maps
In order to unify the existing memlock charging code with the
memcg-based memory accounting, which will be added later, let's
rework the current scheme.

Currently the following design is used:
  1) .alloc() callback optionally checks if the allocation will likely
     succeed using bpf_map_precharge_memlock()
  2) .alloc() performs actual allocations
  3) .alloc() callback calculates map cost and sets map.memory.pages
  4) map_create() calls bpf_map_init_memlock() which sets map.memory.user
     and performs actual charging; in case of failure the map is
     destroyed
  <map is in use>
  1) bpf_map_free_deferred() calls bpf_map_release_memlock(), which
     performs uncharge and releases the user
  2) .map_free() callback releases the memory

The scheme can be simplified and made more robust:
  1) .alloc() calculates map cost and calls bpf_map_charge_init()
  2) bpf_map_charge_init() sets map.memory.user and performs actual
    charge
  3) .alloc() performs actual allocations
  <map is in use>
  1) .map_free() callback releases the memory
  2) bpf_map_charge_finish() performs uncharge and releases the user

The new scheme also allows to reuse bpf_map_charge_init()/finish()
functions for memcg-based accounting. Because charges are performed
before actual allocations and uncharges after freeing the memory,
no bogus memory pressure can be created.

In cases when the map structure is not available (e.g. it's not
created yet, or is already destroyed), on-stack bpf_map_memory
structure is used. The charge can be transferred with the
bpf_map_charge_move() function.

Signed-off-by: Roman Gushchin <guro@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2019-05-31 16:52:56 -07:00
..
arraymap.c bpf: rework memlock-based memory accounting for maps 2019-05-31 16:52:56 -07:00
bpf_lru_list.c bpf: lru: Lower the PERCPU_NR_SCANS from 16 to 4 2017-04-17 13:55:52 -04:00
bpf_lru_list.h bpf: Only set node->ref = 1 if it has not been set 2017-09-01 09:57:39 -07:00
btf.c bpf: allow for key-less BTF in array map 2019-04-09 17:05:46 -07:00
cgroup.c bpf: Update __cgroup_bpf_run_filter_skb with cn 2019-05-31 16:41:29 -07:00
core.c bpf: remove __rcu annotations from bpf_prog_array 2019-05-29 15:17:35 +02:00
cpumap.c bpf: rework memlock-based memory accounting for maps 2019-05-31 16:52:56 -07:00
devmap.c bpf: rework memlock-based memory accounting for maps 2019-05-31 16:52:56 -07:00
disasm.c bpf: implement lookup-free direct value access for maps 2019-04-09 17:05:46 -07:00
disasm.h bpf: Remove struct bpf_verifier_env argument from print_bpf_insn 2018-03-23 17:38:57 +01:00
hashtab.c bpf: rework memlock-based memory accounting for maps 2019-05-31 16:52:56 -07:00
helpers.c bpf: Introduce bpf_strtol and bpf_strtoul helpers 2019-04-12 13:54:59 -07:00
inode.c bpf: relax inode permission check for retrieving bpf program 2019-05-16 11:31:49 -07:00
local_storage.c bpf: rework memlock-based memory accounting for maps 2019-05-31 16:52:56 -07:00
lpm_trie.c bpf: rework memlock-based memory accounting for maps 2019-05-31 16:52:56 -07:00
Makefile bpf: add queue and stack maps 2018-10-19 13:24:31 -07:00
map_in_map.c bpf: set inner_map_meta->spin_lock_off correctly 2019-02-27 17:03:13 -08:00
map_in_map.h bpf: Add syscall lookup support for fd array and htab 2017-06-29 13:13:25 -04:00
offload.c bpf: offload: add priv field for drivers 2019-02-12 17:07:09 +01:00
percpu_freelist.c bpf: fix lockdep false positive in percpu_freelist 2019-01-31 23:18:21 +01:00
percpu_freelist.h bpf: fix lockdep false positive in percpu_freelist 2019-01-31 23:18:21 +01:00
queue_stack_maps.c bpf: rework memlock-based memory accounting for maps 2019-05-31 16:52:56 -07:00
reuseport_array.c bpf: rework memlock-based memory accounting for maps 2019-05-31 16:52:56 -07:00
stackmap.c bpf: rework memlock-based memory accounting for maps 2019-05-31 16:52:56 -07:00
syscall.c bpf: rework memlock-based memory accounting for maps 2019-05-31 16:52:56 -07:00
tnum.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
verifier.c bpf: cgroup inet skb programs can return 0 to 3 2019-05-31 16:41:29 -07:00
xskmap.c bpf: rework memlock-based memory accounting for maps 2019-05-31 16:52:56 -07:00