linux/include/crypto
Suren Baghdasaryan 2c321f3f70 mm: change inlined allocation helpers to account at the call site
Main goal of memory allocation profiling patchset is to provide accounting
that is cheap enough to run in production.  To achieve that we inject
counters using codetags at the allocation call sites to account every time
allocation is made.  This injection allows us to perform accounting
efficiently because injected counters are immediately available as opposed
to the alternative methods, such as using _RET_IP_, which would require
counter lookup and appropriate locking that makes accounting much more
expensive.  This method requires all allocation functions to inject
separate counters at their call sites so that their callers can be
individually accounted.  Counter injection is implemented by allocation
hooks which should wrap all allocation functions.

Inlined functions which perform allocations but do not use allocation
hooks are directly charged for the allocations they perform.  In most
cases these functions are just specialized allocation wrappers used from
multiple places to allocate objects of a specific type.  It would be more
useful to do the accounting at their call sites instead.  Instrument these
helpers to do accounting at the call site.  Simple inlined allocation
wrappers are converted directly into macros.  More complex allocators or
allocators with documentation are converted into _noprof versions and
allocation hooks are added.  This allows memory allocation profiling
mechanism to charge allocations to the callers of these functions.

Link: https://lkml.kernel.org/r/20240415020731.1152108-1-surenb@google.com
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Acked-by: Jan Kara <jack@suse.cz>		[jbd2]
Cc: Anna Schumaker <anna@kernel.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Benjamin Tissoires <benjamin.tissoires@redhat.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: David Rientjes <rientjes@google.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: Dennis Zhou <dennis@kernel.org>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Jakub Kicinski <kuba@kernel.org>
Cc: Jakub Sitnicki <jakub@cloudflare.com>
Cc: Jiri Kosina <jikos@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Paolo Abeni <pabeni@redhat.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Trond Myklebust <trond.myklebust@hammerspace.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2024-04-25 20:55:59 -07:00
..
internal mm: change inlined allocation helpers to account at the call site 2024-04-25 20:55:59 -07:00
acompress.h crypto: introduce: acomp_is_async to expose if comp drivers might sleep 2024-03-13 12:12:21 -07:00
aead.h Revert "crypto: remove CONFIG_CRYPTO_STATS" 2024-03-13 09:49:37 +08:00
aes.h crypto: aes - helper function to validate key length for AES algorithms 2019-08-09 15:11:43 +10:00
akcipher.h Revert "crypto: remove CONFIG_CRYPTO_STATS" 2024-03-13 09:49:37 +08:00
algapi.h Revert "crypto: remove CONFIG_CRYPTO_STATS" 2024-03-13 09:49:37 +08:00
arc4.h crypto: arc4 - refactor arc4 core code into separate library 2019-06-20 14:18:33 +08:00
aria.h crypto: Prepare to move crypto_tfm_ctx 2022-12-02 18:12:40 +08:00
authenc.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
b128ops.h cyrpto/b128ops: Remove struct u128 2023-06-05 09:36:35 +02:00
blake2b.h crypto: api - Replace kernel.h with the necessary inclusions 2021-12-17 16:59:48 +11:00
blake2s.h lib/crypto: blake2s: move hmac construction into wireguard 2022-01-18 13:03:55 +01:00
blowfish.h
cast5.h
cast6.h crypto: remove CRYPTO_TFM_RES_BAD_KEY_LEN 2020-01-09 11:30:53 +08:00
cast_common.h
chacha20poly1305.h crypto: lib/chacha20poly1305 - Add missing function declaration 2020-07-16 21:49:04 +10:00
chacha.h random: early initialization of ChaCha constants 2022-01-07 00:25:25 +01:00
cryptd.h crypto: api - Replace kernel.h with the necessary inclusions 2021-12-17 16:59:48 +11:00
ctr.h crypto: ctr - add helper for performing a CTR encryption walk 2019-07-26 14:56:07 +10:00
curve25519.h crypto: lib/curve25519 - Move selftest prototype into header file 2020-11-20 14:45:33 +11:00
des.h crypto: des - remove now unused __des3_ede_setkey() 2019-08-22 14:57:33 +10:00
dh.h crypto: dh - split out deserialization code from crypto_dh_decode() 2022-03-03 10:47:50 +12:00
drbg.h crypto: drbg - reseed 'nopr' drbgs periodically from get_random_bytes() 2021-11-26 16:16:50 +11:00
ecc_curve.h crypto: ecc - add curve25519 params and expose them 2021-03-13 00:04:04 +11:00
ecdh.h Merge branch 'ecc' 2021-03-26 19:55:55 +11:00
engine.h crypto: engine - Make crypto_engine_exit() return void 2023-10-01 16:28:15 +08:00
gcm.h crypto: lib/aesgcm - Provide minimal library implementation 2022-11-11 18:14:59 +08:00
gf128mul.h mm, treewide: rename kzfree() to kfree_sensitive() 2020-08-07 11:33:22 -07:00
ghash.h crypto: ghash - add comment and improve help text 2019-07-27 21:08:38 +10:00
hash_info.h crypto: FIPS 202 SHA-3 register in hash info for IMA 2023-10-27 18:04:30 +08:00
hash.h mm: change inlined allocation helpers to account at the call site 2024-04-25 20:55:59 -07:00
hmac.h
if_alg.h crypto: algif_skcipher - Fix stream cipher chaining 2023-12-08 11:59:46 +08:00
kdf_sp800108.h crypto: kdf - add SP800-108 counter key derivation function 2021-11-26 16:25:17 +11:00
kpp.h Revert "crypto: remove CONFIG_CRYPTO_STATS" 2024-03-13 09:49:37 +08:00
md5.h
nhpoly1305.h crypto: poly1305 - add new 32 and 64-bit generic versions 2020-01-16 15:18:12 +08:00
null.h
padlock.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
pcrypt.h crypto: api - Replace kernel.h with the necessary inclusions 2021-12-17 16:59:48 +11:00
pkcs7.h PKCS#7: Introduce pkcs7_get_digest() 2019-08-05 18:40:19 -04:00
poly1305.h crypto: poly1305 - fix poly1305_core_setkey() declaration 2021-04-02 18:28:12 +11:00
polyval.h crypto: x86/polyval - Add PCLMULQDQ accelerated implementation of POLYVAL 2022-06-10 16:40:17 +08:00
public_key.h KEYS: include header for EINVAL definition 2024-01-26 16:39:32 +08:00
rng.h Revert "crypto: remove CONFIG_CRYPTO_STATS" 2024-03-13 09:49:37 +08:00
scatterwalk.h crypto: scatterwalk - use kmap_local() not kmap_atomic() 2022-12-30 22:56:27 +08:00
serpent.h crypto: x86 - Regularize glue function prototypes 2019-12-11 16:36:54 +08:00
sha1_base.h crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
sha1.h crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
sha2.h crypto: lib/sha256 - Remove redundant and unused sha224_update 2023-05-19 16:45:43 +08:00
sha3.h
sha256_base.h crypto: lib/sha256 - Use generic code from sha256_base 2023-05-19 16:45:43 +08:00
sha512_base.h crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
sig.h crypto: sig - fix kernel-doc typo 2023-10-05 18:16:31 +08:00
skcipher.h mm: change inlined allocation helpers to account at the call site 2024-04-25 20:55:59 -07:00
sm2.h crypto: sm2 - Provide sm2_compute_z_digest when sm2 is disabled 2023-06-26 16:14:01 +08:00
sm3_base.h crypto: hash - Use memzero_explicit() for clearing state 2020-10-30 17:35:03 +11:00
sm3.h crypto: sm3 - make dependent on sm3 library 2022-01-28 16:51:11 +11:00
sm4.h crypto: sm4 - export sm4 constant arrays 2022-04-08 16:12:46 +08:00
streebog.h
twofish.h crypto: remove CRYPTO_TFM_RES_BAD_KEY_LEN 2020-01-09 11:30:53 +08:00
utils.h crypto: lib/utils - Move utilities into new header 2023-03-31 17:50:09 +08:00
xts.h crypto: xts - drop xts_check_key() 2023-01-06 17:15:47 +08:00