linux/lib
Alexander Potapenko f80be4571b kmsan: add KMSAN runtime core
For each memory location KernelMemorySanitizer maintains two types of
metadata:

1. The so-called shadow of that location - а byte:byte mapping describing
   whether or not individual bits of memory are initialized (shadow is 0)
   or not (shadow is 1).
2. The origins of that location - а 4-byte:4-byte mapping containing
   4-byte IDs of the stack traces where uninitialized values were
   created.

Each struct page now contains pointers to two struct pages holding KMSAN
metadata (shadow and origins) for the original struct page.  Utility
routines in mm/kmsan/core.c and mm/kmsan/shadow.c handle the metadata
creation, addressing, copying and checking.  mm/kmsan/report.c performs
error reporting in the cases an uninitialized value is used in a way that
leads to undefined behavior.

KMSAN compiler instrumentation is responsible for tracking the metadata
along with the kernel memory.  mm/kmsan/instrumentation.c provides the
implementation for instrumentation hooks that are called from files
compiled with -fsanitize=kernel-memory.

To aid parameter passing (also done at instrumentation level), each
task_struct now contains a struct kmsan_task_state used to track the
metadata of function parameters and return values for that task.

Finally, this patch provides CONFIG_KMSAN that enables KMSAN, and declares
CFLAGS_KMSAN, which are applied to files compiled with KMSAN.  The
KMSAN_SANITIZE:=n Makefile directive can be used to completely disable
KMSAN instrumentation for certain files.

Similarly, KMSAN_ENABLE_CHECKS:=n disables KMSAN checks and makes newly
created stack memory initialized.

Users can also use functions from include/linux/kmsan-checks.h to mark
certain memory regions as uninitialized or initialized (this is called
"poisoning" and "unpoisoning") or check that a particular region is
initialized.

Link: https://lkml.kernel.org/r/20220915150417.722975-12-glider@google.com
Signed-off-by: Alexander Potapenko <glider@google.com>
Acked-by: Marco Elver <elver@google.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Christoph Lameter <cl@linux.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Eric Biggers <ebiggers@google.com>
Cc: Eric Biggers <ebiggers@kernel.org>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Ilya Leoshkevich <iii@linux.ibm.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Petr Mladek <pmladek@suse.com>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2022-10-03 14:03:19 -07:00
..
842
crypto crypto: lib - add module license to libsha1 2022-07-20 15:28:07 +08:00
dim dim: initialize all struct fields 2022-05-09 17:20:37 -07:00
fonts lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
kunit kunit: executor: Fix a memory leak on failure in kunit_filter_tests 2022-07-14 10:35:56 -06:00
livepatch selftests/livepatch: better synchronize test_klp_callbacks_busy 2022-06-15 10:29:10 +02:00
lz4 lib: make LZ4_decompress_safe_forceExtDict() static 2022-07-17 17:31:39 -07:00
lzo lib/lzo/lzo1x_compress.c: replace ternary operator with min() and min_t() 2022-07-29 18:12:34 -07:00
math math: make RATIONAL tristate 2021-09-08 11:50:26 -07:00
mpi Updates to various subsystems which I help look after. lib, ocfs2, 2022-08-07 10:03:24 -07:00
pldmfw lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
raid6 lib/raid6: Include <asm/ppc-opcode.h> for VPERMXOR 2022-03-08 15:20:21 -08:00
reed_solomon lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
test_fortify fortify: Detect struct member overflows in memset() at compile-time 2022-02-13 16:50:06 -08:00
vdso lib/vdso: Add vdso_data pointer as input to __arch_get_timens_vdso_data() 2021-04-14 23:04:44 +10:00
xz lib/xz, lib/decompress_unxz.c: Fix spelling in comments 2021-10-19 23:44:30 +08:00
zlib_deflate
zlib_dfltcc
zlib_inflate lib/zlib_inflate/inffast: check config in C to avoid unused function warning 2021-09-24 16:13:35 -07:00
zstd lib: zstd: Don't add -O3 to cflags 2021-11-18 13:16:22 -08:00
.gitignore bootconfig: Support embedding a bootconfig file in kernel 2022-04-26 17:58:51 -04:00
argv_split.c
ashldi3.c
ashrdi3.c
asn1_decoder.c
asn1_encoder.c lib: remove redundant assignment to variable ret 2022-01-20 08:52:55 +02:00
assoc_array.c assoc_array: Fix BUG_ON during garbage collect 2022-06-01 18:29:06 -07:00
atomic64_test.c
atomic64.c locking/atomic: atomic64: Remove unusable atomic ops 2021-12-13 10:56:09 +01:00
audit.c audit: add support for the openat2 syscall 2021-10-01 16:52:48 -04:00
base64.c lib/base64: RFC4648-compliant base64 encoding 2022-08-02 17:14:47 -06:00
bcd.c
bch.c lib/bch.c: fix a typo in the file bch.c 2021-05-06 19:24:12 -07:00
bitfield_kunit.c
bitmap.c Bitmap patches for v6.0-rc1 2022-08-07 17:52:35 -07:00
bitrev.c
bootconfig-data.S bootconfig: Support embedding a bootconfig file in kernel 2022-04-26 17:58:51 -04:00
bootconfig.c bootconfig: Support embedding a bootconfig file in kernel 2022-04-26 17:58:51 -04:00
bsearch.c
btree.c lib/btree: simplify btree_{lookup|update} 2022-06-16 19:58:21 -07:00
bucket_locks.c
bug.c bug: Use normal relative pointers in 'struct bug_entry' 2022-05-19 23:46:10 +02:00
build_OID_registry
buildid.c kdump: use vmlinux_build_id to simplify 2021-07-08 11:48:22 -07:00
bust_spinlocks.c
check_signature.c
checksum.c
clz_ctz.c
clz_tab.c
cmdline_kunit.c lib/cmdline_kunit: Remove a cast which are no-longer required 2021-06-23 16:41:41 -06:00
cmdline.c lib/cmdline: Export next_arg() for being used in modules 2021-05-05 16:07:40 +02:00
cmpdi2.c
compat_audit.c audit: add support for the openat2 syscall 2021-10-01 16:52:48 -04:00
cpu_rmap.c
cpumask_kunit.c lib/cpumask_kunit: log mask contents 2022-08-24 08:35:42 -07:00
cpumask.c lib/cpumask: drop always-true preprocessor guard 2022-08-15 11:00:44 -07:00
crc4.c
crc7.c
crc8.c lib: crc8: pointer to data block should be const 2021-05-06 19:24:12 -07:00
crc16.c
crc32.c lib/crc32: Make crc32_be weak for arch override 2022-01-31 11:21:43 +11:00
crc32defs.h
crc32test.c lib/crc32test: correct printed bytes count 2022-01-31 11:21:43 +11:00
crc64-rocksoft.c crypto: add rocksoft 64b crc guard tag framework 2022-03-07 12:48:35 -07:00
crc64.c lib: add rocksoft model crc64 2022-03-07 12:48:35 -07:00
crc-ccitt.c
crc-itu-t.c crc-itu-t: fix typo in CRC ITU-T polynomial comment 2022-06-07 10:27:38 +02:00
crc-t10dif.c
ctype.c
debug_info.c isystem: ship and use stdarg.h 2021-08-19 09:02:55 +09:00
debug_locks.c locking/lockdep: Improve noinstr vs errors 2021-06-22 13:56:43 +02:00
debugobjects.c debugobjects: Convert to SPDX license identifier 2022-05-13 12:35:17 +02:00
dec_and_lock.c
decompress_bunzip2.c lib/decompressors: fix spelling mistakes 2021-07-01 11:06:05 -07:00
decompress_inflate.c
decompress_unlz4.c lib/decompress_unlz4.c: correctly handle zero-padding around initrds. 2021-07-01 11:06:06 -07:00
decompress_unlzma.c lib: fix inconsistent indenting in process_bit1() 2021-05-06 19:24:12 -07:00
decompress_unlzo.c lib/decompressors: remove set but not used variabled 'level' 2021-07-01 11:06:06 -07:00
decompress_unxz.c lib/xz, lib/decompress_unxz.c: Fix spelling in comments 2021-10-19 23:44:30 +08:00
decompress_unzstd.c lib: zstd: Add decompress_sources.h for decompress_unzstd 2021-11-08 16:55:26 -08:00
decompress.c
devmem_is_allowed.c lib: use PFN_PHYS() in devmem_is_allowed() 2021-08-13 14:09:32 -10:00
devres.c lib: devres: use numa aware allocation 2022-07-17 17:31:41 -07:00
digsig.c
dump_stack.c printk: rename cpulock functions 2022-04-22 21:30:57 +02:00
dynamic_debug.c dyndbg: refine verbosity 1-4 summary-detail 2021-10-21 13:01:25 +02:00
dynamic_queue_limits.c
earlycpio.c lib: remove "expecting prototype" kernel-doc warnings 2021-04-16 16:10:37 -07:00
errname.c
error-inject.c lib/error-inject: traverse list with mutex 2022-07-17 17:31:38 -07:00
errseq.c
extable.c
fault-inject-usercopy.c
fault-inject.c mm: fix missing handler for __GFP_NOWARN 2022-05-19 14:08:55 -07:00
fdt_addresses.c
fdt_empty_tree.c
fdt_ro.c
fdt_rw.c
fdt_strerror.c
fdt_sw.c
fdt_wip.c
fdt.c
find_bit_benchmark.c lib: add find_first_and_bit() 2022-01-15 08:47:31 -08:00
find_bit.c lib: add find_first_and_bit() 2022-01-15 08:47:31 -08:00
flex_proportions.c lib/flex_proportions.c: remove local_irq_ops in fprop_new_period() 2022-06-16 19:58:20 -07:00
gen_crc32table.c
gen_crc64table.c lib: add rocksoft model crc64 2022-03-07 12:48:35 -07:00
genalloc.c all: replace find_next{,_zero}_bit with find_first{,_zero}_bit where appropriate 2022-01-15 08:47:31 -08:00
generic-radix-tree.c
glob.c lib: remove back_str initialization 2022-04-29 14:38:01 -07:00
globtest.c
hexdump.c hex2bin: fix access beyond string end 2022-04-27 10:57:33 -07:00
hweight.c
idr.c ida: don't use BUG_ON() for debugging 2022-07-10 13:55:49 -07:00
inflate.c
interval_tree_test.c
interval_tree.c
iomap_copy.c
iomap.c
iommu-helper.c
iov_iter.c instrumented.h: allow instrumenting both sides of copy_from_user() 2022-10-03 14:03:18 -07:00
irq_poll.c lib/irq_poll: Prevent softirq pending leak in irq_poll_cpu_dead() 2022-04-13 21:32:21 +02:00
irq_regs.c
is_single_threaded.c
kasprintf.c isystem: ship and use stdarg.h 2021-08-19 09:02:55 +09:00
Kconfig Updates to various subsystems which I help look after. lib, ocfs2, 2022-08-07 10:03:24 -07:00
Kconfig.debug kmsan: add KMSAN runtime core 2022-10-03 14:03:19 -07:00
Kconfig.kasan kasan: drop CONFIG_KASAN_TAGS_IDENTIFY 2022-10-03 14:02:57 -07:00
Kconfig.kcsan objtool: Make noinstr hacks optional 2022-04-22 12:32:04 +02:00
Kconfig.kfence kfence: allow use of a deferrable timer 2022-03-22 15:57:11 -07:00
Kconfig.kgdb
Kconfig.kmsan kmsan: add KMSAN runtime core 2022-10-03 14:03:19 -07:00
Kconfig.ubsan ubsan: disable UBSAN_DIV_ZERO for clang 2022-07-14 15:45:26 -07:00
kfifo.c lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
klist.c
kobject_uevent.c kobject: remove kset from struct kset_uevent_ops callbacks 2021-12-28 11:26:18 +01:00
kobject.c kobject: kobj_type: remove default_attrs 2022-04-05 15:39:19 +02:00
kstrtox.c lib/kstrtox.c: add "false"/"true" support to kstrtobool() 2022-05-13 07:20:13 -07:00
kstrtox.h lib: vsprintf: Fix handling of number field widths in vsscanf 2021-05-19 15:05:11 +02:00
libcrc32c.c
linear_ranges.c lib: add linear range get selector within 2021-08-13 18:37:38 +02:00
list_debug.c lib/list_debug.c: Detect uninitialized lists 2022-06-16 19:58:20 -07:00
list_sort.c lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
list-test.c list: test: Test the hlist structure 2022-04-05 13:32:27 -06:00
llist.c
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-rtmutex.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c lockdep/selftests: Adapt ww-tests for PREEMPT_RT 2021-12-04 10:56:24 +01:00
lockref.c lockref: remove unused 'lockref_get_or_lock()' function 2022-07-03 14:40:28 -07:00
logic_iomem.c lib/logic_iomem: correct fallback config references 2022-03-11 10:42:56 +01:00
logic_pio.c
lru_cache.c lib/lru_cache: fix error free handing in lc_create 2022-07-17 17:31:37 -07:00
lshrdi3.c
Makefile kasan: move tests to mm/kasan/ 2022-10-03 14:03:02 -07:00
maple_tree.c Maple Tree: add new data structure 2022-09-26 19:46:13 -07:00
memcat_p.c
memcpy_kunit.c string.h: Introduce memset_startat() for wiping trailing members and padding 2021-10-18 12:28:52 -07:00
memneq.c crypto: memneq - move into lib/ 2022-06-12 14:51:51 +08:00
memory-notifier-error-inject.c
memregion.c
memweight.c
muldi3.c
net_utils.c
netdev-notifier-error-inject.c
nlattr.c lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
nmi_backtrace.c printk: rename cpulock functions 2022-04-22 21:30:57 +02:00
notifier-error-inject.c
notifier-error-inject.h
objagg.c lib: objagg: Use the bitmap API when applicable 2021-12-24 14:54:29 -08:00
of-reconfig-notifier-error-inject.c
oid_registry.c lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
once.c once: Fix panic when module unload 2021-08-08 13:00:20 +01:00
overflow_kunit.c lib: overflow: Do not define 64-bit tests on 32-bit 2022-07-01 17:39:49 -07:00
packing.c net: update NXP copyright text 2021-09-17 13:52:17 +01:00
parman.c lib: remove "expecting prototype" kernel-doc warnings 2021-04-16 16:10:37 -07:00
parser.c kernel.h: split out kstrtox() and simple_strtox() to a separate header 2021-07-01 11:06:05 -07:00
pci_iomap.c pci_iounmap'2: Electric Boogaloo: try to make sense of it all 2021-09-19 17:13:35 -07:00
percpu_counter.c lib/percpu_counter: tame kernel-doc compile warning 2021-05-06 19:24:12 -07:00
percpu_test.c
percpu-refcount.c percpu_ref_init(): clean ->percpu_count_ref on failure 2022-05-18 02:20:17 -04:00
plist.c
pm-notifier-error-inject.c
polynomial.c lib: add generic polynomial calculation 2022-05-22 11:32:30 -07:00
radix-tree.c lib/radix-tree: remove unused argument of insert_entries 2022-07-17 17:31:38 -07:00
random32.c random32: use real rng for non-deterministic randomness 2022-05-18 15:53:52 +02:00
ratelimit.c ratelimit: Fix data-races in ___ratelimit(). 2022-08-24 13:46:57 +01:00
rbtree_test.c
rbtree.c
ref_tracker.c ref_tracker: remove filter_irq_stacks() call 2022-02-06 11:05:28 +00:00
refcount.c lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
rhashtable.c lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
sbitmap.c lib/sbitmap: Fix invalid loop in __sbitmap_queue_get_batch() 2022-06-25 10:58:55 -06:00
scatterlist.c lib/scatterlist: use matched parameter type when calling __sg_free_table() 2022-07-17 17:31:39 -07:00
seq_buf.c lib: fix spelling mistakes 2021-07-08 11:48:20 -07:00
sg_pool.c lib/scatterlist: Fix wrong update of orig_nents 2021-08-24 19:52:40 -03:00
sg_split.c
show_mem.c mm: reduce noise in show_mem for lowmem allocations 2022-09-26 19:46:29 -07:00
siphash.c SPDX changes for 5.19-rc1 2022-06-03 10:34:34 -07:00
slub_kunit.c mm/slub, kunit: Make slub_kunit unaffected by user specified flags 2022-04-06 10:11:48 +02:00
smp_processor_id.c lib/smp_processor_id: fix imbalanced instrumentation_end() call 2022-07-17 17:31:41 -07:00
sort.c lib/sort: Add priv pointer to swap function 2022-03-17 20:17:18 -07:00
stackdepot.c stackdepot: reserve 5 extra bits in depot_stack_handle_t 2022-10-03 14:03:18 -07:00
stackinit_kunit.c lib: stackinit: Convert to KUnit 2022-03-21 08:13:04 -07:00
stmp_device.c
string_helpers.c lib/string_helpers: fix not adding strarray to device's resource list 2022-05-12 20:38:36 -07:00
string.c lib/string.c: simplify str[c]spn 2022-04-29 14:38:01 -07:00
strncpy_from_user.c lib/strn*,objtool: Enforce user_access_begin() rules 2022-04-19 21:58:47 +02:00
strnlen_user.c lib/strn*,objtool: Enforce user_access_begin() rules 2022-04-19 21:58:47 +02:00
syscall.c sched: Change task_struct::state 2021-06-18 11:43:09 +02:00
test_bitmap.c lib/test_bitmap: test the tail after bitmap_to_arr64() 2022-07-12 08:00:50 -07:00
test_bitops.c lib/test: fix spelling mistakes 2021-07-08 11:48:20 -07:00
test_bits.c
test_blackhole_dev.c
test_bpf.c test_bpf: fix incorrect netdev features 2022-06-22 19:20:20 -07:00
test_debug_virtual.c
test_firmware.c test_firmware: fix end of loop test in upload_read_show() 2022-05-19 19:28:37 +02:00
test_fprobe.c fprobe: Add a selftest for fprobe 2022-03-17 20:17:14 -07:00
test_fpu.c
test_free_pages.c lib/test_free_pages.c: pass a pointer to virt_to_page() 2022-07-17 17:14:36 -07:00
test_hash.c test_hash.c: refactor into kunit 2022-01-20 08:52:54 +02:00
test_hexdump.c
test_hmm_uapi.h lib: add support for device coherent type in test_hmm 2022-07-17 17:14:29 -07:00
test_hmm.c mm/hmm/test: use char dev with struct device to get device node 2022-10-03 14:03:03 -07:00
test_ida.c
test_kmod.c lib/test: use after free in register_test_dev_kmod() 2022-03-29 15:13:36 -07:00
test_kprobes.c test_kprobes: Move it from kernel/ to lib/ 2021-10-26 17:23:46 -04:00
test_linear_ranges.c
test_list_sort.c lib/test: convert lib/test_list_sort.c to use KUnit 2021-06-25 11:31:03 -06:00
test_lockup.c lib/test_lockup: fix kernel pointer check for separate address spaces 2022-02-25 09:36:06 +01:00
test_maple_tree.c lib/test_maple_tree: add testing for maple tree 2022-09-26 19:46:14 -07:00
test_memcat_p.c
test_meminit.c lib/test_meminit: optimize do_kmem_cache_rcu_persistent() test 2022-04-29 14:38:00 -07:00
test_min_heap.c
test_module.c
test_objagg.c
test_parman.c
test_printf.c lib/test_printf.c: fix clang -Wformat warnings 2022-07-28 10:38:30 +02:00
test_ref_tracker.c lib: add tests for reference tracker 2021-12-06 16:04:44 -08:00
test_rhashtable.c
test_scanf.c lib/test_scanf: split up number parsing test routines 2021-09-06 11:04:03 -07:00
test_siphash.c siphash: add SPDX tags as sole licensing authority 2022-05-19 18:54:22 +02:00
test_sort.c lib/test: convert test_sort.c to use KUnit 2021-09-08 11:50:26 -07:00
test_static_key_base.c
test_static_keys.c
test_string.c lib/test_string.c: add strspn and strcspn tests 2022-04-29 14:38:00 -07:00
test_strscpy.c
test_sysctl.c selftests/sysctl: add sysctl macro test 2022-05-03 10:15:07 +02:00
test_ubsan.c ubsan: remove CONFIG_UBSAN_OBJECT_SIZE 2022-01-20 08:52:55 +02:00
test_user_copy.c
test_uuid.c
test_vmalloc.c lib/test_vmalloc: switch to prandom_u32() 2022-07-03 18:08:42 -07:00
test_xarray.c XArray: Fix xas_create_range() when multi-order entry present 2022-03-28 19:25:11 -04:00
test-kstrtox.c
test-string_helpers.c string_helpers: Escape double quotes in escape_special 2021-07-19 11:39:28 +02:00
textsearch.c
timerqueue.c
trace_readwrite.c lib: Add register read/write tracing support 2022-06-15 17:41:12 +02:00
ts_bm.c lib/ts_bm.c: remove redundant store to variable consumed after addition 2022-07-17 17:31:39 -07:00
ts_fsm.c
ts_kmp.c
ubsan.c ubsan: no need to unset panic_on_warn in ubsan_epilogue() 2022-03-23 19:00:35 -07:00
ubsan.h
ucmpdi2.c
ucs2_string.c
usercopy.c instrumented.h: allow instrumenting both sides of copy_from_user() 2022-10-03 14:03:18 -07:00
uuid.c
vsprintf.c random: remove rng_has_arch_random() 2022-06-10 11:29:48 +02:00
win_minmax.c
xarray.c mm/huge_memory: Fix xarray node memory leak 2022-06-09 16:24:25 -04:00
xxhash.c