linux/arch/arm64/include/asm
Marco Elver 840b239863 arm64, kfence: enable KFENCE for ARM64
Add architecture specific implementation details for KFENCE and enable
KFENCE for the arm64 architecture. In particular, this implements the
required interface in <asm/kfence.h>.

KFENCE requires that attributes for pages from its memory pool can
individually be set. Therefore, force the entire linear map to be mapped
at page granularity. Doing so may result in extra memory allocated for
page tables in case rodata=full is not set; however, currently
CONFIG_RODATA_FULL_DEFAULT_ENABLED=y is the default, and the common case
is therefore not affected by this change.

[elver@google.com: add missing copyright and description header]
  Link: https://lkml.kernel.org/r/20210118092159.145934-3-elver@google.com

Link: https://lkml.kernel.org/r/20201103175841.3495947-4-elver@google.com
Signed-off-by: Alexander Potapenko <glider@google.com>
Signed-off-by: Marco Elver <elver@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Co-developed-by: Alexander Potapenko <glider@google.com>
Reviewed-by: Jann Horn <jannh@google.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
Cc: Andrey Konovalov <andreyknvl@google.com>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christopher Lameter <cl@linux.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Hillf Danton <hdanton@sina.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Joern Engel <joern@purestorage.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: SeongJae Park <sjpark@amazon.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2021-02-26 09:41:02 -08:00
..
vdso vdso/treewide: Add vdso_data pointer argument to __arch_get_hw_counter() 2020-08-06 10:57:30 +02:00
xen xen/arm: do not setup the runstate info page if kpti is enabled 2020-10-04 18:41:33 -05:00
acenv.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
acpi.h arm64/acpi: disallow AML memory opregions to access kernel memory 2020-07-14 18:02:03 +01:00
alternative-macros.h arm64: alternatives: Split up alternative.h 2020-11-09 21:49:34 +00:00
alternative.h arm64: alternatives: Split up alternative.h 2020-11-09 21:49:34 +00:00
arch_gicv3.h irqchip/gic-v3: Support pseudo-NMIs when SCR_EL3.FIQ == 0 2020-09-13 17:52:04 +01:00
arch_timer.h arm64: arch_timer: Allow an workaround descriptor to disable compat vdso 2020-07-08 21:57:51 +01:00
archrandom.h arm64: Add support for SMCCC TRNG entropy source 2021-01-21 17:42:46 +00:00
arm_dsu_pmu.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
arm-cci.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
asm_pointer_auth.h arm64: simplify ptrauth initialization 2020-04-28 11:23:21 +01:00
asm-bug.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
asm-offsets.h
asm-prototypes.h
asm-uaccess.h arm64: vmlinux.ld.S: add assertion for reserved_pg_dir offset 2021-02-03 20:43:45 +00:00
assembler.h Merge branch 'for-next/crypto' into for-next/core 2021-02-12 14:54:55 +00:00
atomic_ll_sc.h arm64: Move the LSE gas support detection to Kconfig 2020-01-15 12:50:48 +00:00
atomic_lse.h arm64: lse: fix LSE atomics with LLVM's integrated assembler 2020-01-16 17:25:10 +00:00
atomic.h arm64: make atomic helpers __always_inline 2021-01-13 15:09:06 +00:00
barrier.h iommu/arm-smmu-v3: Ensure queue is read after updating prod pointer 2020-09-28 22:57:43 +01:00
bitops.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
bitrev.h
boot.h arm64: get rid of TEXT_OFFSET 2020-09-07 15:00:52 +01:00
brk-imm.h arm64: kprobes: Use BRK instead of single-step when executing instructions out-of-line 2020-11-03 14:03:38 +00:00
bug.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
cache.h arm64: kasan: align allocations for HW_TAGS 2020-12-22 12:55:08 -08:00
cacheflush.h arm64: cacheflush: Remove stale comment 2021-01-26 23:40:23 +00:00
checksum.h arm64: csum: Fix handling of bad packets 2020-07-30 17:01:38 +01:00
clocksource.h arm64: Introduce asm/vdso/clocksource.h 2020-03-21 15:23:55 +01:00
cmpxchg.h arm64: fix unreachable code issue with cmpxchg 2019-09-17 12:11:50 +01:00
compat.h compat: lift compat_s64 and compat_u64 to <asm-generic/compat.h> 2020-09-17 13:00:46 -04:00
compiler.h arm64: use a common .arch preamble for inline assembly 2020-08-28 11:15:15 +01:00
cpu_ops.h arm64: fix some spelling mistakes in the comments by codespell 2020-09-07 14:18:50 +01:00
cpu.h Revert "arm64: initialize per-cpu offsets earlier" 2020-10-09 11:24:17 +01:00
cpucaps.h Merge remote-tracking branch 'origin/kvm-arm64/psci-relay' into kvmarm-master/next 2020-12-09 10:00:24 +00:00
cpufeature.h arm64: cpufeatures: Allow disabling of Pointer Auth from the command-line 2021-02-09 13:50:57 +00:00
cpuidle.h
cputype.h arm64: Add MIDR value for KRYO2XX gold/silver CPU cores 2020-11-13 09:47:08 +00:00
current.h
daifflags.h arm64: entry: fix non-NMI kernel<->kernel transitions 2020-11-30 12:11:38 +00:00
dcc.h
debug-monitors.h arm64: kprobes: Use BRK instead of single-step when executing instructions out-of-line 2020-11-03 14:03:38 +00:00
device.h arm64: Remove dev->archdata.iommu pointer 2020-06-30 11:59:49 +02:00
dmi.h
efi.h efi: stub: get rid of efi_get_max_fdt_addr() 2020-12-09 08:37:27 +01:00
el2_setup.h arm64: Simplify init_el2_state to be non-VHE only 2021-02-09 13:47:11 +00:00
elf.h Split the old READ_IMPLIES_EXEC workaround from executable PT_GNU_STACK 2020-06-05 13:45:21 -07:00
esr.h arm64: mte: add in-kernel MTE helpers 2020-12-22 12:55:07 -08:00
exception.h arm64: entry: suppress W=1 prototype warnings 2020-12-15 12:38:02 +00:00
exec.h arm64: uaccess: remove set_fs() 2020-12-02 19:49:11 +00:00
extable.h arm64: Improve diagnostics when trapping BRK with FAULT_BRK_IMM 2020-09-18 16:35:54 +01:00
fb.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
fixmap.h arm64/fixmap: make notes of fixed_addresses more precisely 2020-08-08 19:25:04 +01:00
fpsimd.h arm64/sve: Implement a helper to load SVE registers from FPSIMD state 2020-09-21 18:06:33 +01:00
fpsimdmacros.h arm64/sve: Implement a helper to flush SVE registers 2020-09-21 18:06:33 +01:00
ftrace.h arm64: implement ftrace with regs 2019-11-06 14:17:35 +00:00
futex.h arm64: uaccess: rename privileged uaccess routines 2020-12-02 19:49:10 +00:00
hardirq.h arm64: irqstat: Get rid of duplicated declaration 2020-11-23 10:31:05 +01:00
hugetlb.h arm64/hugetlb: Reserve CMA areas for gigantic pages on 16K and 64K configs 2020-07-15 13:38:03 +01:00
hw_breakpoint.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
hwcap.h Merge branch 'for-next/mte' into for-next/core 2020-10-02 12:16:11 +01:00
hyp_image.h KVM: arm64: Add symbol at the beginning of each hyp section 2021-01-23 14:00:57 +00:00
hypervisor.h
image.h
insn.h arm64: alternatives: Split up alternative.h 2020-11-09 21:49:34 +00:00
io.h Add and use a generic version of devmem_is_allowed() 2020-12-11 12:30:26 -08:00
irq_work.h arm64: Kill __smp_cross_call and co 2020-09-17 16:37:28 +01:00
irq.h
irqflags.h arm64: Implement arch_irqs_disabled() 2020-08-26 12:41:55 +02:00
jump_label.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
kasan.h kasan, arm64: move initialization message 2020-12-22 12:55:07 -08:00
Kbuild local64.h: make <asm/local64.h> mandatory 2020-12-29 15:36:49 -08:00
kernel-pgtable.h arm64: consistently use reserved_pg_dir 2020-11-10 17:30:40 +00:00
kexec.h arm64: kexec: move relocation function setup 2021-01-27 15:41:12 +00:00
kfence.h arm64, kfence: enable KFENCE for ARM64 2021-02-26 09:41:02 -08:00
kgdb.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
kprobes.h arm64: kprobes: Remove redundant kprobe_step_ctx 2020-11-10 17:12:30 +00:00
kvm_arm.h KVM: arm64: Trap host SMCs in protected mode 2020-12-04 10:08:36 +00:00
kvm_asm.h KVM: arm64: Remove hyp_symbol_addr 2021-01-23 14:01:00 +00:00
kvm_emulate.h Merge branch 'kvm-master' into kvm-next 2021-01-07 18:06:52 -05:00
kvm_host.h KVM/arm64 updates for Linux 5.12 2021-02-12 11:23:44 -05:00
kvm_hyp.h KVM: arm64: Bootstrap PSCI SMC handler in nVHE EL2 2020-12-04 10:08:34 +00:00
kvm_mmu.h KVM: arm64: Remove patching of fn pointers in hyp 2021-01-23 14:01:00 +00:00
kvm_pgtable.h KVM: arm64: Filter out the case of only changing permissions from stage-2 map path 2021-01-25 16:30:20 +00:00
kvm_ptrauth.h KVM: arm64: Update context references from host to hyp 2020-09-15 18:39:02 +01:00
kvm_ras.h
kvm_types.h KVM: arm64: Use common KVM implementation of MMU memory caches 2020-07-09 13:29:43 -04:00
linkage.h arm64: Don't insert a BTI instruction at inner labels 2020-06-24 14:24:29 +01:00
lse.h arm64: lse: Fix LSE atomics with LLVM 2020-02-18 18:10:49 +00:00
memory.h kasan, arm64: allow using KUnit tests with HW_TAGS mode 2021-02-24 13:38:31 -08:00
mman.h arm64: mte: Validate the PROT_MTE request via arch_validate_flags() 2020-09-04 12:46:07 +01:00
mmu_context.h arm64: mm: Always update TCR_EL1 from __cpu_set_tcr_t0sz() 2021-01-27 15:41:12 +00:00
mmu.h KVM: arm64: Move BP hardening helpers into spectre.h 2020-11-16 10:40:18 +00:00
mmzone.h
module.h arch: split MODULE_ARCH_VERMAGIC definitions out to <asm/vermagic.h> 2020-04-23 10:50:26 +09:00
module.lds.h kbuild: preprocess module linker script 2020-09-25 00:36:41 +09:00
mte-def.h arm64: mte: switch GCR_EL1 in kernel entry and exit 2020-12-22 12:55:07 -08:00
mte-kasan.h kasan, arm64: allow using KUnit tests with HW_TAGS mode 2021-02-24 13:38:31 -08:00
mte.h arm64: mte: switch GCR_EL1 in kernel entry and exit 2020-12-22 12:55:07 -08:00
neon-intrinsics.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
neon.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
numa.h arm64/mm: return cpu_all_mask when node is NUMA_NO_NODE 2020-09-21 22:05:01 +01:00
page-def.h arm64/mm: Unify CONT_PTE_SHIFT 2020-09-11 16:33:43 +01:00
page.h arm64: mte: Add PROT_MTE support to mmap() and mprotect() 2020-09-04 12:46:07 +01:00
paravirt.h arm64: Retrieve stolen time as paravirtualized guest 2019-10-21 19:20:31 +01:00
pci.h arm64: Enable PCI write-combine resources under sysfs 2020-09-18 14:05:13 +01:00
percpu.h KVM: arm64: Support per_cpu_ptr in nVHE hyp code 2020-12-04 10:08:34 +00:00
perf_event.h arm64: perf: Add support caps under sysfs 2020-09-28 14:53:45 +01:00
pgalloc.h asm-generic: pgalloc: provide generic pgd_free() 2020-08-07 11:33:26 -07:00
pgtable-hwdef.h kasan: arm64: set TCR_EL1.TBID1 when enabled 2020-11-25 16:53:19 +00:00
pgtable-prot.h ARM: 2020-10-23 11:17:56 -07:00
pgtable-types.h arm64: add support for folded p4d page tables 2020-06-04 19:06:21 -07:00
pgtable.h arm64: mm: Implement arch_wants_old_prefaulted_pte() 2021-01-20 14:46:04 +00:00
pointer_auth.h arm64: Defer enabling pointer authentication on boot core 2021-02-09 13:50:57 +00:00
preempt.h sched/rt, arm64: Use CONFIG_PREEMPTION 2019-12-08 14:37:32 +01:00
probes.h arm64/fpsimd: add <asm/insn.h> to <asm/kprobes.h> to fix fpsimd build 2020-11-23 10:59:00 +00:00
proc-fns.h arm64: mm: convert cpu_do_switch_mm() to C 2020-02-27 14:30:50 +00:00
processor.h arm64: Remove arm64_dma32_phys_limit and its uses 2021-01-12 17:49:25 +00:00
ptdump.h arm64: mm: convert mm/dump.c to use walk_page_range() 2020-02-04 03:05:25 +00:00
ptrace.h Merge branch 'kvm-master' into kvm-next 2021-01-07 18:06:52 -05:00
pvclock-abi.h KVM: arm64: Implement PV_TIME_FEATURES call 2019-10-21 19:20:27 +01:00
rwonce.h arm64: lto: Strengthen READ_ONCE() to acquire when CONFIG_LTO=y 2020-11-09 21:49:34 +00:00
scs.h scs: Move scs_overflow_check() out of architecture code 2020-05-18 17:47:40 +01:00
sdei.h
seccomp.h arm64: Enable seccomp architecture tracking 2020-11-20 11:16:34 -08:00
sections.h KVM: arm64: Apply hyp relocations at runtime 2021-01-23 14:01:00 +00:00
setup.h arm64: Extract early FDT mapping from kaslr_early_init() 2021-02-09 13:47:50 +00:00
shmparam.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
signal32.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
signal.h arm64: expose FAR_EL1 tag bits in siginfo 2020-11-23 18:17:39 +00:00
simd.h arm64: fpsimd: Make sure SVE setup is complete before SIMD is used 2020-01-14 17:11:21 +00:00
smp_plat.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
smp.h arm64: Make cpu_logical_map() take unsigned int 2020-12-04 08:44:24 +00:00
sparsemem.h arm64/sparsemem: reduce SECTION_SIZE_BITS 2021-01-21 18:06:37 +00:00
spectre.h Merge remote-tracking branch 'origin/kvm-arm64/csv3' into kvmarm-master/queue 2020-12-03 19:12:24 +00:00
spinlock_types.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
spinlock.h locking/arch: Move qrwlock.h include after qspinlock.h 2021-02-11 07:59:54 -05:00
stack_pointer.h
stackprotector.h arm64: Defer enabling pointer authentication on boot core 2021-02-09 13:50:57 +00:00
stacktrace.h arm64: stacktrace: Make stack walk callback consistent with generic code 2020-09-18 14:24:16 +01:00
stage2_pgtable.h KVM: arm64: Remove unused page-table code 2020-09-11 15:51:15 +01:00
stat.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
string.h kasan, arm64: expand CONFIG_KASAN checks 2020-12-22 12:55:08 -08:00
suspend.h arm64: Preserve register x18 when CPU is suspended 2020-05-15 16:35:50 +01:00
sync_bitops.h
syscall_wrapper.h arm64: simplify syscall wrapper ifdeffery 2019-10-14 10:55:00 +01:00
syscall.h arm64: compat: Ensure upper 32 bits of x0 are zero on syscall return 2020-07-16 11:41:31 +01:00
sysreg.h Char/Misc driver patches for 5.12-rc1 2021-02-24 10:25:37 -08:00
system_misc.h arm64: expose FAR_EL1 tag bits in siginfo 2020-11-23 18:17:39 +00:00
thread_info.h arm64: make _TIF_WORK_MASK bits contiguous 2020-12-16 14:08:47 -08:00
timex.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
tlb.h arm64: tlb: Set the TTL field in flush_tlb_range 2020-07-07 11:23:47 +01:00
tlbflush.h arm64: use a common .arch preamble for inline assembly 2020-08-28 11:15:15 +01:00
topology.h arm64: wrap and generalise counter read functions 2020-11-13 20:05:10 +00:00
trans_pgd.h arm64: trans_pgd: hibernate: idmap the single page that holds the copy page routines 2021-01-27 15:41:12 +00:00
traps.h arm64: expose FAR_EL1 tag bits in siginfo 2020-11-23 18:17:39 +00:00
uaccess.h arm64: vmlinux.ld.S: add assertion for reserved_pg_dir offset 2021-02-03 20:43:45 +00:00
unistd32.h fs: add mount_setattr() 2021-01-24 14:42:45 +01:00
unistd.h fs: add mount_setattr() 2021-01-24 14:42:45 +01:00
uprobes.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
vdso.h arm64/vdso: Add time namespace page 2020-07-24 13:15:20 +01:00
vermagic.h arch: split MODULE_ARCH_VERMAGIC definitions out to <asm/vermagic.h> 2020-04-23 10:50:26 +09:00
virt.h arm64: Provide an 'upgrade to VHE' stub hypercall 2021-02-08 12:51:26 +00:00
vmalloc.h mm/vmalloc: Add empty <asm/vmalloc.h> headers and use them from <linux/vmalloc.h> 2019-12-10 10:12:55 +01:00
vmap_stack.h mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
word-at-a-time.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
xor.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00