leandrof/linux
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7cd1ea1010
linux/arch/arm64/include/asm
History
Mark Rutland 7cd1ea1010 arm64: entry: fix non-NMI kernel<->kernel transitions 
There are periods in kernel mode when RCU is not watching and/or the
scheduler tick is disabled, but we can still take exceptions such as
interrupts. The arm64 exception handlers do not account for this, and
it's possible that RCU is not watching while an exception handler runs.

The x86/generic entry code handles this by ensuring that all (non-NMI)
kernel exception handlers call irqentry_enter() and irqentry_exit(),
which handle RCU, lockdep, and IRQ flag tracing. We can't yet move to
the generic entry code, and already hadnle the user<->kernel transitions
elsewhere, so we add new kernel<->kernel transition helpers alog the
lines of the generic entry code.

Since we now track interrupts becoming masked when an exception is
taken, local_daif_inherit() is modified to track interrupts becoming
re-enabled when the original context is inherited. To balance the
entry/exit paths, each handler masks all DAIF exceptions before
exit_to_kernel_mode().

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20201130115950.22492-10-mark.rutland@arm.com
Signed-off-by: Will Deacon <will@kernel.org>
2020-11-30 12:11:38 +00:00
..
vdso vdso/treewide: Add vdso_data pointer argument to __arch_get_hw_counter() 2020-08-06 10:57:30 +02:00
xen xen/arm: do not setup the runstate info page if kpti is enabled 2020-10-04 18:41:33 -05:00
acenv.h
acpi.h arm64/acpi: disallow AML memory opregions to access kernel memory 2020-07-14 18:02:03 +01:00
alternative.h arm64/alternatives: move length validation inside the subsection 2020-07-30 16:50:14 +01:00
arch_gicv3.h irqchip/gic-v3: Support pseudo-NMIs when SCR_EL3.FIQ == 0 2020-09-13 17:52:04 +01:00
arch_timer.h arm64: arch_timer: Allow an workaround descriptor to disable compat vdso 2020-07-08 21:57:51 +01:00
archrandom.h arm64: random: Remove no longer needed prototypes 2020-10-07 14:17:20 +01:00
arm_dsu_pmu.h
arm-cci.h
asm_pointer_auth.h arm64: simplify ptrauth initialization 2020-04-28 11:23:21 +01:00
asm-bug.h
asm-offsets.h
asm-prototypes.h
asm-uaccess.h arm64 updates for 5.5: 2019-11-25 15:39:19 -08:00
assembler.h kvm: arm64: Remove hyp_adr/ldr_this_cpu 2020-09-30 08:37:07 +01:00
atomic_ll_sc.h arm64: Move the LSE gas support detection to Kconfig 2020-01-15 12:50:48 +00:00
atomic_lse.h arm64: lse: fix LSE atomics with LLVM's integrated assembler 2020-01-16 17:25:10 +00:00
atomic.h locking/atomic: Move ATOMIC_INIT into linux/types.h 2020-07-29 16:14:18 +02:00
barrier.h iommu/arm-smmu-v3: Ensure queue is read after updating prod pointer 2020-09-28 22:57:43 +01:00
bitops.h
bitrev.h
boot.h arm64: get rid of TEXT_OFFSET 2020-09-07 15:00:52 +01:00
brk-imm.h arm64: kprobes: Use BRK instead of single-step when executing instructions out-of-line 2020-11-03 14:03:38 +00:00
bug.h
cache.h arm64: avoid -Woverride-init warning 2020-10-28 13:38:36 +00:00
cacheflush.h arm64: use asm-generic/cacheflush.h 2020-06-08 11:05:57 -07:00
checksum.h arm64: csum: Fix handling of bad packets 2020-07-30 17:01:38 +01:00
clocksource.h arm64: Introduce asm/vdso/clocksource.h 2020-03-21 15:23:55 +01:00
cmpxchg.h
compat.h compat: lift compat_s64 and compat_u64 to <asm-generic/compat.h> 2020-09-17 13:00:46 -04:00
compiler.h arm64: use a common .arch preamble for inline assembly 2020-08-28 11:15:15 +01:00
cpu_ops.h arm64: fix some spelling mistakes in the comments by codespell 2020-09-07 14:18:50 +01:00
cpu.h Revert "arm64: initialize per-cpu offsets earlier" 2020-10-09 11:24:17 +01:00
cpucaps.h arm64: Add workaround for Arm Cortex-A77 erratum 1508412 2020-10-29 12:56:01 +00:00
cpufeature.h arm64: errata: Fix handling of 1418040 with late CPU onlining 2020-11-10 13:09:39 +00:00
cpuidle.h
cputype.h arm64: Add MIDR value for KRYO2XX gold/silver CPU cores 2020-11-13 09:47:08 +00:00
current.h
daifflags.h arm64: entry: fix non-NMI kernel<->kernel transitions 2020-11-30 12:11:38 +00:00
dcc.h
debug-monitors.h arm64: kprobes: Use BRK instead of single-step when executing instructions out-of-line 2020-11-03 14:03:38 +00:00
device.h arm64: Remove dev->archdata.iommu pointer 2020-06-30 11:59:49 +02:00
dmi.h
efi.h efi/libstub: arm32: Base FDT and initrd placement on image address 2020-09-16 18:53:42 +03:00
elf.h Split the old READ_IMPLIES_EXEC workaround from executable PT_GNU_STACK 2020-06-05 13:45:21 -07:00
esr.h arm64: ptrauth: Introduce Armv8.3 pointer authentication enhancements 2020-09-14 12:07:02 +01:00
exception.h arm64: entry: fix non-NMI user<->kernel transitions 2020-11-30 12:11:38 +00:00
exec.h
extable.h arm64: Improve diagnostics when trapping BRK with FAULT_BRK_IMM 2020-09-18 16:35:54 +01:00
fb.h
fixmap.h arm64/fixmap: make notes of fixed_addresses more precisely 2020-08-08 19:25:04 +01:00
fpsimd.h arm64/sve: Implement a helper to load SVE registers from FPSIMD state 2020-09-21 18:06:33 +01:00
fpsimdmacros.h arm64/sve: Implement a helper to flush SVE registers 2020-09-21 18:06:33 +01:00
ftrace.h
futex.h futex: arch_futex_atomic_op_inuser() calling conventions change 2020-03-27 23:58:51 -04:00
hardirq.h arm64: Remove custom IRQ stat accounting 2020-09-17 16:37:28 +01:00
hugetlb.h arm64/hugetlb: Reserve CMA areas for gigantic pages on 16K and 64K configs 2020-07-15 13:38:03 +01:00
hw_breakpoint.h
hwcap.h Merge branch 'for-next/mte' into for-next/core 2020-10-02 12:16:11 +01:00
hyp_image.h kvm: arm64: Move nVHE hyp namespace macros to hyp_image.h 2020-09-30 08:33:52 +01:00
hypervisor.h
image.h
insn.h arm64: kprobe: add checks for ARMv8.3-PAuth combined instructions 2020-09-14 12:07:02 +01:00
io.h iommu/arm-smmu-v3: Ensure queue is read after updating prod pointer 2020-09-28 22:57:43 +01:00
irq_work.h arm64: Kill __smp_cross_call and co 2020-09-17 16:37:28 +01:00
irq.h
irqflags.h arm64: Implement arch_irqs_disabled() 2020-08-26 12:41:55 +02:00
jump_label.h
kasan.h
Kbuild KVM: arm64: Use common KVM implementation of MMU memory caches 2020-07-09 13:29:43 -04:00
kernel-pgtable.h arm64: get rid of TEXT_OFFSET 2020-09-07 15:00:52 +01:00
kexec.h Revert "arm64: kexec: make dtb_mem always enabled" 2020-01-10 16:00:50 +00:00
kgdb.h
kprobes.h arm64: kprobes: Use BRK instead of single-step when executing instructions out-of-line 2020-11-03 14:03:38 +00:00
kvm_arm.h Merge branch 'for-next/mte' into for-next/core 2020-10-02 12:16:11 +01:00
kvm_asm.h Merge branch 'kvm-arm64/hyp-pcpu' into kvmarm-master/next 2020-09-30 14:05:35 +01:00
kvm_coproc.h KVM/arm64 updates for Linux 5.9: 2020-08-09 12:58:23 -04:00
kvm_emulate.h KVM/arm64 updates for Linux 5.10 2020-10-20 08:14:25 -04:00
kvm_host.h arm64 updates for 5.10 2020-10-12 10:00:51 -07:00
kvm_hyp.h KVM: arm64: nVHE: Handle hyp panics 2020-09-15 18:39:03 +01:00
kvm_mmu.h Merge branch 'kvm-arm64/hyp-pcpu' into kvmarm-master/next 2020-09-30 14:05:35 +01:00
kvm_pgtable.h KVM: arm64: Add support for relaxing stage-2 perms in generic page-table code 2020-09-11 15:51:15 +01:00
kvm_ptrauth.h KVM: arm64: Update context references from host to hyp 2020-09-15 18:39:02 +01:00
kvm_ras.h
kvm_types.h KVM: arm64: Use common KVM implementation of MMU memory caches 2020-07-09 13:29:43 -04:00
linkage.h arm64: Don't insert a BTI instruction at inner labels 2020-06-24 14:24:29 +01:00
lse.h arm64: lse: Fix LSE atomics with LLVM 2020-02-18 18:10:49 +00:00
memory.h arm64: mm: use single quantity to represent the PA to VA translation 2020-10-15 11:14:57 +01:00
mman.h arm64: mte: Validate the PROT_MTE request via arch_validate_flags() 2020-09-04 12:46:07 +01:00
mmu_context.h arm64: mm: Pin down ASIDs for sharing mm with devices 2020-09-28 22:15:38 +01:00
mmu.h Merge branch 'for-next/ghostbusters' into for-next/core 2020-10-02 12:15:24 +01:00
mmzone.h
module.h arch: split MODULE_ARCH_VERMAGIC definitions out to <asm/vermagic.h> 2020-04-23 10:50:26 +09:00
module.lds.h kbuild: preprocess module linker script 2020-09-25 00:36:41 +09:00
mte.h arm64: mte: Enable swap of tagged pages 2020-09-04 12:46:07 +01:00
neon-intrinsics.h
neon.h
numa.h arm64/mm: return cpu_all_mask when node is NUMA_NO_NODE 2020-09-21 22:05:01 +01:00
page-def.h arm64/mm: Unify CONT_PTE_SHIFT 2020-09-11 16:33:43 +01:00
page.h arm64: mte: Add PROT_MTE support to mmap() and mprotect() 2020-09-04 12:46:07 +01:00
paravirt.h
pci.h arm64: Enable PCI write-combine resources under sysfs 2020-09-18 14:05:13 +01:00
percpu.h kvm: arm64: Remove __hyp_this_cpu_read 2020-09-30 08:33:52 +01:00
perf_event.h arm64: perf: Add support caps under sysfs 2020-09-28 14:53:45 +01:00
pgalloc.h asm-generic: pgalloc: provide generic pgd_free() 2020-08-07 11:33:26 -07:00
pgtable-hwdef.h ARM: 2020-10-23 11:17:56 -07:00
pgtable-prot.h ARM: 2020-10-23 11:17:56 -07:00
pgtable-types.h arm64: add support for folded p4d page tables 2020-06-04 19:06:21 -07:00
pgtable.h arm64: pgtable: Ensure dirty bit is preserved across pte_wrprotect() 2020-11-23 16:13:18 +00:00
pointer_auth.h arm64: sync kernel APIAKey when installing 2020-04-21 15:52:56 +01:00
preempt.h sched/rt, arm64: Use CONFIG_PREEMPTION 2019-12-08 14:37:32 +01:00
probes.h arm64/fpsimd: add <asm/insn.h> to <asm/kprobes.h> to fix fpsimd build 2020-11-23 10:59:00 +00:00
proc-fns.h arm64: mm: convert cpu_do_switch_mm() to C 2020-02-27 14:30:50 +00:00
processor.h Merge branch 'for-next/mte' into for-next/core 2020-10-02 12:16:11 +01:00
ptdump.h arm64: mm: convert mm/dump.c to use walk_page_range() 2020-02-04 03:05:25 +00:00
ptrace.h arm64: ptrace: prepare for EL1 irq/rcu tracking 2020-11-30 12:11:38 +00:00
pvclock-abi.h
scs.h scs: Move scs_overflow_check() out of architecture code 2020-05-18 17:47:40 +01:00
sdei.h
seccomp.h
sections.h arm64: Remove __exception_text_start and __exception_text_end from asm/section.h 2020-01-08 17:30:19 +00:00
shmparam.h
signal32.h
simd.h arm64: fpsimd: Make sure SVE setup is complete before SIMD is used 2020-01-14 17:11:21 +00:00
smp_plat.h
smp.h arm64: Remove custom IRQ stat accounting 2020-09-17 16:37:28 +01:00
sparsemem.h
spectre.h arm64: Rewrite Spectre-v4 mitigation code 2020-09-29 16:08:16 +01:00
spinlock_types.h
spinlock.h arm64/spinlock: fix a -Wunused-function warning 2020-02-10 11:29:24 +00:00
stack_pointer.h
stackprotector.h arm64: initialize ptrauth keys for kernel booting task 2020-03-18 09:50:20 +00:00
stacktrace.h arm64: stacktrace: Make stack walk callback consistent with generic code 2020-09-18 14:24:16 +01:00
stage2_pgtable.h KVM: arm64: Remove unused page-table code 2020-09-11 15:51:15 +01:00
stat.h
string.h
suspend.h arm64: Preserve register x18 when CPU is suspended 2020-05-15 16:35:50 +01:00
sync_bitops.h
syscall_wrapper.h
syscall.h arm64: compat: Ensure upper 32 bits of x0 are zero on syscall return 2020-07-16 11:41:31 +01:00
sysreg.h arm64: Add workaround for Arm Cortex-A77 erratum 1508412 2020-10-29 12:56:01 +00:00
system_misc.h
thread_info.h arm64: mte: Handle synchronous and asynchronous tag check faults 2020-09-04 12:46:06 +01:00
timex.h
tlb.h arm64: tlb: Set the TTL field in flush_tlb_range 2020-07-07 11:23:47 +01:00
tlbflush.h arm64: use a common .arch preamble for inline assembly 2020-08-28 11:15:15 +01:00
topology.h cpufreq,arm,arm64: restructure definitions of arch_set_freq_scale() 2020-10-08 17:17:27 +02:00
traps.h arm64: traps: Allow force_signal_inject to pass esr error code 2020-09-14 12:07:02 +01:00
uaccess.h uaccess: remove segment_eq 2020-08-12 10:57:58 -07:00
unistd32.h mm/madvise: introduce process_madvise() syscall: an external memory hinting API 2020-10-18 09:27:10 -07:00
unistd.h mm/madvise: introduce process_madvise() syscall: an external memory hinting API 2020-10-18 09:27:10 -07:00
uprobes.h
vdso.h arm64/vdso: Add time namespace page 2020-07-24 13:15:20 +01:00
vermagic.h arch: split MODULE_ARCH_VERMAGIC definitions out to <asm/vermagic.h> 2020-04-23 10:50:26 +09:00
virt.h KVM: arm64: Use build-time defines in has_vhe() 2020-07-05 18:38:01 +01:00
vmalloc.h mm/vmalloc: Add empty <asm/vmalloc.h> headers and use them from <linux/vmalloc.h> 2019-12-10 10:12:55 +01:00
vmap_stack.h mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
word-at-a-time.h
xor.h