linux/arch/x86/kernel
Arvind Sankar aa5cacdc29 x86/asm: Replace __force_order with a memory clobber
The CRn accessor functions use __force_order as a dummy operand to
prevent the compiler from reordering CRn reads/writes with respect to
each other.

The fact that the asm is volatile should be enough to prevent this:
volatile asm statements should be executed in program order. However GCC
4.9.x and 5.x have a bug that might result in reordering. This was fixed
in 8.1, 7.3 and 6.5. Versions prior to these, including 5.x and 4.9.x,
may reorder volatile asm statements with respect to each other.

There are some issues with __force_order as implemented:
- It is used only as an input operand for the write functions, and hence
  doesn't do anything additional to prevent reordering writes.
- It allows memory accesses to be cached/reordered across write
  functions, but CRn writes affect the semantics of memory accesses, so
  this could be dangerous.
- __force_order is not actually defined in the kernel proper, but the
  LLVM toolchain can in some cases require a definition: LLVM (as well
  as GCC 4.9) requires it for PIE code, which is why the compressed
  kernel has a definition, but also the clang integrated assembler may
  consider the address of __force_order to be significant, resulting in
  a reference that requires a definition.

Fix this by:
- Using a memory clobber for the write functions to additionally prevent
  caching/reordering memory accesses across CRn writes.
- Using a dummy input operand with an arbitrary constant address for the
  read functions, instead of a global variable. This will prevent reads
  from being reordered across writes, while allowing memory loads to be
  cached/reordered across CRn reads, which should be safe.

Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu>
Signed-off-by: Borislav Petkov <bp@suse.de>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
Tested-by: Nathan Chancellor <natechancellor@gmail.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82602
Link: https://lore.kernel.org/lkml/20200527135329.1172644-1-arnd@arndb.de/
Link: https://lkml.kernel.org/r/20200902232152.3709896-1-nivedita@alum.mit.edu
2020-10-01 10:31:48 +02:00
..
acpi mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
apic Three interrupt related fixes for X86: 2020-08-30 12:01:23 -07:00
cpu x86/asm: Replace __force_order with a memory clobber 2020-10-01 10:31:48 +02:00
fpu Misc fixes and small updates all around the place: 2020-08-15 10:38:03 -07:00
kprobes Merge branch 'perf/vlbr' 2020-07-02 15:51:48 +02:00
.gitignore .gitignore: add SPDX License Identifier 2020-03-25 11:50:48 +01:00
alternative.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
amd_gart_64.c docs: fix references for DMA*.txt files 2020-06-26 10:01:32 -06:00
amd_nb.c x86/amd_nb: Add AMD family 17h model 60h PCI IDs 2020-05-22 18:24:40 +02:00
apb_timer.c x86/apb_timer: Drop unused TSC calibration 2020-05-27 13:05:59 +02:00
aperture_64.c
apm_32.c
asm-offsets_32.c x86 entry code updates: 2020-03-30 19:14:28 -07:00
asm-offsets_64.c x86/entry: Remove DBn stacks 2020-06-11 15:15:23 +02:00
asm-offsets.c efi/x86: Avoid using code32_start 2020-03-08 09:58:17 +01:00
audit_64.c x86/audit: Fix a -Wmissing-prototypes warning for ia32_classify_syscall() 2020-05-19 18:03:07 +02:00
bootflag.c
check.c
cpuid.c
crash_core_32.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
crash_core_64.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
crash_dump_32.c
crash_dump_64.c fs/core/vmcore: Move sev_active() reference to x86 arch code 2019-08-09 22:52:10 +10:00
crash.c x86/crash: Correct the address boundary of function parameters 2020-08-07 01:32:00 +02:00
devicetree.c x86/headers: Remove APIC headers from <asm/smp.h> 2020-08-06 16:13:09 +02:00
doublefault_32.c x86/entry: Convert double fault exception to IDTENTRY_DF 2020-06-11 15:15:03 +02:00
dumpstack_32.c x86/32: Remove CONFIG_DOUBLEFAULT 2020-04-14 14:24:05 +02:00
dumpstack_64.c x86/entry: Remove DBn stacks 2020-06-11 15:15:23 +02:00
dumpstack.c Improve x86 debuggability: print registers with the same log level as the backtrace. 2020-08-03 17:00:00 -07:00
e820.c Rebase locking/kcsan to locking/urgent 2020-06-11 20:02:46 +02:00
early_printk.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
early-quirks.c x86/gpu: add RKL stolen memory support 2020-05-20 08:35:22 -07:00
ebda.c
eisa.c
espfix_64.c mm: introduce include/linux/pgtable.h 2020-06-09 09:39:13 -07:00
ftrace_32.S x86: Change {JMP,CALL}_NOSPEC argument 2020-04-30 20:14:34 +02:00
ftrace_64.S x86/ftrace: Do not jump to direct code in created trampolines 2020-06-29 11:42:48 -04:00
ftrace.c x86/ftrace: Do not jump to direct code in created trampolines 2020-06-29 11:42:48 -04:00
head32.c
head64.c mm: reorder includes after introduction of linux/pgtable.h 2020-06-09 09:39:13 -07:00
head_32.S x86/xen: remove 32-bit Xen PV guest support 2020-08-11 08:26:48 +02:00
head_64.S x86/entry: Remove the apic/BUILD interrupt leftovers 2020-06-11 15:15:16 +02:00
hpet.c remove ioremap_nocache and devm_ioremap_nocache 2020-01-06 09:45:59 +01:00
hw_breakpoint.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
i8237.c
i8253.c
i8259.c x86/i8259: Use printk_deferred() to prevent deadlock 2020-07-29 16:27:16 +02:00
idt.c x86/idt: Make idt_descr static 2020-06-20 11:47:35 +02:00
ima_arch.c EFI updates for v5.7: 2020-02-26 15:21:22 +01:00
io_delay.c
ioport.c x86/ioperm: Prevent a memory leak when fork fails 2020-05-28 21:36:20 +02:00
irq_32.c x86/irq: Rework handle_irq() for 64-bit 2020-06-11 15:15:12 +02:00
irq_64.c x86/entry/64: Move do_softirq_own_stack() to C 2020-06-11 15:15:07 +02:00
irq_work.c x86/entry: Convert various system vectors 2020-06-11 15:15:14 +02:00
irq.c x86/entry: Convert KVM vectors to IDTENTRY_SYSVEC* 2020-06-11 15:15:15 +02:00
irqflags.S x86/asm: Change all ENTRY+ENDPROC to SYM_FUNC_* 2019-10-18 11:58:33 +02:00
irqinit.c x86/headers: Remove APIC headers from <asm/smp.h> 2020-08-06 16:13:09 +02:00
itmt.c sysctl: pass kernel pointers to ->proc_handler 2020-04-27 02:07:40 -04:00
jailhouse.c locking/seqlock, headers: Untangle the spaghetti monster 2020-08-06 16:13:13 +02:00
jump_label.c x86/jump_label: Move 'inline' keyword placement 2020-03-27 11:05:41 +01:00
kdebugfs.c x86/boot: Introduce setup_indirect 2019-11-12 16:21:15 +01:00
kexec-bzimage64.c x86/efi: Remove references to no-longer-used efi_have_uv1_memmap() 2020-07-17 16:47:47 +02:00
kgdb.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
ksysfs.c x86/boot: Introduce setup_indirect 2019-11-12 16:21:15 +01:00
kvm.c s390: implement diag318 2020-08-06 12:59:31 -07:00
kvmclock.c x86/vdso: Use generic VDSO clock mode storage 2020-02-17 14:40:23 +01:00
ldt.c x86/ldt: use "pr_info_once()" instead of open-coding it badly 2020-07-05 12:50:20 -07:00
machine_kexec_32.c mm: don't include asm/pgtable.h if linux/mm.h is already included 2020-06-09 09:39:13 -07:00
machine_kexec_64.c mm: don't include asm/pgtable.h if linux/mm.h is already included 2020-06-09 09:39:13 -07:00
Makefile Rebase locking/kcsan to locking/urgent 2020-06-11 20:02:46 +02:00
mmconf-fam10h_64.c
module.c mm: don't include asm/pgtable.h if linux/mm.h is already included 2020-06-09 09:39:13 -07:00
mpparse.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
msr.c x86/msr: Filter MSR writes 2020-06-25 10:39:02 +02:00
nmi_selftest.c
nmi.c x86/entry: Fix NMI vs IRQ state tracking 2020-07-10 12:00:01 +02:00
paravirt_patch.c
paravirt-spinlocks.c
paravirt.c x86/ioperm: Fix io bitmap invalidation on Xen PV 2020-07-18 12:31:49 +02:00
pci-dma.c dma-mapping updates for 5.5-rc1 2019-11-28 11:16:43 -08:00
pci-iommu_table.c
pci-swiotlb.c dma-mapping: fix filename references 2019-09-03 08:36:30 +02:00
pcspeaker.c
perf_regs.c
platform-quirks.c
pmem.c
probe_roms.c maccess: make get_kernel_nofault() check for minimal type compatibility 2020-06-18 12:10:37 -07:00
process_32.c x86/dumpstack: Add log_lvl to __show_regs() 2020-07-22 23:56:53 +02:00
process_64.c x86/fsgsbase/64: Fix NULL deref in 86_fsgsbase_read_task 2020-08-14 13:30:18 -07:00
process.c cpuidle: Move trace_cpu_idle() into generic code 2020-08-26 12:41:54 +02:00
process.h x86: Use the correct SPDX License Identifier in headers 2019-10-01 20:31:35 +02:00
ptrace.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
pvclock.c x86/vdso: Use generic VDSO clock mode storage 2020-02-17 14:40:23 +01:00
quirks.c treewide: Remove uninitialized_var() usage 2020-07-16 12:35:15 -07:00
reboot_fixups_32.c
reboot.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
relocate_kernel_32.S x86/asm: Annotate relocate_kernel_{32,64}.c 2019-10-18 09:53:19 +02:00
relocate_kernel_64.S x86/kexec: Make relocate_kernel_64.S objtool clean 2020-03-25 18:28:28 +01:00
resource.c
rtc.c
setup_percpu.c x86/mm: remove vmalloc faulting 2020-06-02 10:59:12 -07:00
setup.c A set of locking fixes and updates: 2020-08-10 19:07:44 -07:00
signal_compat.c
signal.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
smp.c x86/entry: Convert reschedule interrupt to IDTENTRY_SYSVEC_SIMPLE 2020-06-11 15:15:16 +02:00
smpboot.c x86/hotplug: Silence APIC only after all interrupts are migrated 2020-08-27 09:29:23 +02:00
stacktrace.c x86/stacktrace: Fix reliable check for empty user task stacks 2020-07-22 23:47:47 +02:00
step.c
sys_ia32.c fork: fold legacy_clone_args_valid() into _do_fork() 2020-06-22 14:38:38 +02:00
sys_x86_64.c x86: Remove unneeded includes 2020-03-21 16:03:25 +01:00
sysfb_efi.c x86/sysfb_efi: Add quirks for some devices with swapped width and height 2019-07-22 10:47:11 +02:00
sysfb_simplefb.c x86/sysfb: Fix check for bad VRAM size 2020-01-20 10:57:53 +01:00
sysfb.c
tboot.c mmap locking API: add MMAP_LOCK_INITIALIZER 2020-06-09 09:39:14 -07:00
time.c A set of fixes and updates for x86: 2020-06-11 15:54:31 -07:00
tls.c x86: switch to ->regset_get() 2020-07-27 14:31:07 -04:00
tls.h x86: switch to ->regset_get() 2020-07-27 14:31:07 -04:00
topology.c x86/headers: Remove APIC headers from <asm/smp.h> 2020-08-06 16:13:09 +02:00
trace_clock.c
tracepoint.c x86/entry: Convert reschedule interrupt to IDTENTRY_SYSVEC_SIMPLE 2020-06-11 15:15:16 +02:00
traps.c mm: remove unneeded includes of <asm/pgalloc.h> 2020-08-07 11:33:26 -07:00
tsc_msr.c Misc fixes and small updates all around the place: 2020-08-15 10:38:03 -07:00
tsc_sync.c x86: Fix a handful of typos 2020-02-16 20:58:06 +01:00
tsc.c x86/tsc: Add tsc_early_khz command line parameter 2020-05-21 23:07:00 +02:00
umip.c x86/umip: Make umip_insns static 2020-04-15 11:13:12 +02:00
unwind_frame.c fork-v5.9 2020-08-04 14:47:45 -07:00
unwind_guess.c
unwind_orc.c x86/unwind/orc: Fix ORC for newly forked tasks 2020-07-22 23:47:47 +02:00
uprobes.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
verify_cpu.S x86/asm: Annotate local pseudo-functions 2019-10-18 10:04:04 +02:00
vm86_32.c mmap locking API: use coccinelle to convert mmap_sem rwsem call sites 2020-06-09 09:39:14 -07:00
vmlinux.lds.S x86, vmlinux.lds: Page-align end of ..page_aligned sections 2020-07-22 09:38:37 +02:00
vsmp_64.c
x86_init.c x86/kvm: Handle async page faults directly through do_page_fault() 2020-05-19 15:53:57 +02:00