linux/arch/x86/kernel
Thomas Gleixner 59f5ede3bc x86/fpu: Prevent FPU state corruption
The FPU usage related to task FPU management is either protected by
disabling interrupts (switch_to, return to user) or via fpregs_lock() which
is a wrapper around local_bh_disable(). When kernel code wants to use the
FPU then it has to check whether it is possible by calling irq_fpu_usable().

But the condition in irq_fpu_usable() is wrong. It allows FPU to be used
when:

   !in_interrupt() || interrupted_user_mode() || interrupted_kernel_fpu_idle()

The latter is checking whether some other context already uses FPU in the
kernel, but if that's not the case then it allows FPU to be used
unconditionally even if the calling context interrupted a fpregs_lock()
critical region. If that happens then the FPU state of the interrupted
context becomes corrupted.

Allow in kernel FPU usage only when no other context has in kernel FPU
usage and either the calling context is not hard interrupt context or the
hard interrupt did not interrupt a local bottomhalf disabled region.

It's hard to find a proper Fixes tag as the condition was broken in one way
or the other for a very long time and the eager/lazy FPU changes caused a
lot of churn. Picked something remotely connected from the history.

This survived undetected for quite some time as FPU usage in interrupt
context is rare, but the recent changes to the random code unearthed it at
least on a kernel which had FPU debugging enabled. There is probably a
higher rate of silent corruption as not all issues can be detected by the
FPU debugging code. This will be addressed in a subsequent change.

Fixes: 5d2bd7009f ("x86, fpu: decouple non-lazy/eager fpu restore from xsave")
Reported-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Borislav Petkov <bp@suse.de>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20220501193102.588689270@linutronix.de
2022-05-05 02:40:19 +02:00
..
acpi Power management updates for 5.18-rc1 2022-03-21 14:26:28 -07:00
apic bitmap patches for 5.17-rc1 2022-01-23 06:20:44 +02:00
cpu x86/cpu: Load microcode during restore_processor_state() 2022-04-19 19:37:05 +02:00
fpu x86/fpu: Prevent FPU state corruption 2022-05-05 02:40:19 +02:00
kprobes x86,kprobes: Fix optprobe trampoline to generate complete pt_regs 2022-03-28 19:38:51 -07:00
.gitignore
alternative.c Add support for Intel CET-IBT, available since Tigerlake (11th gen), which is a 2022-03-27 10:17:23 -07:00
amd_gart_64.c x86/amd_gart: don't set failed sg dma_address to DMA_MAPPING_ERROR 2021-08-09 17:13:06 +02:00
amd_nb.c hwmon updates for v5.17 2022-01-11 10:25:36 -08:00
aperture_64.c proc/vmcore: convert oldmem_pfn_is_ram callback to more generic vmcore callbacks 2021-11-09 10:02:48 -08:00
apm_32.c x86/ibt: Disable IBT around firmware 2022-03-15 10:32:40 +01:00
asm-offsets_32.c x86/stackprotector/32: Make the canary into a regular percpu variable 2021-03-08 13:19:05 +01:00
asm-offsets_64.c
asm-offsets.c um, x86: bury crypto_tfm_ctx_offset 2021-12-21 21:31:35 +01:00
audit_64.c audit: add support for the openat2 syscall 2021-10-01 16:52:48 -04:00
bootflag.c
check.c
cpuid.c
crash_core_32.c
crash_core_64.c
crash_dump_32.c
crash_dump_64.c mm/vmalloc: fix spinning drain_vmap_work after reading from /proc/vmcore 2022-04-15 14:49:56 -07:00
crash.c x86/crash: Remove crash_reserve_low_1M() 2021-06-07 12:14:45 +02:00
devicetree.c Devicetree updates for v5.16: 2021-11-02 22:22:13 -07:00
doublefault_32.c exit/doublefault: Remove apparently bogus comment about rewind_stack_do_exit 2021-10-20 13:09:43 -05:00
dumpstack_32.c
dumpstack_64.c x86/mm/64: Improve stack overflow warnings 2021-09-21 13:57:43 +02:00
dumpstack.c x86: remove __range_not_ok() 2022-02-25 09:36:05 +01:00
e820.c x86/boot: Fix memremap of setup_indirect structures 2022-03-09 12:49:44 +01:00
early_printk.c usb: early: xhci-dbc: Remove duplicate keep parsing 2022-03-15 18:20:34 +01:00
early-quirks.c Merge drm/drm-next into drm-intel-next 2022-01-31 13:19:33 -05:00
ebda.c
eisa.c
espfix_64.c
ftrace_32.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
ftrace_64.S x86/ibt,ftrace: Annotate ftrace code patching 2022-03-15 10:32:41 +01:00
ftrace.c x86/ibt,ftrace: Make function-graph play nice 2022-03-15 10:32:37 +01:00
head32.c
head64.c x86/head64: Add missing __head annotation to sme_postprocess_startup() 2022-02-12 11:37:09 +01:00
head_32.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
head_64.S x86/ibt,sev: Annotations 2022-03-15 10:32:41 +01:00
hpet.c rtc: Check return value from mc146818_get_time() 2021-12-16 21:50:06 +01:00
hw_breakpoint.c
i8237.c
i8253.c
i8259.c x86: Avoid magic number with ELCR register accesses 2021-08-10 23:31:43 +02:00
idt.c x86/ibt: Add IBT feature, MSR and #CP handling 2022-03-15 10:32:39 +01:00
io_delay.c
ioport.c
irq_32.c x86/softirq: Disable softirq stacks on PREEMPT_RT 2021-09-27 12:28:32 +02:00
irq_64.c
irq_work.c
irq.c x86/irq: Ensure PI wakeup handler is unregistered before module unload 2021-10-22 12:45:35 -04:00
irqflags.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
irqinit.c
itmt.c x86/sched: Decrease further the priorities of SMT siblings 2021-10-05 15:51:59 +02:00
jailhouse.c
jump_label.c jump_labels: Mark __jump_label_transform() as __always_inlined to work around aggressive compiler un-inlining 2021-07-13 06:32:05 +02:00
kdebugfs.c x86/boot: Fix memremap of setup_indirect structures 2022-03-09 12:49:44 +01:00
kexec-bzimage64.c x86: Use ELF fields defined in 'struct kimage' 2021-03-08 12:06:29 -07:00
kgdb.c x86: Fix various typos in comments 2021-03-18 15:31:53 +01:00
ksysfs.c x86/boot: Fix memremap of setup_indirect structures 2022-03-09 12:49:44 +01:00
kvm.c x86/kvm: Preserve BSP MSR_KVM_POLL_CONTROL across suspend/resume 2022-04-21 13:16:14 -04:00
kvmclock.c x86/kvmclock: Fix Hyper-V Isolated VM's boot issue when vCPUs > 64 2022-02-25 10:09:34 -05:00
ldt.c memcg: enable accounting for ldt_struct objects 2021-09-03 09:58:13 -07:00
machine_kexec_32.c x86/kexec: Set_[gi]dt() -> native_[gi]dt_invalidate() in machine_kexec_*.c 2021-05-21 12:36:45 +02:00
machine_kexec_64.c x86/ibt,kexec: Disable CET on kexec 2022-03-15 10:32:39 +01:00
Makefile x86,rethook,kprobes: Replace kretprobe with rethook on x86 2022-03-28 19:38:51 -07:00
mmconf-fam10h_64.c x86/msr: Rename MSR_K8_SYSCFG to MSR_AMD64_SYSCFG 2021-05-10 07:51:38 +02:00
module.c Add support for Intel CET-IBT, available since Tigerlake (11th gen), which is a 2022-03-27 10:17:23 -07:00
mpparse.c x86: Avoid magic number with ELCR register accesses 2021-08-10 23:31:43 +02:00
msr.c
nmi_selftest.c
nmi.c x86/nmi: Remove the 'strange power saving mode' hint from unknown NMI handler 2022-03-16 11:02:41 +01:00
paravirt-spinlocks.c x86/paravirt: Add new features for paravirt patching 2021-03-11 19:51:49 +01:00
paravirt.c x86/ibt,paravirt: Sprinkle ENDBR 2022-03-15 10:32:36 +01:00
pci-dma.c
pci-iommu_table.c
pci-swiotlb.c x86/sme: Replace occurrences of sme_active() with cc_platform_has() 2021-10-04 11:46:46 +02:00
pcspeaker.c
perf_regs.c
platform-quirks.c
pmem.c
probe_roms.c x86/pci/probe_roms: Use to_pci_driver() instead of pci_dev->driver 2021-10-18 09:20:15 -05:00
process_32.c x86/fpu: Move context switch and exit to user inlines into sched.h 2021-10-20 15:27:27 +02:00
process_64.c x86: Remove toolchain check for X32 ABI capability 2022-03-15 10:32:48 +01:00
process.c Merge branch 'x86/cpu' into x86/core, to resolve conflicts 2022-03-15 12:52:51 +01:00
process.h x86: Snapshot thread flags 2021-12-01 00:06:43 +01:00
ptrace.c ptrace: Cleanups for v5.18 2022-03-28 17:29:53 -07:00
pvclock.c x86: Fix various typos in comments 2021-03-18 15:31:53 +01:00
quirks.c
reboot_fixups_32.c
reboot.c x86/mm: Flush global TLB when switching to trampoline page-table 2021-12-06 09:54:10 +01:00
relocate_kernel_32.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
relocate_kernel_64.S x86/ibt: Annotate text references 2022-03-15 10:32:40 +01:00
resource.c x86/PCI: revert "Ignore E820 reservations for bridge windows on newer systems" 2022-02-09 19:42:58 +01:00
rethook.c x86,rethook: Fix arch_rethook_trampoline() to generate a complete pt_regs 2022-03-28 19:38:51 -07:00
rtc.c
setup_percpu.c mm: percpu: add generic pcpu_populate_pte() function 2022-01-20 08:52:52 +02:00
setup.c x86/setup: use IS_ENABLED(CONFIG_KEXEC_CORE) instead of #ifdef 2022-03-23 19:00:34 -07:00
sev_verify_cbit.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sev-shared.c x86/sev: Shorten GHCB terminate macro names 2021-11-15 20:31:16 +01:00
sev.c - Add support for decoding instructions which do MMIO accesses in order 2022-01-10 10:00:03 -08:00
signal_compat.c signal: Verify the alignment and size of siginfo_t 2021-07-23 13:15:31 -05:00
signal.c tracehook: Remove tracehook.h 2022-03-10 16:51:51 -06:00
smp.c x86: Fix various typos in comments, take #2 2021-03-21 23:50:28 +01:00
smpboot.c x86/ACPI: CPPC: Move init_freq_invariance_cppc() into x86 CPPC 2022-03-08 19:16:43 +01:00
stacktrace.c x86: remove __range_not_ok() 2022-02-25 09:36:05 +01:00
static_call.c x86,static_call: Fix __static_call_return0 for i386 2022-04-05 09:59:37 +02:00
step.c
sys_ia32.c
sys_x86_64.c
tboot.c x86/boot/tboot: Avoid Wstringop-overread-warning 2021-03-23 00:16:13 +01:00
time.c
tls.c x86/stackprotector/32: Make the canary into a regular percpu variable 2021-03-08 13:19:05 +01:00
tls.h
topology.c drivers/base/node: consolidate node device subsystem initialization in node_dev_init() 2022-03-22 15:57:10 -07:00
trace_clock.c
trace.c trace/osnoise: Fix an ifdef comment 2021-10-25 23:02:36 -04:00
tracepoint.c
traps.c Reenable ENQCMD/PASID support: 2022-03-21 12:28:13 -07:00
tsc_msr.c
tsc_sync.c x86/tsc: Add a timer to make sure TSC_adjust is always checked 2021-12-02 00:40:35 +01:00
tsc.c x86/tsc: Be consistent about use_tsc_delay() 2022-03-15 18:20:33 +01:00
umip.c x86/umip: Downgrade warning messages to debug loglevel 2021-09-25 13:23:28 +02:00
unwind_frame.c x86/unwind: Recover kretprobe trampoline entry 2021-09-30 21:24:07 -04:00
unwind_guess.c x86/unwind: Recover kretprobe trampoline entry 2021-09-30 21:24:07 -04:00
unwind_orc.c x86/unwind/orc: Recheck address range after stack info was updated 2022-04-19 21:58:46 +02:00
uprobes.c x86/uprobes: Convert to insn_decode() 2021-03-15 12:05:03 +01:00
verify_cpu.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
vm86_32.c signal: Replace force_fatal_sig with force_exit_sig when in doubt 2021-11-19 09:15:58 -06:00
vmlinux.lds.S objtool: Find unused ENDBR instructions 2022-03-15 10:32:47 +01:00
vsmp_64.c
x86_init.c x86/mm/cpa: Generalize __set_memory_enc_pgtable() 2022-02-23 19:14:29 +01:00