linux/arch/x86/kernel
Linus Torvalds b6e6cc1f78 Fix kCFI/FineIBT weaknesses
The primary bug Alyssa noticed was that with FineIBT enabled function
 prologues have a spurious ENDBR instruction:
 
   __cfi_foo:
 	endbr64
 	subl	$hash, %r10d
 	jz	1f
 	ud2
 	nop
   1:
   foo:
 	endbr64 <--- *sadface*
 
 This means that any indirect call that fails to target the __cfi symbol
 and instead targets (the regular old) foo+0, will succeed due to that
 second ENDBR.
 
 Fixing this lead to the discovery of a single indirect call that was
 still doing this: ret_from_fork(), since that's an assembly stub the
 compmiler would not generate the proper kCFI indirect call magic and it
 would not get patched.
 
 Brian came up with the most comprehensive fix -- convert the thing to C
 with only a very thin asm wrapper. This ensures the kernel thread
 boostrap is a proper kCFI call.
 
 While discussing all this, Kees noted that kCFI hashes could/should be
 poisoned to seal all functions whose address is never taken, further
 limiting the valid kCFI targets -- much like we already do for IBT.
 
 So what was a 'simple' observation and fix cascaded into a bunch of
 inter-related CFI infrastructure fixes.
 -----BEGIN PGP SIGNATURE-----
 
 iQJJBAABCgAzFiEEv3OU3/byMaA0LqWJdkfhpEvA5LoFAmSxr64VHHBldGVyekBp
 bmZyYWRlYWQub3JnAAoJEHZH4aRLwOS6L7kQAIjDWbxqVtmiBiz+IBcWcsxt7BXX
 pRBaSe/eBp3KLhqgzYUY0mXIi0ua7y3CBtW4SdQUSPsAKtCgBUuq2JjQWToRghjN
 4ndCky4oxb9z8ADr/R/qfU8ZpSOwoX3kgBHqyjcQ0fQsg/DFKs3sWKqluwT0PtvU
 vLYAw2QKSv56NG/u3CujWPdcIWgzJ+M3214xuqIWCTwEcqdP+xkXmQstkXkyPQ6d
 XE0iG/wo9uiX4icfsRVp8JL0TkzNqGJfgr9Mv1rBKT4wbT64zKI6RyMJVlUS0yrk
 1jeDgNbVfx4ZpvtHmTsQn1jogWI3pqGkqoPwHqJSFg42Eer5OSodH/uVd3HK/0tD
 1nlhCfue6zc4smu480064s3fWAE7kC6ySdmijQXOJo3YWVGdagxVp/CSE4Ek0TFq
 y+CltNEA6bthKImWg8GFWxS8bMnuZv2joJ8yhgfpnG5sppVOYs2HJ3ipIks9sZjO
 o65auDeOkGg1+NhgDx+2uay6/fbxTNjbAyjV4HttkN70SO5kTTT4zWyh2PLwXaTy
 wv0B4i0laxTRU7boIA4nFJAKz5xKfyh9e2idxbmPlrV5FY4mEPA2oLeWsn8cS4VG
 0SWJ30ky7C4r7VWd9DWhGcCRcrlCvCM8LdjwzImZHXRQ2KweEuGMmrXYtHCrTRZn
 IMijS/9q653h9ws7
 =RhPI
 -----END PGP SIGNATURE-----

Merge tag 'x86_urgent_for_6.5_rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull x86 CFI fixes from Peter Zijlstra:
 "Fix kCFI/FineIBT weaknesses

  The primary bug Alyssa noticed was that with FineIBT enabled function
  prologues have a spurious ENDBR instruction:

    __cfi_foo:
	endbr64
	subl	$hash, %r10d
	jz	1f
	ud2
	nop
    1:
    foo:
	endbr64 <--- *sadface*

  This means that any indirect call that fails to target the __cfi
  symbol and instead targets (the regular old) foo+0, will succeed due
  to that second ENDBR.

  Fixing this led to the discovery of a single indirect call that was
  still doing this: ret_from_fork(). Since that's an assembly stub the
  compiler would not generate the proper kCFI indirect call magic and it
  would not get patched.

  Brian came up with the most comprehensive fix -- convert the thing to
  C with only a very thin asm wrapper. This ensures the kernel thread
  boostrap is a proper kCFI call.

  While discussing all this, Kees noted that kCFI hashes could/should be
  poisoned to seal all functions whose address is never taken, further
  limiting the valid kCFI targets -- much like we already do for IBT.

  So what was a 'simple' observation and fix cascaded into a bunch of
  inter-related CFI infrastructure fixes"

* tag 'x86_urgent_for_6.5_rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86/cfi: Only define poison_cfi() if CONFIG_X86_KERNEL_IBT=y
  x86/fineibt: Poison ENDBR at +0
  x86: Rewrite ret_from_fork() in C
  x86/32: Remove schedule_tail_wrapper()
  x86/cfi: Extend ENDBR sealing to kCFI
  x86/alternative: Rename apply_ibt_endbr()
  x86/cfi: Extend {JMP,CAKK}_NOSPEC comment
2023-07-14 20:19:25 -07:00
..
acpi - Address -Wmissing-prototype warnings 2023-06-26 16:43:54 -07:00
apic Add UV platform support for sub-NUMA clustering 2023-06-26 16:26:44 -07:00
cpu - Yosry Ahmed brought back some cgroup v1 stats in OOM logs. 2023-06-28 10:28:11 -07:00
fpu Updates for the x86 boot process: 2023-06-26 13:39:10 -07:00
kprobes probes updates for 6.3: 2023-02-23 13:03:08 -08:00
.gitignore
alternative.c x86/cfi: Only define poison_cfi() if CONFIG_X86_KERNEL_IBT=y 2023-07-11 10:17:55 +02:00
amd_gart_64.c x86/mm: Remove P*D_PAGE_MASK and P*D_PAGE_SIZE macros 2022-12-15 10:37:27 -08:00
amd_nb.c x86/amd_nb: Re-sort and re-indent PCI defines 2023-06-05 12:26:54 +02:00
aperture_64.c x86: Fix various duplicate-word comment typos 2022-08-15 19:17:52 +02:00
apm_32.c efi: x86: Wire up IBT annotation in memory attributes table 2023-02-09 19:30:54 +01:00
asm-offsets_32.c
asm-offsets_64.c x86: Fixup asm-offsets duplicate 2022-10-17 16:41:06 +02:00
asm-offsets.c x86/smpboot: Remove initial_stack on 64-bit 2023-03-21 13:35:53 +01:00
audit_64.c
bootflag.c
callthunks.c objtool changes for v6.5: 2023-06-27 15:05:41 -07:00
cfi.c x86: Add support for CONFIG_CFI_CLANG 2022-09-26 10:13:16 -07:00
check.c
cpuid.c driver core: class: remove module * from class_create() 2023-03-17 15:16:33 +01:00
crash_core_32.c
crash_core_64.c
crash_dump_32.c vmcore: convert copy_oldmem_page() to take an iov_iter 2022-04-29 14:37:59 -07:00
crash_dump_64.c use less confusing names for iov_iter direction initializers 2022-11-25 13:01:55 -05:00
crash.c x86/crash: Disable virt in core NMI crash handler to avoid double shootdown 2023-01-24 10:05:21 -08:00
devicetree.c x86/of: Add support for boot time interrupt delivery mode configuration 2022-12-02 14:57:14 +01:00
doublefault_32.c x86: Avoid missing-prototype warnings for doublefault code 2023-05-18 11:56:18 -07:00
dumpstack_32.c x86/percpu: Move irq_stack variables next to current_task 2022-10-17 16:41:05 +02:00
dumpstack_64.c x86/percpu: Move irq_stack variables next to current_task 2022-10-17 16:41:05 +02:00
dumpstack.c x86/show_trace_log_lvl: Ensure stack pointer is aligned, again 2023-05-16 06:31:04 -07:00
e820.c x86/setup: Move duplicate boot_cpu_data definition out of the ifdeffery 2023-01-11 12:45:16 +01:00
early_printk.c x86/earlyprintk: Clean up pciserial 2022-08-29 12:19:25 +02:00
early-quirks.c drm/i915/rpl-p: Add PCI IDs 2022-04-19 17:14:09 -07:00
ebda.c
eisa.c
espfix_64.c x86/espfix: Use get_random_long() rather than archrandom 2022-10-31 20:12:50 +01:00
ftrace_32.S x86/ftrace: Enable HAVE_FUNCTION_GRAPH_RETVAL 2023-06-20 18:38:38 -04:00
ftrace_64.S x86/ftrace: Enable HAVE_FUNCTION_GRAPH_RETVAL 2023-06-20 18:38:38 -04:00
ftrace.c x86/ftrace: Remove unsued extern declaration ftrace_regs_caller_ret() 2023-07-10 21:38:13 -04:00
head32.c x86: Add dummy prototype for mk_early_pgtbl_32() 2023-05-18 11:56:16 -07:00
head64.c x86/head: Mark *_start_kernel() __noreturn 2023-04-14 17:31:24 +02:00
head_32.S x86/smpboot: Restrict soft_restart_cpu() to SEV 2023-05-15 13:44:50 +02:00
head_64.S A large update for SMP management: 2023-06-26 13:59:56 -07:00
hpet.c clocksource: Verify HPET and PMTMR when TSC unverified 2023-02-02 14:23:02 -08:00
hw_breakpoint.c x86/amd: Cache debug register values in percpu variables 2023-01-31 20:09:26 +01:00
i8237.c
i8253.c
i8259.c x86/i8259: Mark legacy PIC interrupts with IRQ_LEVEL 2023-01-16 17:24:56 +01:00
idt.c x86/traps: Add #VE support for TDX guest 2022-04-07 08:27:51 -07:00
io_delay.c
ioport.c
irq_32.c x86/percpu: Move irq_stack variables next to current_task 2022-10-17 16:41:05 +02:00
irq_64.c x86/percpu: Move irq_stack variables next to current_task 2022-10-17 16:41:05 +02:00
irq_work.c
irq.c x86/irq: Add hardcoded hypervisor interrupts to /proc/stat 2023-06-08 08:28:08 -07:00
irqflags.S
irqinit.c x86/i8259: Mark legacy PIC interrupts with IRQ_LEVEL 2023-01-16 17:24:56 +01:00
itmt.c x86/sched/itmt: Give all SMT siblings of a core the same priority 2023-05-08 10:58:38 +02:00
jailhouse.c
jump_label.c jump_label: make initial NOP patching the special case 2022-06-24 09:48:55 +02:00
kdebugfs.c
kexec-bzimage64.c docs: move x86 documentation into Documentation/arch/ 2023-03-30 12:58:51 -06:00
kgdb.c
ksysfs.c
kvm.c ARM64: 2022-12-15 11:12:21 -08:00
kvmclock.c x86/tsc: Provide sched_clock_noinstr() 2023-06-05 21:11:08 +02:00
ldt.c x86: allow get_locked_pte() to fail 2023-06-19 16:19:10 -07:00
machine_kexec_32.c
machine_kexec_64.c x86/kexec: remove unnecessary arch_kexec_kernel_image_load() 2023-04-08 13:45:38 -07:00
Makefile rethook, fprobe: do not trace rethook related functions 2023-05-18 07:08:01 +09:00
mmconf-fam10h_64.c
module.c x86/alternative: Rename apply_ibt_endbr() 2023-07-10 09:52:23 +02:00
mpparse.c
msr.c driver core: class: remove module * from class_create() 2023-03-17 15:16:33 +01:00
nmi_selftest.c
nmi.c locking/atomic: treewide: use raw_atomic*_<op>() 2023-06-05 09:57:20 +02:00
paravirt-spinlocks.c
paravirt.c x86/paravirt: Convert simple paravirt functions to asm 2023-03-17 13:29:47 +01:00
pci-dma.c docs: move x86 documentation into Documentation/arch/ 2023-03-30 12:58:51 -06:00
pcspeaker.c
perf_regs.c
platform-quirks.c x86/quirks: Include linux/pnp.h for arch_pnpbios_disabled() 2023-05-18 11:56:18 -07:00
pmem.c x86/pmem: Fix platform-device leak in error path 2022-06-20 18:01:16 +02:00
probe_roms.c x86/kernel: Validate ROM memory before accessing when SEV-SNP is active 2022-04-06 13:23:09 +02:00
process_32.c x86/resctl: fix scheduler confusion with 'current' 2023-03-08 11:48:11 -08:00
process_64.c IOMMU Updates for Linux 6.4 2023-04-30 13:00:38 -07:00
process.c x86: Rewrite ret_from_fork() in C 2023-07-10 09:52:25 +02:00
process.h
ptrace.c x86: Improve formatting of user_regset arrays 2022-11-01 15:36:52 -07:00
pvclock.c locking/atomic: treewide: use raw_atomic*_<op>() 2023-06-05 09:57:20 +02:00
quirks.c
reboot_fixups_32.c
reboot.c cpu: Mark nmi_panic_self_stop() __noreturn 2023-04-14 17:31:26 +02:00
relocate_kernel_32.S x86/kexec: Disable RET on kexec 2022-07-09 13:12:32 +02:00
relocate_kernel_64.S x86,objtool: Split UNWIND_HINT_EMPTY in two 2023-03-23 23:18:58 +01:00
resource.c x86/PCI: Tidy E820 removal messages 2022-12-10 10:33:11 -06:00
rethook.c
rtc.c x86/rtc: Simplify PNP ids check 2023-01-06 04:22:34 +01:00
setup_percpu.c - Add the call depth tracking mitigation for Retbleed which has 2022-12-14 15:03:00 -08:00
setup.c xen: branch for v6.5-rc1 2023-06-27 16:03:20 -07:00
sev_verify_cbit.S
sev-shared.c x86/sev: Add SNP-specific unaccepted memory support 2023-06-06 18:31:37 +02:00
sev.c - Some SEV and CC platform helpers cleanup and simplifications now that 2023-06-27 13:26:30 -07:00
signal_32.c - Cache the AMD debug registers in per-CPU variables to avoid MSR writes 2023-02-21 14:51:40 -08:00
signal_64.c x86/signal/compat: Move sigaction_compat_abi() to signal_64.c 2023-01-06 04:16:02 +01:00
signal.c x86/init: Initialize signal frame size late 2023-06-16 10:16:00 +02:00
smp.c A set of fixes for kexec(), reboot and shutdown issues 2023-06-26 14:45:53 -07:00
smpboot.c A single fix for the mechanism to park CPUs with an INIT IPI. 2023-07-09 10:08:38 -07:00
stacktrace.c
static_call.c x86/static_call: Add support for Jcc tail-calls 2023-01-31 15:05:31 +01:00
step.c ptrace: Reimplement PTRACE_KILL by always sending SIGKILL 2022-05-11 14:34:28 -05:00
sys_ia32.c
sys_x86_64.c x86/mm: Cleanup the control_va_addr_alignment() __setup handler 2022-05-04 18:20:42 +02:00
tboot.c mm: remove rb tree. 2022-09-26 19:46:16 -07:00
time.c
tls.c x86/gsseg: Move load_gs_index() to its own new header file 2023-01-12 13:06:36 +01:00
tls.h
topology.c x86/topology: Remove CPU0 hotplug option 2023-05-15 13:44:49 +02:00
trace_clock.c
trace.c
tracepoint.c x86/traceponit: Fix comment about irq vector tracepoints 2022-05-26 22:03:52 -04:00
traps.c IOMMU Updates for Linux 6.4 2023-04-30 13:00:38 -07:00
tsc_msr.c
tsc_sync.c x86/smpboot: Make TSC synchronization function call based 2023-05-15 13:44:53 +02:00
tsc.c Scheduler changes for v6.5: 2023-06-27 14:03:21 -07:00
umip.c
unwind_frame.c x86: kmsan: don't instrument stack walking functions 2022-10-03 14:03:25 -07:00
unwind_guess.c
unwind_orc.c objtool changes for v6.5: 2023-06-27 15:05:41 -07:00
uprobes.c uprobes/x86: Allow to probe a NOP instruction with 0x66 prefix 2022-12-05 11:55:18 +01:00
verify_cpu.S
vm86_32.c x86/32: Remove lazy GS macros 2022-04-14 14:09:43 +02:00
vmlinux.lds.S x86/retbleed: Add __x86_return_thunk alignment checks 2023-05-17 12:14:21 +02:00
vsmp_64.c
x86_init.c - Fix a race window where load_unaligned_zeropad() could cause 2023-06-26 16:32:47 -07:00