linux/arch/x86/mm
Thomas Gleixner 4c71a2b6fd x86/speculation: Prepare for conditional IBPB in switch_mm()
The IBPB speculation barrier is issued from switch_mm() when the kernel
switches to a user space task with a different mm than the user space task
which ran last on the same CPU.

An additional optimization is to avoid IBPB when the incoming task can be
ptraced by the outgoing task. This optimization only works when switching
directly between two user space tasks. When switching from a kernel task to
a user space task the optimization fails because the previous task cannot
be accessed anymore. So for quite some scenarios the optimization is just
adding overhead.

The upcoming conditional IBPB support will issue IBPB only for user space
tasks which have the TIF_SPEC_IB bit set. This requires to handle the
following cases:

  1) Switch from a user space task (potential attacker) which has
     TIF_SPEC_IB set to a user space task (potential victim) which has
     TIF_SPEC_IB not set.

  2) Switch from a user space task (potential attacker) which has
     TIF_SPEC_IB not set to a user space task (potential victim) which has
     TIF_SPEC_IB set.

This needs to be optimized for the case where the IBPB can be avoided when
only kernel threads ran in between user space tasks which belong to the
same process.

The current check whether two tasks belong to the same context is using the
tasks context id. While correct, it's simpler to use the mm pointer because
it allows to mangle the TIF_SPEC_IB bit into it. The context id based
mechanism requires extra storage, which creates worse code.

When a task is scheduled out its TIF_SPEC_IB bit is mangled as bit 0 into
the per CPU storage which is used to track the last user space mm which was
running on a CPU. This bit can be used together with the TIF_SPEC_IB bit of
the incoming task to make the decision whether IBPB needs to be issued or
not to cover the two cases above.

As conditional IBPB is going to be the default, remove the dubious ptrace
check for the IBPB always case and simply issue IBPB always when the
process changes.

Move the storage to a different place in the struct as the original one
created a hole.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Jiri Kosina <jkosina@suse.cz>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: David Woodhouse <dwmw@amazon.co.uk>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Casey Schaufler <casey.schaufler@intel.com>
Cc: Asit Mallick <asit.k.mallick@intel.com>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Jon Masters <jcm@redhat.com>
Cc: Waiman Long <longman9394@gmail.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: Dave Stewart <david.c.stewart@intel.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20181125185005.466447057@linutronix.de
2018-11-28 11:57:11 +01:00
..
amdtopology.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
cpu_entry_area.c x86/pti/64: Remove the SYSCALL64 entry trampoline 2018-09-12 21:33:53 +02:00
debug_pagetables.c x86/efi: Replace efi_pgd with efi_mm.pgd 2018-03-12 11:05:05 +01:00
dump_pagetables.c x86/mm: Do not warn about PCI BIOS W+X mappings 2018-10-10 15:16:13 +02:00
extable.c x86/mm: Fix typo in comment 2018-10-03 16:14:05 +02:00
fault.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
highmem_32.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
hugetlbpage.c x86/compat: Adjust in_compat_syscall() to generic code under !COMPAT 2018-11-01 12:59:25 +01:00
ident_map.c x86/mm: Stop pretending pgtable_l5_enabled is a variable 2018-05-19 11:56:57 +02:00
init_32.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
init_64.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
init.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
iomap_32.c x86/mm: Do not auto-massage page protections 2018-04-12 09:04:22 +02:00
ioremap.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
kasan_init_64.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
kaslr.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
kmmio.c x86/mm/kmmio: Make the tracer robust against L1TF 2018-08-08 22:28:34 +02:00
Makefile x86/mm/sme: Disable stack protection for mem_encrypt_identity.c 2018-02-28 15:24:12 +01:00
mem_encrypt_boot.S x86/mm/sme, objtool: Annotate indirect call in sme_encrypt_execute() 2018-02-21 09:05:04 +01:00
mem_encrypt_identity.c x86/paravirt: Introduce new config option PARAVIRT_XXL 2018-09-03 16:50:35 +02:00
mem_encrypt.c x86/mm: Add .bss..decrypted section to hold shared variables 2018-09-15 20:48:45 +02:00
mm_internal.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
mmap.c x86/compat: Adjust in_compat_syscall() to generic code under !COMPAT 2018-11-01 12:59:25 +01:00
mmio-mod.c x86: do not use print_symbol() 2018-01-05 15:23:01 +01:00
mpx.c signal/x86: Move mpx siginfo generation into do_bounds 2018-09-19 15:53:11 +02:00
numa_32.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
numa_64.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
numa_emulation.c Merge branch 'core/urgent' into x86/urgent, to pick up objtool fix 2018-11-03 23:42:16 +01:00
numa_internal.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
numa.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
pageattr-test.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
pageattr.c Merge branch 'core/urgent' into x86/urgent, to pick up objtool fix 2018-11-03 23:42:16 +01:00
pat_internal.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
pat_rbtree.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
pat.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
pf_in.c
pf_in.h
pgtable_32.c x86/mm: Rename flush_tlb_single() and flush_tlb_one() to __flush_tlb_one_[user|kernel]() 2018-02-15 01:15:52 +01:00
pgtable.c x86/mm: Avoid VLA in pgd_alloc() 2018-10-09 08:55:07 +02:00
physaddr.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
physaddr.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
pkeys.c x86/pkeys: Override pkey when moving away from PROT_EXEC 2018-05-14 11:14:45 +02:00
pti.c x86/pti/64: Remove the SYSCALL64 entry trampoline 2018-09-12 21:33:53 +02:00
setup_nx.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
srat.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
testmmiotrace.c Annotate hardware config module parameters in arch/x86/mm/ 2017-04-04 16:54:21 +01:00
tlb.c x86/speculation: Prepare for conditional IBPB in switch_mm() 2018-11-28 11:57:11 +01:00