leandrof/linux
1
0
Fork 0
forked from Minki/linux
Code Issues Pull Requests Packages Projects Releases Wiki Activity
55fa19d3e5
linux/arch/x86/include/uapi/asm
History
Peter Zijlstra 6fd166aae7 x86/mm: Use/Fix PCID to optimize user/kernel switches 
We can use PCID to retain the TLBs across CR3 switches; including those now
part of the user/kernel switch. This increases performance of kernel
entry/exit at the cost of more expensive/complicated TLB flushing.

Now that we have two address spaces, one for kernel and one for user space,
we need two PCIDs per mm. We use the top PCID bit to indicate a user PCID
(just like we use the PFN LSB for the PGD). Since we do TLB invalidation
from kernel space, the existing code will only invalidate the kernel PCID,
we augment that by marking the corresponding user PCID invalid, and upon
switching back to userspace, use a flushing CR3 write for the switch.

In order to access the user_pcid_flush_mask we use PER_CPU storage, which
means the previously established SWAPGS vs CR3 ordering is now mandatory
and required.

Having to do this memory access does require additional registers, most
sites have a functioning stack and we can spill one (RAX), sites without
functional stack need to otherwise provide the second scratch register.

Note: PCID is generally available on Intel Sandybridge and later CPUs.
Note: Up until this point TLB flushing was broken in this series.

Based-on-code-from: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Laight <David.Laight@aculab.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Eduardo Valentin <eduval@amazon.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Will Deacon <will.deacon@arm.com>
Cc: aliguori@amazon.com
Cc: daniel.gruss@iaik.tugraz.at
Cc: hughd@google.com
Cc: keescook@google.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-12-23 21:13:00 +01:00
..
a.out.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
auxvec.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
bitsperlong.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
boot.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
bootparam.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
byteorder.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
debugreg.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
e820.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
errno.h
fcntl.h
hw_breakpoint.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
hwcap2.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hyperv.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
ioctl.h
ioctls.h
ipcbuf.h
ist.h License cleanup: add SPDX license identifier to uapi header files with a license 2017-11-02 11:20:11 +01:00
Kbuild kbuild: replace genhdr-y with generated-y 2017-06-22 08:55:21 +09:00
kvm_para.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
kvm_perf.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
kvm.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
ldt.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
mce.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
mman.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
msgbuf.h
msr.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
mtrr.h License cleanup: add SPDX license identifier to uapi header files with a license 2017-11-02 11:20:11 +01:00
param.h
perf_regs.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
poll.h
posix_types_32.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
posix_types_64.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
posix_types_x32.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
posix_types.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
prctl.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
processor-flags.h x86/mm: Use/Fix PCID to optimize user/kernel switches 2017-12-23 21:13:00 +01:00
ptrace-abi.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
ptrace.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
resource.h
sembuf.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
setup.h create non-empty arch/x86/include/uapi/asm/ files 2012-12-17 17:15:11 -08:00
shmbuf.h
sigcontext32.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sigcontext.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
siginfo.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
signal.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
socket.h
sockios.h
stat.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
statfs.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
svm.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
swab.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
termbits.h
termios.h
types.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ucontext.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
unistd.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
vm86.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00
vmx.h License cleanup: add SPDX license identifier to uapi header files with a license 2017-11-02 11:20:11 +01:00
vsyscall.h License cleanup: add SPDX license identifier to uapi header files with no license 2017-11-02 11:19:54 +01:00