linux/arch/x86/xen
Andy Lutomirski 3d28ebceaf x86/mm: Rework lazy TLB to track the actual loaded mm
Lazy TLB state is currently managed in a rather baroque manner.
AFAICT, there are three possible states:

 - Non-lazy.  This means that we're running a user thread or a
   kernel thread that has called use_mm().  current->mm ==
   current->active_mm == cpu_tlbstate.active_mm and
   cpu_tlbstate.state == TLBSTATE_OK.

 - Lazy with user mm.  We're running a kernel thread without an mm
   and we're borrowing an mm_struct.  We have current->mm == NULL,
   current->active_mm == cpu_tlbstate.active_mm, cpu_tlbstate.state
   != TLBSTATE_OK (i.e. TLBSTATE_LAZY or 0).  The current cpu is set
   in mm_cpumask(current->active_mm).  CR3 points to
   current->active_mm->pgd.  The TLB is up to date.

 - Lazy with init_mm.  This happens when we call leave_mm().  We
   have current->mm == NULL, current->active_mm ==
   cpu_tlbstate.active_mm, but that mm is only relelvant insofar as
   the scheduler is tracking it for refcounting.  cpu_tlbstate.state
   != TLBSTATE_OK.  The current cpu is clear in
   mm_cpumask(current->active_mm).  CR3 points to swapper_pg_dir,
   i.e. init_mm->pgd.

This patch simplifies the situation.  Other than perf, x86 stops
caring about current->active_mm at all.  We have
cpu_tlbstate.loaded_mm pointing to the mm that CR3 references.  The
TLB is always up to date for that mm.  leave_mm() just switches us
to init_mm.  There are no longer any special cases for mm_cpumask,
and switch_mm() switches mms without worrying about laziness.

After this patch, cpu_tlbstate.state serves only to tell the TLB
flush code whether it may switch to init_mm instead of doing a
normal flush.

This makes fairly extensive changes to xen_exit_mmap(), which used
to look a bit like black magic.

Perf is unchanged.  With or without this change, perf may behave a bit
erratically if it tries to read user memory in kernel thread context.
We should build on this patch to teach perf to never look at user
memory when cpu_tlbstate.loaded_mm != current->mm.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Borislav Petkov <bpetkov@suse.de>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Nadav Amit <nadav.amit@gmail.com>
Cc: Nadav Amit <namit@vmware.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-mm@kvack.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-06-05 09:59:44 +02:00
..
apic.c x86/xen: Fix APIC id mismatch warning on Intel 2017-01-29 18:59:16 -05:00
debugfs.c x86/xen: Audit and remove any unnecessary uses of module.h 2016-07-14 15:06:59 +02:00
debugfs.h debugfs: Add support to print u32 array in debugfs 2012-04-17 00:18:36 -04:00
efi.c xen: Implement EFI reset_system callback 2017-05-02 12:06:50 +02:00
enlighten_hvm.c xen: Move xen_have_vector_callback definition to enlighten.c 2017-05-03 06:46:11 +02:00
enlighten_pv.c xen: cleanup pvh leftovers from pv-only sources 2017-05-19 08:04:20 +02:00
enlighten_pvh.c xen/pvh: Do not fill kernel's e820 map in init_pvh_bootparams() 2017-05-02 11:17:39 +02:00
enlighten.c xen: Move xen_have_vector_callback definition to enlighten.c 2017-05-03 06:46:11 +02:00
grant-table.c xen/grant-table: Use kmalloc_array() in arch_gnttab_valloc() 2016-08-26 10:44:22 +01:00
irq.c x86/xen: use xen_vcpu_id mapping for HYPERVISOR_vcpu_op 2016-07-25 13:32:34 +01:00
Kconfig x86/xen: enable PVHVM-only builds 2017-05-02 11:10:16 +02:00
Makefile x86/xen: enable PVHVM-only builds 2017-05-02 11:10:16 +02:00
mmu_hvm.c x86/xen: split off mmu_hvm.c 2017-05-02 11:05:10 +02:00
mmu_pv.c x86/mm: Rework lazy TLB to track the actual loaded mm 2017-06-05 09:59:44 +02:00
mmu.c xen: make xen_flush_tlb_all() static 2017-05-19 08:04:25 +02:00
mmu.h x86/xen: Change __xen_pgd_walk() and xen_cleanmfnmap() to support p4d 2017-03-27 08:56:49 +02:00
multicalls.c x86: Replace __get_cpu_var uses 2014-08-26 13:45:49 -04:00
multicalls.h xen: use this_cpu_xxx replace percpu_xxx funcs 2012-01-24 12:20:24 -05:00
p2m.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
pci-swiotlb-xen.c treewide: Constify most dma_map_ops structures 2017-01-24 12:23:35 -05:00
platform-pci-unplug.c xen/pvh: PVH guests always have PV devices 2017-02-07 08:07:01 -05:00
pmu.c xen: Make VPMU init message look less scary 2016-08-24 18:45:38 +01:00
pmu.h x86/xen: put setup.c, pmu.c and apic.c under CONFIG_XEN_PV 2017-05-02 11:09:28 +02:00
setup.c xen/x86: Remove PVH support 2017-02-07 08:07:01 -05:00
smp_hvm.c xen: Revert commits da72ff5bfc and 72a9b18629 2017-05-02 11:18:05 +02:00
smp_pv.c xen/x86: Call xen_smp_intr_init_pv() on BSP 2017-05-02 11:18:13 +02:00
smp.c x86/xen: split off smp_pv.c 2017-05-02 11:05:00 +02:00
smp.h x86/xen: split off smp_pv.c 2017-05-02 11:05:00 +02:00
spinlock.c locking/spinlocks/x86, paravirt: Remove paravirt_ticketlocks_enabled 2017-01-14 09:33:46 +01:00
suspend_hvm.c x86/xen: split suspend.c for PV and PVHVM guests 2017-05-02 11:09:17 +02:00
suspend_pv.c x86/xen: split suspend.c for PV and PVHVM guests 2017-05-02 11:09:17 +02:00
suspend.c x86/xen: split suspend.c for PV and PVHVM guests 2017-05-02 11:09:17 +02:00
time.c xen/x86: Do not call xen_init_time_ops() until shared_info is initialized 2017-05-05 10:43:15 +02:00
trace.c xen: use generated hypervisor symbols in arch/x86/xen/trace.c 2015-03-16 14:49:13 +00:00
vdso.h
vga.c xen/vga: add the xen EFI video mode support 2012-09-24 09:28:57 -04:00
xen-asm_32.S x86/paravirt: Remove the unused irq_enable_sysexit pv op 2015-11-23 10:48:16 +01:00
xen-asm_64.S x86/asm/xen: Set ELF function type for xen_adjust_exception_frame() 2016-02-24 08:35:41 +01:00
xen-asm.h
xen-asm.S x86/asm/xen: Create stack frames in xen-asm.S 2016-02-24 08:35:42 +01:00
xen-head.S x86/xen: define startup_xen for XEN PV only 2017-05-02 11:09:37 +02:00
xen-ops.h xen: Export xen_reboot 2017-05-02 11:50:06 +02:00
xen-pvh.S xen/pvh: Bootstrap PVH guest 2017-02-07 08:07:01 -05:00