mirror of
https://github.com/torvalds/linux.git
synced 2024-12-16 08:02:17 +00:00
3d28ebceaf
Lazy TLB state is currently managed in a rather baroque manner. AFAICT, there are three possible states: - Non-lazy. This means that we're running a user thread or a kernel thread that has called use_mm(). current->mm == current->active_mm == cpu_tlbstate.active_mm and cpu_tlbstate.state == TLBSTATE_OK. - Lazy with user mm. We're running a kernel thread without an mm and we're borrowing an mm_struct. We have current->mm == NULL, current->active_mm == cpu_tlbstate.active_mm, cpu_tlbstate.state != TLBSTATE_OK (i.e. TLBSTATE_LAZY or 0). The current cpu is set in mm_cpumask(current->active_mm). CR3 points to current->active_mm->pgd. The TLB is up to date. - Lazy with init_mm. This happens when we call leave_mm(). We have current->mm == NULL, current->active_mm == cpu_tlbstate.active_mm, but that mm is only relelvant insofar as the scheduler is tracking it for refcounting. cpu_tlbstate.state != TLBSTATE_OK. The current cpu is clear in mm_cpumask(current->active_mm). CR3 points to swapper_pg_dir, i.e. init_mm->pgd. This patch simplifies the situation. Other than perf, x86 stops caring about current->active_mm at all. We have cpu_tlbstate.loaded_mm pointing to the mm that CR3 references. The TLB is always up to date for that mm. leave_mm() just switches us to init_mm. There are no longer any special cases for mm_cpumask, and switch_mm() switches mms without worrying about laziness. After this patch, cpu_tlbstate.state serves only to tell the TLB flush code whether it may switch to init_mm instead of doing a normal flush. This makes fairly extensive changes to xen_exit_mmap(), which used to look a bit like black magic. Perf is unchanged. With or without this change, perf may behave a bit erratically if it tries to read user memory in kernel thread context. We should build on this patch to teach perf to never look at user memory when cpu_tlbstate.loaded_mm != current->mm. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bpetkov@suse.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <nadav.amit@gmail.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar <mingo@kernel.org> |
||
|---|---|---|
| .. | ||
| apic.c | ||
| debugfs.c | ||
| debugfs.h | ||
| efi.c | ||
| enlighten_hvm.c | ||
| enlighten_pv.c | ||
| enlighten_pvh.c | ||
| enlighten.c | ||
| grant-table.c | ||
| irq.c | ||
| Kconfig | ||
| Makefile | ||
| mmu_hvm.c | ||
| mmu_pv.c | ||
| mmu.c | ||
| mmu.h | ||
| multicalls.c | ||
| multicalls.h | ||
| p2m.c | ||
| pci-swiotlb-xen.c | ||
| platform-pci-unplug.c | ||
| pmu.c | ||
| pmu.h | ||
| setup.c | ||
| smp_hvm.c | ||
| smp_pv.c | ||
| smp.c | ||
| smp.h | ||
| spinlock.c | ||
| suspend_hvm.c | ||
| suspend_pv.c | ||
| suspend.c | ||
| time.c | ||
| trace.c | ||
| vdso.h | ||
| vga.c | ||
| xen-asm_32.S | ||
| xen-asm_64.S | ||
| xen-asm.h | ||
| xen-asm.S | ||
| xen-head.S | ||
| xen-ops.h | ||
| xen-pvh.S | ||