linux/arch/x86/mm
Dave Hansen 1de14c3c5c x86-32: Fix possible incomplete TLB invalidate with PAE pagetables
This patch attempts to fix:

	https://bugzilla.kernel.org/show_bug.cgi?id=56461

The symptom is a crash and messages like this:

	chrome: Corrupted page table at address 34a03000
	*pdpt = 0000000000000000 *pde = 0000000000000000
	Bad pagetable: 000f [#1] PREEMPT SMP

Ingo guesses this got introduced by commit 611ae8e3f5 ("x86/tlb:
enable tlb flush range support for x86") since that code started to free
unused pagetables.

On x86-32 PAE kernels, that new code has the potential to free an entire
PMD page and will clear one of the four page-directory-pointer-table
(aka pgd_t entries).

The hardware aggressively "caches" these top-level entries and invlpg
does not actually affect the CPU's copy.  If we clear one we *HAVE* to
do a full TLB flush, otherwise we might continue using a freed pmd page.
(note, we do this properly on the population side in pud_populate()).

This patch tracks whenever we clear one of these entries in the 'struct
mmu_gather', and ensures that we follow up with a full tlb flush.

BTW, I disassembled and checked that:

	if (tlb->fullmm == 0)
and
	if (!tlb->fullmm && !tlb->need_flush_all)

generate essentially the same code, so there should be zero impact there
to the !PAE case.

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Peter Anvin <hpa@zytor.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Artem S Tashkinov <t.artem@mailcity.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-04-12 16:56:47 -07:00
..
kmemcheck bug.h: add include of it to various implicit C users 2012-02-29 17:15:08 -05:00
amdtopology.c x86, NUMA: Enable CONFIG_AMD_NUMA on 32bit too 2011-05-02 17:24:48 +02:00
dump_pagetables.c x86, mm: Create symbolic index into address_markers array 2010-07-20 16:56:19 -07:00
extable.c x86, extable: Switch to relative exception table entries 2012-04-20 17:22:34 -07:00
fault.c Revert "x86, mm: Make spurious_fault check explicitly check explicitly check the PRESENT bit" 2013-02-24 13:01:44 +01:00
gup.c thp: add compound tail page _mapcount when mapped 2011-12-09 07:50:28 -08:00
highmem_32.c highmem: kill all __kmap_atomic() 2012-03-20 21:48:30 +08:00
hugetlbpage.c mm: use vm_unmapped_area() in hugetlbfs on i386 architecture 2012-12-11 17:22:25 -08:00
init_32.c memory-hotplug: introduce new arch_remove_memory() for removing page table 2013-02-23 17:50:12 -08:00
init_64.c memory-hotplug: remove memmap of sparse-vmemmap 2013-02-23 17:50:12 -08:00
init.c x86, mm: Make sure to find a 2M free block for the first mapped area 2013-03-06 20:18:32 -08:00
iomap_32.c mm: fix race in kunmap_atomic() 2010-10-27 18:03:05 -07:00
ioremap.c Revert "x86, mm: Include the entire kernel memory map in trampoline_pgd" 2012-12-15 12:29:54 -08:00
kmmio.c x86, kmmio/mmiotrace: Fix double free of kmmio_fault_pages 2010-06-18 11:30:09 +02:00
Makefile memblock, x86: Replace memblock_x86_reserve/free_range() with generic ones 2011-07-14 11:47:53 -07:00
memtest.c x86/memtest: Shorten time for tests 2013-02-18 09:28:42 +01:00
mm_internal.h x86, mm: Move after_bootmem to mm_internel.h 2012-11-17 11:59:45 -08:00
mmap.c x86: Fix mmap random address range 2011-12-05 17:07:23 +01:00
mmio-mod.c module_param: make bool parameters really bool (arch) 2012-01-13 09:32:18 +10:30
numa_32.c x86-32, mm: Rip out x86_32 NUMA remapping code 2013-01-31 14:12:30 -08:00
numa_64.c x86, mm: kill numa_free_all_bootmem() 2012-11-17 11:59:47 -08:00
numa_emulation.c x86: print physical addresses consistently with other parts of kernel 2012-05-29 16:22:21 -07:00
numa_internal.h x86-32, mm: Rip out x86_32 NUMA remapping code 2013-01-31 14:12:30 -08:00
numa.c x86, ACPI, mm: Revert movablemem_map support 2013-03-02 09:34:39 -08:00
pageattr-test.c x86: Convert vmalloc()+memset() to vzalloc() 2011-05-28 19:53:57 +02:00
pageattr.c Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2013-02-26 19:45:29 -08:00
pat_internal.h x86, pat: Fix memory leak in free_memtype 2010-05-26 11:26:04 -07:00
pat_rbtree.c rbtree: move augmented rbtree functionality to rbtree_augmented.h 2012-10-09 16:22:40 +09:00
pat.c x86: Do not try to sync identity map for non-mapped pages 2013-03-07 13:23:28 -08:00
pf_in.c x86: Eliminate various 'set but not used' warnings 2011-05-21 19:10:33 +02:00
pf_in.h
pgtable_32.c Disintegrate asm/system.h for X86 2012-03-28 18:11:12 +01:00
pgtable.c x86-32: Fix possible incomplete TLB invalidate with PAE pagetables 2013-04-12 16:56:47 -07:00
physaddr.c x86, mm: Make DEBUG_VIRTUAL work earlier in boot 2013-01-25 16:33:22 -08:00
physaddr.h
setup_nx.c x86, cpu: Only CPU features determine NX capabilities 2010-11-10 15:43:15 -08:00
srat.c x86, ACPI, mm: Revert movablemem_map support 2013-03-02 09:34:39 -08:00
testmmiotrace.c x86, kmmio/mmiotrace: Fix double free of kmmio_fault_pages 2010-06-18 11:30:09 +02:00
tlb.c x86: Convert a few mistaken __cpuinit annotations to __init 2013-01-24 17:12:19 +01:00