linux/arch/powerpc/mm
Christophe Leroy 0e36b0d125 powerpc/mm: Only read faulting instruction when necessary in do_page_fault()
Commit a7a9dcd882 ("powerpc: Avoid taking a data miss on every
userspace instruction miss") has shown that limiting the read of
faulting instruction to likely cases improves performance.

This patch goes further into this direction by limiting the read
of the faulting instruction to the only cases where it is likely
needed.

On an MPC885, with the same benchmark app as in the commit referred
above, we see a reduction of about 3900 dTLB misses (approx 3%):

Before the patch:
 Performance counter stats for './fault 500' (10 runs):

         683033312      cpu-cycles                                                    ( +-  0.03% )
            134538      dTLB-load-misses                                              ( +-  0.03% )
             46099      iTLB-load-misses                                              ( +-  0.02% )
             19681      faults                                                        ( +-  0.02% )

       5.389747878 seconds time elapsed                                          ( +-  0.06% )

With the patch:

 Performance counter stats for './fault 500' (10 runs):

         682112862      cpu-cycles                                                    ( +-  0.03% )
            130619      dTLB-load-misses                                              ( +-  0.03% )
             46073      iTLB-load-misses                                              ( +-  0.05% )
             19681      faults                                                        ( +-  0.01% )

       5.381342641 seconds time elapsed                                          ( +-  0.07% )

The proper work of the huge stack expansion was tested with the
following app:

int main(int argc, char **argv)
{
	char buf[1024 * 1025];

	sprintf(buf, "Hello world !\n");
	printf(buf);

	exit(0);
}

Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr>
Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
[mpe: Add include of pagemap.h to fix build errors]
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-05-25 00:08:25 +10:00
..
8xx_mmu.c powerpc/mm/slice: Fix hugepage allocation at hint address on 8xx 2018-03-06 09:21:23 +11:00
40x_mmu.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
44x_mmu.c
copro_fault.c powerpc/mm: Add support for handling > 512TB address in SLB miss 2018-03-31 00:10:38 +11:00
dma-noncoherent.c powerpc/mm: Rename map_page() to map_kernel_page() on 32-bit 2017-06-05 19:59:03 +10:00
drmem.c powerpc/mm/drmem: Fix unexpected flag value in ibm,dynamic-memory-v2 2018-02-23 16:45:51 +11:00
dump_hashpagetable.c powerpc/64s: Replace CONFIG_PPC_STD_MMU_64 with CONFIG_PPC_BOOK3S_64 2017-11-06 16:48:14 +11:00
dump_linuxpagetables.c powerpc/mm: Introduce _PAGE_NA 2018-01-16 23:47:14 +11:00
fault.c powerpc/mm: Only read faulting instruction when necessary in do_page_fault() 2018-05-25 00:08:25 +10:00
fsl_booke_mmu.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
hash64_4k.c powerpc/mm/hash64: Store the slot information at the right offset for hugetlb 2018-02-13 22:37:48 +11:00
hash64_64k.c powerpc/mm/hash64: Store the slot information at the right offset for hugetlb 2018-02-13 22:37:48 +11:00
hash_low_32.S powerpc: fix location of two EXPORT_SYMBOL 2017-09-01 16:42:45 +10:00
hash_native_64.c Merge branch 'fixes' into next 2018-03-28 22:59:50 +11:00
hash_utils_64.c powerpc/mm: Use page fragments for allocation page table at PMD level 2018-05-15 22:29:12 +10:00
highmem.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hugepage-hash64.c powerpc/mm: Move hash table ops to a separate structure 2016-07-21 18:59:09 +10:00
hugetlbpage-book3e.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hugetlbpage-hash64.c powerpc/mm/hash64: Store the slot information at the right offset for hugetlb 2018-02-13 22:37:48 +11:00
hugetlbpage-radix.c powerpc updates for 4.15 2017-11-16 12:47:46 -08:00
hugetlbpage.c powerpc/fadump: Do not use hugepages when fadump is active 2018-05-03 23:09:25 +10:00
init_32.c powerpc/mm/32: Remove the reserved memory hack 2018-04-01 00:47:44 +11:00
init_64.c powerpc/mm/radix: Parse disable_radix commandline correctly. 2018-04-04 16:59:36 +10:00
init-common.c powerpc/mm: Fix crashes with 16G huge pages 2018-02-13 22:37:47 +11:00
Makefile powerpc: initial pkey plumbing 2018-01-20 21:45:03 +11:00
mem.c powerpc/mm: Flush cache on memory hot(un)plug 2018-04-24 09:46:56 +10:00
mmap.c exec: pass stack rlimit into mm layout functions 2018-04-11 10:28:37 -07:00
mmu_context_book3s64.c powerpc/mm: Implement helpers for pagetable fragment support at PMD level 2018-05-15 22:29:12 +10:00
mmu_context_hash32.c powerpc: Remove power3 from comments 2014-07-28 14:10:26 +10:00
mmu_context_iommu.c mm, migrate: remove reason argument from new_page_t 2018-04-11 10:28:32 -07:00
mmu_context_nohash.c powerpc/mm/slice: Simplify and optimise slice context initialisation 2018-03-13 23:43:05 +11:00
mmu_context.c powerpc, membarrier: Skip memory barrier in switch_mm() 2018-02-05 21:34:02 +01:00
mmu_decl.h powerpc/mm/32: Remove the reserved memory hack 2018-04-01 00:47:44 +11:00
numa.c Merge branch 'topic/paca' into next 2018-03-31 09:09:36 +11:00
pgtable_32.c powerpc/mm/32: Remove the reserved memory hack 2018-04-01 00:47:44 +11:00
pgtable_64.c powerpc/mm: Use page fragments for allocation page table at PMD level 2018-05-15 22:29:12 +10:00
pgtable-book3e.c powerpc/mm: Make page table size a variable 2016-05-01 18:32:48 +10:00
pgtable-book3s64.c powerpc/mm: Use page fragments for allocation page table at PMD level 2018-05-15 22:29:12 +10:00
pgtable-hash64.c powerpc/mm: Use pmd_lockptr instead of opencoding it 2018-05-15 22:29:09 +10:00
pgtable-radix.c powerpc/mm: Use page fragments for allocation page table at PMD level 2018-05-15 22:29:12 +10:00
pgtable.c powerpc/mm: extend _PAGE_PRIVILEGED to all CPUs 2018-01-16 23:47:13 +11:00
pkeys.c powerpc/mm/keys: Update documentation and remove unnecessary check 2018-04-04 15:23:09 +10:00
ppc_mmu_32.c powerpc32: refactor x_mapped_by_bats() and x_mapped_by_tlbcam() together 2016-03-11 17:18:02 -06:00
slb_low.S powerpc/mm: Add support for handling > 512TB address in SLB miss 2018-03-31 00:10:38 +11:00
slb.c powerpc/mm: Add support for handling > 512TB address in SLB miss 2018-03-31 00:10:38 +11:00
slice.c powerpc/8xx: Fix build with hugetlbfs enabled 2018-04-11 12:00:23 +10:00
subpage-prot.c powerpc/syscalls: Switch trivial cases to SYSCALL_DEFINE 2018-05-10 23:25:12 +10:00
tlb_hash32.c powerpc/mm: remove flush_tlb_page_nohash 2016-08-01 11:15:13 +10:00
tlb_hash64.c powerpc/mm: Add support for handling > 512TB address in SLB miss 2018-03-31 00:10:38 +11:00
tlb_low_64e.S powerpc: Fix misspellings in comments. 2016-03-01 19:27:20 +11:00
tlb_nohash_low.S powerpc/8xx: Getting rid of remaining use of CONFIG_8xx 2017-08-10 23:32:12 +10:00
tlb_nohash.c powerpc/mm/nohash: do not flush the entire mm when range is a single page 2018-01-27 20:24:44 +11:00
tlb-radix.c powerpc/mm/radix: Fix checkstops caused by invalid tlbiel 2018-04-12 23:49:55 +10:00
vphn.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
vphn.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00