linux/arch/powerpc/mm
Peter Xu 4064b98270 mm: allow VM_FAULT_RETRY for multiple times
The idea comes from a discussion between Linus and Andrea [1].

Before this patch we only allow a page fault to retry once.  We achieved
this by clearing the FAULT_FLAG_ALLOW_RETRY flag when doing
handle_mm_fault() the second time.  This was majorly used to avoid
unexpected starvation of the system by looping over forever to handle the
page fault on a single page.  However that should hardly happen, and after
all for each code path to return a VM_FAULT_RETRY we'll first wait for a
condition (during which time we should possibly yield the cpu) to happen
before VM_FAULT_RETRY is really returned.

This patch removes the restriction by keeping the FAULT_FLAG_ALLOW_RETRY
flag when we receive VM_FAULT_RETRY.  It means that the page fault handler
now can retry the page fault for multiple times if necessary without the
need to generate another page fault event.  Meanwhile we still keep the
FAULT_FLAG_TRIED flag so page fault handler can still identify whether a
page fault is the first attempt or not.

Then we'll have these combinations of fault flags (only considering
ALLOW_RETRY flag and TRIED flag):

  - ALLOW_RETRY and !TRIED:  this means the page fault allows to
                             retry, and this is the first try

  - ALLOW_RETRY and TRIED:   this means the page fault allows to
                             retry, and this is not the first try

  - !ALLOW_RETRY and !TRIED: this means the page fault does not allow
                             to retry at all

  - !ALLOW_RETRY and TRIED:  this is forbidden and should never be used

In existing code we have multiple places that has taken special care of
the first condition above by checking against (fault_flags &
FAULT_FLAG_ALLOW_RETRY).  This patch introduces a simple helper to detect
the first retry of a page fault by checking against both (fault_flags &
FAULT_FLAG_ALLOW_RETRY) and !(fault_flag & FAULT_FLAG_TRIED) because now
even the 2nd try will have the ALLOW_RETRY set, then use that helper in
all existing special paths.  One example is in __lock_page_or_retry(), now
we'll drop the mmap_sem only in the first attempt of page fault and we'll
keep it in follow up retries, so old locking behavior will be retained.

This will be a nice enhancement for current code [2] at the same time a
supporting material for the future userfaultfd-writeprotect work, since in
that work there will always be an explicit userfault writeprotect retry
for protected pages, and if that cannot resolve the page fault (e.g., when
userfaultfd-writeprotect is used in conjunction with swapped pages) then
we'll possibly need a 3rd retry of the page fault.  It might also benefit
other potential users who will have similar requirement like userfault
write-protection.

GUP code is not touched yet and will be covered in follow up patch.

Please read the thread below for more information.

[1] https://lore.kernel.org/lkml/20171102193644.GB22686@redhat.com/
[2] https://lore.kernel.org/lkml/20181230154648.GB9832@redhat.com/

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Tested-by: Brian Geffon <bgeffon@google.com>
Cc: Bobby Powers <bobbypowers@gmail.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Denis Plotnikov <dplotnikov@virtuozzo.com>
Cc: "Dr . David Alan Gilbert" <dgilbert@redhat.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Jerome Glisse <jglisse@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: "Kirill A . Shutemov" <kirill@shutemov.name>
Cc: Martin Cracauer <cracauer@cons.org>
Cc: Marty McFadden <mcfadden8@llnl.gov>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Maya Gokhale <gokhale2@llnl.gov>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Mike Rapoport <rppt@linux.vnet.ibm.com>
Cc: Pavel Emelyanov <xemul@openvz.org>
Link: http://lkml.kernel.org/r/20200220160246.9790-1-peterx@redhat.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-04-02 09:35:30 -07:00
..
book3s32 powerpc/32s: Fix DSI and ISI exceptions for CONFIG_VMAP_STACK 2020-02-18 21:31:11 +11:00
book3s64 mm/vma: make vma_is_foreign() available for general use 2020-04-02 09:35:29 -07:00
kasan powerpc/kasan: Fix shadow memory protection with CONFIG_KASAN_VMALLOC 2020-03-13 21:10:37 +11:00
nohash powerpc/8xx: Fix permanently mapped IMMR region. 2020-01-23 21:31:14 +11:00
ptdump powerpc/ptdump: Fix W+X verification 2020-01-23 21:31:13 +11:00
copro_fault.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 153 2019-05-30 11:26:32 -07:00
dma-noncoherent.c dma-mapping: drop the dev argument to arch_sync_dma_for_* 2019-11-20 20:31:38 +01:00
drmem.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
fault.c mm: allow VM_FAULT_RETRY for multiple times 2020-04-02 09:35:30 -07:00
highmem.c powerpc/highmem: Change BUG_ON() to WARN_ON() 2019-04-20 22:02:11 +10:00
hugetlbpage.c powerpc/hugetlb: Fix 512k hugepages on 8xx with 16k page size 2020-02-17 12:47:05 +11:00
init_32.c powerpc: move memstart_addr and kernstart_addr to init-common.c 2019-11-13 19:27:28 +11:00
init_64.c Merge branch 'topic/kaslr-book3e32' into next 2019-11-14 19:23:33 +11:00
init-common.c powerpc: introduce kernstart_virt_addr to store the kernel base 2019-11-13 19:27:32 +11:00
ioremap_32.c powerpc/ioremap: warn on early use of ioremap() 2019-11-19 19:38:38 +11:00
ioremap_64.c powerpc/ioremap: warn on early use of ioremap() 2019-11-19 19:38:38 +11:00
ioremap.c powerpc/mm: split out early ioremap path. 2019-08-27 13:03:35 +10:00
Makefile powerpc/mm: Move ioremap functions out of pgtable_32/64.c 2019-08-27 13:03:35 +10:00
mem.c powerpc/mm: Fix missing KUAP disable in flush_coherent_icache() 2020-03-05 17:15:08 +11:00
mmap.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 156 2019-05-30 11:26:35 -07:00
mmu_context.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
mmu_decl.h powerpc/ptdump: don't entirely rebuild kernel when selecting CONFIG_PPC_DEBUG_WX 2020-01-23 21:31:11 +11:00
numa.c proc: convert everything to "struct proc_ops" 2020-02-04 03:05:26 +00:00
pgtable_32.c powerpc/ptdump: Fix W+X verification call in mark_rodata_ro() 2020-01-23 21:31:12 +11:00
pgtable_64.c powerpc/mm: Move ioremap functions out of pgtable_32/64.c 2019-08-27 13:03:35 +10:00
pgtable-frag.c mm: treewide: clarify pgtable_page_{ctor,dtor}() naming 2019-09-26 10:10:44 -07:00
pgtable.c powerpc updates for 5.3 2019-07-13 16:08:36 -07:00
slice.c powerpc/mm: Mark get_slice_psize() & slice_addr_is_low() as notrace 2019-12-23 21:12:51 +11:00