linux/arch/arm64/mm
Kefeng Wang faab3d0f25 arm64: mm: accelerate pagefault when VM_FAULT_BADACCESS
The vm_flags of vma already checked under per-VMA lock, if it is a bad
access, directly set fault to VM_FAULT_BADACCESS and handle error, no need
to retry with mmap_lock again, the latency time reduces 34% in 'lat_sig -P
1 prot lat_sig' from lmbench testcase.

Since the page fault is handled under per-VMA lock, count it as a vma lock
event with VMA_LOCK_SUCCESS.

Link: https://lkml.kernel.org/r/20240403083805.1818160-3-wangkefeng.wang@huawei.com
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2024-04-25 20:56:38 -07:00
..
cache.S efi: arm64: enter with MMU and caches enabled 2023-01-24 11:51:08 +00:00
context.c arm64/mm: remove now-superfluous ISBs from TTBR writes 2023-06-15 17:47:54 +01:00
contpte.c arm64/mm: improve comment in contpte_ptep_get_lockless() 2024-03-04 17:01:18 -08:00
copypage.c arm64: Also reset KASAN tag if page is not PG_mte_tagged 2023-05-16 14:58:54 +01:00
dma-mapping.c iommu: Remove struct iommu_ops *iommu from arch_setup_dma_ops() 2023-12-12 10:18:45 +01:00
extable.c arm64: extable: cleanup redundant extable type EX_TYPE_FIXUP 2022-06-28 12:11:47 +01:00
fault.c arm64: mm: accelerate pagefault when VM_FAULT_BADACCESS 2024-04-25 20:56:38 -07:00
fixmap.c - Sumanth Korikkar has taught s390 to allocate hotplug-time page frames 2024-03-14 17:43:30 -07:00
flush.c arm64: implement the new page table range API 2023-08-24 16:20:20 -07:00
hugetlbpage.c mm/treewide: remove pXd_huge() 2024-04-25 20:55:47 -07:00
init.c - Sumanth Korikkar has taught s390 to allocate hotplug-time page frames 2024-03-14 17:43:30 -07:00
ioremap.c arm64 : mm: add wrapper function ioremap_prot() 2023-08-18 10:12:36 -07:00
kasan_init.c - Sumanth Korikkar has taught s390 to allocate hotplug-time page frames 2024-03-14 17:43:30 -07:00
Makefile arm64/mm: wire up PTE_CONT for user mappings 2024-02-22 15:27:18 -08:00
mmap.c arm64: mm: Wire up TCR.DS bit to PTE shareability fields 2024-02-16 12:42:38 +00:00
mmu.c - Sumanth Korikkar has taught s390 to allocate hotplug-time page frames 2024-03-14 17:43:30 -07:00
mteswap.c arm64: mm: swap: support THP_SWAP on hardware with MTE 2024-04-25 20:56:07 -07:00
pageattr.c arm64/mm: new ptep layer to manage contig bit 2024-02-22 15:27:18 -08:00
pgd.c arm64: mm: Add support for folding PUDs at runtime 2024-02-16 12:42:41 +00:00
physaddr.c arm64: Do not pass tagged addresses to __is_lm_address() 2021-02-02 17:44:47 +00:00
proc.S Revert "arm64: mm: add support for WXN memory translation attribute" 2024-03-13 10:53:20 +00:00
ptdump_debugfs.c arm64: Add __init section marker to some functions 2021-04-08 17:45:10 +01:00
ptdump.c - Sumanth Korikkar has taught s390 to allocate hotplug-time page frames 2024-03-14 17:43:30 -07:00
trans_pgd-asm.S arm64: kexec: configure EL2 vectors for kexec 2021-10-01 13:31:00 +01:00
trans_pgd.c arm64/mm: new ptep layer to manage contig bit 2024-02-22 15:27:18 -08:00