linux/include
Ryan Roberts 3931b871c4 mm: madvise: avoid split during MADV_PAGEOUT and MADV_COLD
Rework madvise_cold_or_pageout_pte_range() to avoid splitting any large
folio that is fully and contiguously mapped in the pageout/cold vm range. 
This change means that large folios will be maintained all the way to swap
storage.  This both improves performance during swap-out, by eliding the
cost of splitting the folio, and sets us up nicely for maintaining the
large folio when it is swapped back in (to be covered in a separate
series).

Folios that are not fully mapped in the target range are still split, but
note that behavior is changed so that if the split fails for any reason
(folio locked, shared, etc) we now leave it as is and move to the next pte
in the range and continue work on the proceeding folios.  Previously any
failure of this sort would cause the entire operation to give up and no
folios mapped at higher addresses were paged out or made cold.  Given
large folios are becoming more common, this old behavior would have likely
lead to wasted opportunities.

While we are at it, change the code that clears young from the ptes to use
ptep_test_and_clear_young(), via the new mkold_ptes() batch helper
function.  This is more efficent than get_and_clear/modify/set, especially
for contpte mappings on arm64, where the old approach would require
unfolding/refolding and the new approach can be done in place.

Link: https://lkml.kernel.org/r/20240408183946.2991168-8-ryan.roberts@arm.com
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
Reviewed-by: Barry Song <v-songbaohua@oppo.com>
Acked-by: David Hildenbrand <david@redhat.com>
Cc: Barry Song <21cnbao@gmail.com>
Cc: Chris Li <chrisl@kernel.org>
Cc: Gao Xiang <xiang@kernel.org>
Cc: "Huang, Ying" <ying.huang@intel.com>
Cc: Kefeng Wang <wangkefeng.wang@huawei.com>
Cc: Lance Yang <ioworker0@gmail.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Yang Shi <shy828301@gmail.com>
Cc: Yu Zhao <yuzhao@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2024-04-25 20:56:38 -07:00
..
acpi mm: change inlined allocation helpers to account at the call site 2024-04-25 20:55:59 -07:00
asm-generic mm: change inlined allocation helpers to account at the call site 2024-04-25 20:55:59 -07:00
clocksource
crypto mm: change inlined allocation helpers to account at the call site 2024-04-25 20:55:59 -07:00
drm drm fixes for 6.9-rc1 2024-03-21 19:04:31 -07:00
dt-bindings Char/Misc and other driver subsystem updates for 6.9-rc1 2024-03-21 13:21:31 -07:00
keys
kunit
kvm KVM: arm64: Fix host-programmed guest events in nVHE 2024-03-26 01:51:44 -07:00
linux mm: madvise: avoid split during MADV_PAGEOUT and MADV_COLD 2024-04-25 20:56:38 -07:00
math-emu
media media updates for v6.9-rc1 2024-03-15 11:36:54 -07:00
memory
misc
net mm: change inlined allocation helpers to account at the call site 2024-04-25 20:55:59 -07:00
pcmcia
ras PCI/AER: Generalize TLP Header Log reading 2024-03-08 15:26:46 -06:00
rdma fix missing vmalloc.h includes 2024-04-25 20:55:49 -07:00
rv
scsi scsi: sd: Fix TCG OPAL unlock on system resume 2024-03-25 15:46:12 -04:00
soc Char/Misc and other driver subsystem updates for 6.9-rc1 2024-03-21 13:21:31 -07:00
sound ASoC: Fixes for v6.9 2024-04-05 08:48:12 +02:00
target
trace khugepaged: use a folio throughout hpage_collapse_scan_file() 2024-04-25 20:56:34 -07:00
uapi vhost-vdpa: change ioctl # for VDPA_GET_VRING_SIZE 2024-04-08 04:11:04 -04:00
ufs scsi: ufs: core: Add config_scsi_dev vops comment 2024-03-10 18:10:24 -04:00
vdso vdso: Use CONFIG_PAGE_SHIFT in vdso/datapage.h 2024-04-03 21:50:04 +02:00
video
xen