mirror of
https://github.com/torvalds/linux.git
synced 2024-12-18 17:12:55 +00:00
4b51634cd1
Can the lock_compound_mapcount() bit_spin_lock apparatus be removed now? Yes. Not by atomic64_t or cmpxchg games, those get difficult on 32-bit; but if we slightly abuse subpages_mapcount by additionally demanding that one bit be set there when the compound page is PMD-mapped, then a cascade of two atomic ops is able to maintain the stats without bit_spin_lock. This is harder to reason about than when bit_spin_locked, but I believe safe; and no drift in stats detected when testing. When there are racing removes and adds, of course the sequence of operations is less well- defined; but each operation on subpages_mapcount is atomically good. What might be disastrous, is if subpages_mapcount could ever fleetingly appear negative: but the pte lock (or pmd lock) these rmap functions are called under, ensures that a last remove cannot race ahead of a first add. Continue to make an exception for hugetlb (PageHuge) pages, though that exception can be easily removed by a further commit if necessary: leave subpages_mapcount 0, don't bother with COMPOUND_MAPPED in its case, just carry on checking compound_mapcount too in folio_mapped(), page_mapped(). Evidence is that this way goes slightly faster than the previous implementation in all cases (pmds after ptes now taking around 103ms); and relieves us of worrying about contention on the bit_spin_lock. Link: https://lkml.kernel.org/r/3978f3ca-5473-55a7-4e14-efea5968d892@google.com Signed-off-by: Hugh Dickins <hughd@google.com> Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Dan Carpenter <error27@gmail.com> Cc: David Hildenbrand <david@redhat.com> Cc: James Houghton <jthoughton@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: John Hubbard <jhubbard@nvidia.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Matthew Wilcox <willy@infradead.org> Cc: Miaohe Lin <linmiaohe@huawei.com> Cc: Mike Kravetz <mike.kravetz@oracle.com> Cc: Mina Almasry <almasrymina@google.com> Cc: Muchun Song <songmuchun@bytedance.com> Cc: Naoya Horiguchi <naoya.horiguchi@linux.dev> Cc: Peter Xu <peterx@redhat.com> Cc: Sidhartha Kumar <sidhartha.kumar@oracle.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Yang Shi <shy828301@gmail.com> Cc: Yu Zhao <yuzhao@google.com> Cc: Zach O'Keefe <zokeefe@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|---|---|---|
| .. | ||
| damon | ||
| active_mm.rst | ||
| arch_pgtable_helpers.rst | ||
| balance.rst | ||
| bootmem.rst | ||
| free_page_reporting.rst | ||
| frontswap.rst | ||
| highmem.rst | ||
| hmm.rst | ||
| hugetlbfs_reserv.rst | ||
| hwpoison.rst | ||
| index.rst | ||
| ksm.rst | ||
| memory-model.rst | ||
| mmu_notifier.rst | ||
| multigen_lru.rst | ||
| numa.rst | ||
| oom.rst | ||
| overcommit-accounting.rst | ||
| page_allocation.rst | ||
| page_cache.rst | ||
| page_frags.rst | ||
| page_migration.rst | ||
| page_owner.rst | ||
| page_reclaim.rst | ||
| page_table_check.rst | ||
| page_tables.rst | ||
| physical_memory.rst | ||
| process_addrs.rst | ||
| remap_file_pages.rst | ||
| shmfs.rst | ||
| slab.rst | ||
| slub.rst | ||
| split_page_table_lock.rst | ||
| swap.rst | ||
| transhuge.rst | ||
| unevictable-lru.rst | ||
| vmalloc.rst | ||
| vmalloced-kernel-stacks.rst | ||
| vmemmap_dedup.rst | ||
| z3fold.rst | ||
| zsmalloc.rst | ||