mirror of
https://github.com/torvalds/linux.git
synced 2024-11-27 06:31:52 +00:00
fc37bbb328
Patch series "hugetlb: parallelize hugetlb page init on boot", v6. Introduction ------------ Hugetlb initialization during boot takes up a considerable amount of time. For instance, on a 2TB system, initializing 1,800 1GB huge pages takes 1-2 seconds out of 10 seconds. Initializing 11,776 1GB pages on a 12TB Intel host takes more than 1 minute[1]. This is a noteworthy figure. Inspired by [2] and [3], hugetlb initialization can also be accelerated through parallelization. Kernel already has infrastructure like padata_do_multithreaded, this patch uses it to achieve effective results by minimal modifications. [1] https://lore.kernel.org/all/783f8bac-55b8-5b95-eb6a-11a583675000@google.com/ [2] https://lore.kernel.org/all/20200527173608.2885243-1-daniel.m.jordan@oracle.com/ [3] https://lore.kernel.org/all/20230906112605.2286994-1-usama.arif@bytedance.com/ [4] https://lore.kernel.org/all/76becfc1-e609-e3e8-2966-4053143170b6@google.com/ max_threads ----------- This patch use `padata_do_multithreaded` like this: ``` job.max_threads = num_node_state(N_MEMORY) * multiplier; padata_do_multithreaded(&job); ``` To fully utilize the CPU, the number of parallel threads needs to be carefully considered. `max_threads = num_node_state(N_MEMORY)` does not fully utilize the CPU, so we need to multiply it by a multiplier. Tests below indicate that a multiplier of 2 significantly improves performance, and although larger values also provide improvements, the gains are marginal. multiplier 1 2 3 4 5 ------------ ------- ------- ------- ------- ------- 256G 2node 358ms 215ms 157ms 134ms 126ms 2T 4node 979ms 679ms 543ms 489ms 481ms 50G 2node 71ms 44ms 37ms 30ms 31ms Therefore, choosing 2 as the multiplier strikes a good balance between enhancing parallel processing capabilities and maintaining efficient resource management. Test result ----------- test case no patch(ms) patched(ms) saved ------------------- -------------- ------------- -------- 256c2T(4 node) 1G 4745 2024 57.34% 128c1T(2 node) 1G 3358 1712 49.02% 12T 1G 77000 18300 76.23% 256c2T(4 node) 2M 3336 1051 68.52% 128c1T(2 node) 2M 1943 716 63.15% This patch (of 8): The readability of `hugetlb_hstate_alloc_pages` is poor. By cleaning the code, its readability can be improved, facilitating future modifications. This patch extracts two functions to reduce the complexity of `hugetlb_hstate_alloc_pages` and has no functional changes. - hugetlb_hstate_alloc_pages_node_specific() to handle iterates through each online node and performs allocation if necessary. - hugetlb_hstate_alloc_pages_report() report error during allocation. And the value of h->max_huge_pages is updated accordingly. Link: https://lkml.kernel.org/r/20240222140422.393911-1-gang.li@linux.dev Link: https://lkml.kernel.org/r/20240222140422.393911-2-gang.li@linux.dev Signed-off-by: Gang Li <ligang.bdlg@bytedance.com> Tested-by: David Rientjes <rientjes@google.com> Reviewed-by: Muchun Song <muchun.song@linux.dev> Reviewed-by: Tim Chen <tim.c.chen@linux.intel.com> Cc: Daniel Jordan <daniel.m.jordan@oracle.com> Cc: David Hildenbrand <david@redhat.com> Cc: Jane Chu <jane.chu@oracle.com> Cc: Paul E. McKenney <paulmck@kernel.org> Cc: Randy Dunlap <rdunlap@infradead.org> Cc: Steffen Klassert <steffen.klassert@secunet.com> Cc: Alexey Dobriyan <adobriyan@gmail.com> Cc: Mike Kravetz <mike.kravetz@oracle.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|---|---|---|
| .. | ||
| damon | ||
| kasan | ||
| kfence | ||
| kmsan | ||
| backing-dev.c | ||
| balloon_compaction.c | ||
| bootmem_info.c | ||
| cma_debug.c | ||
| cma_sysfs.c | ||
| cma.c | ||
| cma.h | ||
| compaction.c | ||
| debug_page_alloc.c | ||
| debug_page_ref.c | ||
| debug_vm_pgtable.c | ||
| debug.c | ||
| dmapool_test.c | ||
| dmapool.c | ||
| early_ioremap.c | ||
| fadvise.c | ||
| fail_page_alloc.c | ||
| failslab.c | ||
| filemap.c | ||
| folio-compat.c | ||
| gup_test.c | ||
| gup_test.h | ||
| gup.c | ||
| highmem.c | ||
| hmm.c | ||
| huge_memory.c | ||
| hugetlb_cgroup.c | ||
| hugetlb_vmemmap.c | ||
| hugetlb_vmemmap.h | ||
| hugetlb.c | ||
| hwpoison-inject.c | ||
| init-mm.c | ||
| internal.h | ||
| interval_tree.c | ||
| io-mapping.c | ||
| ioremap.c | ||
| Kconfig | ||
| Kconfig.debug | ||
| khugepaged.c | ||
| kmemleak.c | ||
| ksm.c | ||
| list_lru.c | ||
| maccess.c | ||
| madvise.c | ||
| Makefile | ||
| mapping_dirty_helpers.c | ||
| memblock.c | ||
| memcontrol.c | ||
| memfd.c | ||
| memory_hotplug.c | ||
| memory-failure.c | ||
| memory-tiers.c | ||
| memory.c | ||
| mempolicy.c | ||
| mempool.c | ||
| memremap.c | ||
| memtest.c | ||
| migrate_device.c | ||
| migrate.c | ||
| mincore.c | ||
| mlock.c | ||
| mm_init.c | ||
| mm_slot.h | ||
| mmap_lock.c | ||
| mmap.c | ||
| mmu_gather.c | ||
| mmu_notifier.c | ||
| mmzone.c | ||
| mprotect.c | ||
| mremap.c | ||
| msync.c | ||
| nommu.c | ||
| oom_kill.c | ||
| page_alloc.c | ||
| page_counter.c | ||
| page_ext.c | ||
| page_idle.c | ||
| page_io.c | ||
| page_isolation.c | ||
| page_owner.c | ||
| page_poison.c | ||
| page_reporting.c | ||
| page_reporting.h | ||
| page_table_check.c | ||
| page_vma_mapped.c | ||
| page-writeback.c | ||
| pagewalk.c | ||
| percpu-internal.h | ||
| percpu-km.c | ||
| percpu-stats.c | ||
| percpu-vm.c | ||
| percpu.c | ||
| pgalloc-track.h | ||
| pgtable-generic.c | ||
| process_vm_access.c | ||
| ptdump.c | ||
| readahead.c | ||
| rmap.c | ||
| rodata_test.c | ||
| secretmem.c | ||
| shmem_quota.c | ||
| shmem.c | ||
| show_mem.c | ||
| shrinker_debug.c | ||
| shrinker.c | ||
| shuffle.c | ||
| shuffle.h | ||
| slab_common.c | ||
| slab.h | ||
| slub.c | ||
| sparse-vmemmap.c | ||
| sparse.c | ||
| swap_cgroup.c | ||
| swap_slots.c | ||
| swap_state.c | ||
| swap.c | ||
| swap.h | ||
| swapfile.c | ||
| truncate.c | ||
| usercopy.c | ||
| userfaultfd.c | ||
| util.c | ||
| vmalloc.c | ||
| vmpressure.c | ||
| vmscan.c | ||
| vmstat.c | ||
| workingset.c | ||
| z3fold.c | ||
| zbud.c | ||
| zpool.c | ||
| zsmalloc.c | ||
| zswap.c | ||