linux/mm
Pavel Tatashin 3a2d7fa8a3 mm: disable interrupts while initializing deferred pages
Vlastimil Babka reported about a window issue during which when deferred
pages are initialized, and the current version of on-demand
initialization is finished, allocations may fail.  While this is highly
unlikely scenario, since this kind of allocation request must be large,
and must come from interrupt handler, we still want to cover it.

We solve this by initializing deferred pages with interrupts disabled,
and holding node_size_lock spin lock while pages in the node are being
initialized.  The on-demand deferred page initialization that comes
later will use the same lock, and thus synchronize with
deferred_init_memmap().

It is unlikely for threads that initialize deferred pages to be
interrupted.  They run soon after smp_init(), but before modules are
initialized, and long before user space programs.  This is why there is
no adverse effect of having these threads running with interrupts
disabled.

[pasha.tatashin@oracle.com: v6]
  Link: http://lkml.kernel.org/r/20180313182355.17669-2-pasha.tatashin@oracle.com
Link: http://lkml.kernel.org/r/20180309220807.24961-2-pasha.tatashin@oracle.com
Signed-off-by: Pavel Tatashin <pasha.tatashin@oracle.com>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Steven Sistare <steven.sistare@oracle.com>
Cc: Daniel Jordan <daniel.m.jordan@oracle.com>
Cc: Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: AKASHI Takahiro <takahiro.akashi@linaro.org>
Cc: Gioh Kim <gi-oh.kim@profitbricks.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Yaowei Bai <baiyaowei@cmss.chinamobile.com>
Cc: Wei Yang <richard.weiyang@gmail.com>
Cc: Paul Burton <paul.burton@mips.com>
Cc: Miles Chen <miles.chen@mediatek.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-04-05 21:36:24 -07:00
..
kasan slab, slub: skip unnecessary kasan_cache_shutdown() 2018-04-05 21:36:24 -07:00
backing-dev.c Revert "bdi: add error handle for bdi_debug_register" 2017-12-21 10:01:30 -07:00
balloon_compaction.c virtio_balloon: fix deadlock on OOM 2017-11-14 23:57:38 +02:00
bootmem.c mm: docs: fix parameter names mismatch 2018-02-06 18:32:48 -08:00
cleancache.c
cma_debug.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cma.c mm/cma.c: change pr_info to pr_err for cma_alloc fail log 2017-11-15 18:21:03 -08:00
cma.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
compaction.c mm/compaction.c: fix comment for try_to_compact_pages() 2018-01-31 17:18:39 -08:00
debug_page_ref.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
debug.c mm/debug.c: provide useful debugging information for VM_BUG 2018-01-04 16:45:09 -08:00
dmapool.c
early_ioremap.c mm/early_ioremap: Fix boot hang with earlyprintk=efi,keep 2017-12-11 14:54:44 +01:00
fadvise.c mm: add ksys_fadvise64_64() helper; remove in-kernel call to sys_fadvise64_64() 2018-04-02 20:16:10 +02:00
failslab.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
filemap.c mm/filemap.c: remove include of hardirq.h 2018-01-31 17:18:36 -08:00
frame_vector.c mm/frame_vector.c: release a semaphore in 'get_vaddr_frames()' 2017-12-14 16:00:48 -08:00
frontswap.c
gup_benchmark.c mm: add infrastructure for get_user_pages_fast() benchmarking 2017-11-17 16:10:04 -08:00
gup.c mm/gup.c: teach get_user_pages_unlocked to handle FOLL_NOWAIT 2018-03-09 16:40:01 -08:00
highmem.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hmm.c libnvdimm for 4.16 2018-02-06 10:41:33 -08:00
huge_memory.c mm, thp: do not cause memcg oom for thp 2018-03-22 17:07:02 -07:00
hugetlb_cgroup.c
hugetlb.c hugetlbfs: check for pgoff value overflow 2018-03-22 17:07:01 -07:00
hwpoison-inject.c mm/memory_failure: Remove unused trapno from memory_failure 2018-01-23 12:17:42 -06:00
init-mm.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
internal.h Revert "mm, thp: Do not make pmd/pud dirty without a reason" 2017-11-29 09:01:01 -08:00
interval_tree.c mm/interval_tree.c: use vma_pages() helper 2018-01-31 17:18:37 -08:00
Kconfig treewide: simplify Kconfig dependencies for removed archs 2018-03-26 15:55:57 +02:00
Kconfig.debug kmemcheck: rip it out 2017-11-15 18:21:05 -08:00
khugepaged.c mm, thp: do not cause memcg oom for thp 2018-03-22 17:07:02 -07:00
kmemleak-test.c
kmemleak.c mm/kmemleak.c: wait for scan completion before disabling free 2018-03-28 13:42:05 -10:00
ksm.c mm/ksm.c: make stable_node_dup() static 2018-04-05 21:36:24 -07:00
list_lru.c mm/list_lru.c: mark expected switch fall-through 2017-11-15 18:21:07 -08:00
maccess.c mm: docs: fix parameter names mismatch 2018-02-06 18:32:48 -08:00
madvise.c mm/memory_failure: Remove unused trapno from memory_failure 2018-01-23 12:17:42 -06:00
Makefile mm/swap_slots.c: use conditional compilation 2018-04-05 21:36:24 -07:00
memblock.c Revert "mm: page_alloc: skip over regions of invalid pfns where possible" 2018-03-22 17:07:01 -07:00
memcontrol.c mm/memcontrol.c: fix parameter description mismatch 2018-03-28 13:42:05 -10:00
memory_hotplug.c libnvdimm for 4.16 2018-02-06 10:41:33 -08:00
memory-failure.c x86/mm, mm/hwpoison: Don't unconditionally unmap kernel 1:1 pages 2018-02-13 16:25:06 +01:00
memory.c mm, swap: Add infrastructure for saving page metadata on swap 2018-03-18 07:38:45 -07:00
mempolicy.c Merge branch 'syscalls-next' of git://git.kernel.org/pub/scm/linux/kernel/git/brodo/linux 2018-04-02 21:22:12 -07:00
mempool.c kasan: detect invalid frees for large mempool objects 2018-02-06 18:32:43 -08:00
memtest.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
migrate.c mm: add kernel_move_pages() helper, move compat syscall to mm/migrate.c 2018-04-02 20:15:32 +02:00
mincore.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
mlock.c mm, mlock, vmscan: no more skipping pagevecs 2018-02-21 15:35:42 -08:00
mm_init.c
mmap.c mm: always print RLIMIT_DATA warning 2018-04-05 21:36:24 -07:00
mmu_context.c
mmu_notifier.c mm, mmu_notifier: annotate mmu notifiers with blockable invalidate callbacks 2018-01-31 17:18:38 -08:00
mmzone.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
mprotect.c mm: Clear arch specific VM flags on protection change 2018-03-18 07:38:47 -07:00
mremap.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
msync.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
nobootmem.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
nommu.c Merge branch 'syscalls-next' of git://git.kernel.org/pub/scm/linux/kernel/git/brodo/linux 2018-04-02 21:22:12 -07:00
oom_kill.c mm, oom: avoid reaping only for mm's with blockable invalidate callbacks 2018-01-31 17:18:38 -08:00
page_alloc.c mm: disable interrupts while initializing deferred pages 2018-04-05 21:36:24 -07:00
page_counter.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
page_ext.c mm/page_ext.c: make page_ext_init a noop when CONFIG_PAGE_EXTENSION but nothing uses it 2018-01-31 17:18:39 -08:00
page_idle.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
page_io.c block: convert to bio_first_bvec_all & bio_first_page_all 2018-01-06 09:18:00 -07:00
page_isolation.c mm: distinguish CMA and MOVABLE isolation in has_unmovable_pages() 2017-11-15 18:21:02 -08:00
page_owner.c mm/page_owner: fix recursion bug after changing skip entries 2018-03-28 13:42:05 -10:00
page_poison.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
page_vma_mapped.c mm, page_vma_mapped: Introduce pfn_in_hpage() 2018-01-22 12:15:57 -08:00
page-writeback.c Revert "mm/page-writeback.c: print a warning if the vm dirtiness settings are illogical" 2017-11-29 18:40:43 -08:00
pagewalk.c mm: docs: add blank lines to silence sphinx "Unexpected indentation" errors 2018-02-06 18:32:48 -08:00
percpu-internal.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
percpu-km.c percpu: allow select gfp to be passed to underlying allocators 2018-02-18 05:33:01 -08:00
percpu-stats.c percpu: fix starting offset for chunk statistics traversal 2017-09-27 14:45:57 -07:00
percpu-vm.c percpu: allow select gfp to be passed to underlying allocators 2018-02-18 05:33:01 -08:00
percpu.c arch: remove obsolete architecture ports 2018-04-02 20:20:12 -07:00
pgtable-generic.c mm: do not lose dirty and accessed bits in pmdp_invalidate() 2018-01-31 17:18:38 -08:00
process_vm_access.c mm: docs: add blank lines to silence sphinx "Unexpected indentation" errors 2018-02-06 18:32:48 -08:00
quicklist.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
readahead.c mm: add ksys_readahead() helper; remove in-kernel calls to sys_readahead() 2018-04-02 20:16:12 +02:00
rmap.c mm, swap: Add infrastructure for saving page metadata on swap 2018-03-18 07:38:45 -07:00
rodata_test.c mm: fix RODATA_TEST failure "rodata_test: test data was not read only" 2017-10-03 17:54:24 -07:00
shmem.c mm/shmem: do not wait for lock_page() in shmem_unused_huge_shrink() 2018-03-22 17:07:01 -07:00
slab_common.c mm/slab_common.c: remove test if cache name is accessible 2018-04-05 21:36:24 -07:00
slab.c slab, slub: skip unnecessary kasan_cache_shutdown() 2018-04-05 21:36:24 -07:00
slab.h slab, slub: skip unnecessary kasan_cache_shutdown() 2018-04-05 21:36:24 -07:00
slob.c slab, slub, slob: add slab_flags_t 2017-11-15 18:21:01 -08:00
slub.c slab, slub: skip unnecessary kasan_cache_shutdown() 2018-04-05 21:36:24 -07:00
sparse-vmemmap.c mm: merge vmem_altmap_alloc into altmap_alloc_block_buf 2018-01-08 11:46:23 -08:00
sparse.c arch: remove obsolete architecture ports 2018-04-02 20:20:12 -07:00
swap_cgroup.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
swap_slots.c mm/swap_slots.c: use conditional compilation 2018-04-05 21:36:24 -07:00
swap_state.c mm: remove cold parameter for release_pages 2017-11-15 18:21:06 -08:00
swap.c mm/swap.c: make functions and their kernel-doc agree (again) 2018-02-21 15:35:43 -08:00
swapfile.c vfs: do bulk POLL* -> EPOLL* replacement 2018-02-11 14:34:03 -08:00
truncate.c mm: add unmap_mapping_pages() 2018-01-31 17:18:37 -08:00
usercopy.c usercopy: WARN() on slab cache usercopy region violations 2018-01-15 12:07:48 -08:00
userfaultfd.c mm/userfaultfd.c: remove duplicate include 2018-02-06 18:32:47 -08:00
util.c new primitive: vmemdup_user() 2018-01-07 13:06:15 -05:00
vmacache.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
vmalloc.c vmalloc: fix __GFP_HIGHMEM usage for vmalloc_32 on 32b systems 2018-02-21 15:35:43 -08:00
vmpressure.c
vmscan.c mm/vmscan: wake up flushers for legacy cgroups too 2018-03-22 17:07:01 -07:00
vmstat.c mm/vmstat.c: fix vmstat_update() preemption BUG 2018-03-28 13:42:05 -10:00
workingset.c mm, truncate: do not check mapping for every page being truncated 2017-11-15 18:21:06 -08:00
z3fold.c mm: docs: fix parameter names mismatch 2018-02-06 18:32:48 -08:00
zbud.c mm: docs: fix parameter names mismatch 2018-02-06 18:32:48 -08:00
zpool.c mm/zpool.c: zpool_evictable: fix mismatch in parameter name and kernel-doc 2018-02-21 15:35:43 -08:00
zsmalloc.c mm/zsmalloc: Prepare to variable MAX_PHYSMEM_BITS 2018-02-14 13:11:13 +01:00
zswap.c mm, swap, frontswap: fix THP swap if frontswap enabled 2018-02-21 15:35:43 -08:00