linux/arch/loongarch/include/asm
Baolin Wang 55d2a0bd5e mm: add statistics for PUD level pagetable
Recently, we found that cross-die access to pagetable pages on ARM64
machines can cause performance fluctuations in our business.  Currently,
there are no PMU events available to track this situation on our ARM64
machines, so accurate pagetable accounting can help to analyze this issue,
but now the PUD level pagetable accounting is missed.

So introduce pagetable_pud_ctor/dtor() to help to get accurate PUD
pagetable accounting, as well as converting the architectures which use
generic PUD pagetable allocation to add corresponding PUD pagetable
accounting.  Moreover this patch will mark the PUD level pagetable with
PG_table flag, which will help to do sanity validation in
unpoison_memory().

On my testing machine, I can see more pagetables statistics after the patch
with page-types tool:

Before patch:
        flags           page-count      MB  symbolic-flags                     long-symbolic-flags
0x0000000004000000           27326      106  __________________________g_________________       pgtable
After patch:
0x0000000004000000           27541      107  __________________________g_________________       pgtable

Link: https://lkml.kernel.org/r/876c71c03a7e69c17722a690e3225a4f7b172fb2.1695017383.git.baolin.wang@linux.alibaba.com
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Acked-by: Mike Rapoport (IBM) <rppt@kernel.org>
Acked-by: Vishal Moola (Oracle) <vishal.moola@gmail.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Huacai Chen <chenhuacai@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-10-06 14:44:10 -07:00
..
vdso LoongArch: Add support to clone a time namespace 2023-06-29 20:58:43 +08:00
acenv.h LoongArch: Add boot and setup routines 2022-06-03 20:09:28 +08:00
acpi.h LoongArch: Export some arch-specific pm interfaces 2023-06-29 20:58:44 +08:00
addrspace.h LoongArch: Use _UL() and _ULL() 2023-09-20 14:26:29 +08:00
alternative-asm.h LoongArch: Add alternative runtime patching mechanism 2022-12-14 08:36:11 +08:00
alternative.h LoongArch: Add alternative runtime patching mechanism 2022-12-14 08:36:11 +08:00
asm-extable.h LoongArch: BPF: Add BPF exception tables 2022-12-14 08:36:11 +08:00
asm-offsets.h LoongArch: Add misc common routines 2022-06-03 20:09:28 +08:00
asm-prototypes.h LoongArch: Add Loongson Binary Translation (LBT) extension support 2023-09-06 22:53:55 +08:00
asm.h LoongArch: Mark some assembler symbols as non-kprobe-able 2023-02-25 22:12:17 +08:00
asmmacro.h LoongArch: Add Loongson Binary Translation (LBT) extension support 2023-09-06 22:53:55 +08:00
atomic.h locking/atomic: treewide: delete arch_atomic_*() kerneldoc 2023-06-05 09:57:24 +02:00
barrier.h LoongArch: Support dbar with different hints 2023-06-29 20:58:44 +08:00
bitops.h LoongArch: Add atomic/locking headers 2022-06-03 20:09:28 +08:00
bitrev.h LoongArch: Add atomic/locking headers 2022-06-03 20:09:28 +08:00
bootinfo.h LoongArch: Clean up plat_swiotlb_setup() related code 2023-04-19 12:07:27 +08:00
branch.h LoongArch: Make compute_return_era() return void 2022-06-25 18:06:07 +08:00
bug.h LoongArch: Use generic BUG() handler 2022-10-12 16:36:19 +08:00
cache.h LoongArch: Add memory management 2022-06-03 20:09:28 +08:00
cacheflush.h mm: rationalise flush_icache_pages() and flush_icache_page() 2023-08-24 16:20:25 -07:00
cacheops.h LoongArch: Refactor cache probe and flush methods 2022-10-12 16:36:14 +08:00
checksum.h LoongArch: Add checksum optimization for 64-bit system 2023-05-01 17:19:43 +08:00
clocksource.h LoongArch: Add other common headers 2022-06-03 20:09:28 +08:00
cmpxchg.h locking/arch: Rename all internal __xchg() names to __arch_xchg() 2023-04-29 09:08:44 +02:00
cpu-features.h LoongArch: Introduce hardware page table walker 2023-06-29 20:58:44 +08:00
cpu-info.h LoongArch: Add SMT (Simultaneous Multi-Threading) support 2023-06-29 20:58:43 +08:00
cpu.h LoongArch: Introduce hardware page table walker 2023-06-29 20:58:44 +08:00
cpufeature.h LoongArch: Add ELF and module support 2022-06-03 20:09:28 +08:00
delay.h LoongArch: Add some library functions 2022-06-03 20:09:28 +08:00
dma-direct.h LoongArch: Add Non-Uniform Memory Access (NUMA) support 2022-06-03 20:09:29 +08:00
dma.h LoongArch: Add PCI controller support 2022-08-12 13:10:11 +08:00
dmi.h LoongArch: Add boot and setup routines 2022-06-03 20:09:28 +08:00
efi.h LoongArch changes for v6.2 2022-12-19 08:23:27 -06:00
elf.h LoongArch: Define relocation types for ABI v2.10 2023-09-27 16:19:13 +08:00
entry-common.h LoongArch: Add exception/interrupt handling 2022-06-03 20:09:28 +08:00
exception.h LoongArch: Fix some build warnings with W=1 2023-09-20 14:26:28 +08:00
exec.h LoongArch: Add ELF and module support 2022-06-03 20:09:28 +08:00
extable.h LoongArch: BPF: Add BPF exception tables 2022-12-14 08:36:11 +08:00
fb.h fbdev: Rename fb_mem*() helpers 2023-05-18 11:07:54 +02:00
fixmap.h LoongArch: Use TLB for ioremap() 2022-10-12 16:36:14 +08:00
fpregdef.h LoongArch: Prepare for assemblers with proper FCSR class support 2023-06-29 20:58:43 +08:00
fpu.h LoongArch: Ensure FP/SIMD registers in the core dump file is up to date 2023-08-26 22:21:57 +08:00
ftrace.h LoongArch: ftrace: Enable HAVE_FUNCTION_GRAPH_RETVAL 2023-06-20 18:38:38 -04:00
futex.h LoongArch: extable: Add a dedicated uaccess handler 2022-12-14 08:36:11 +08:00
gpr-num.h LoongArch: extable: Also recognize ABI names of registers 2023-06-29 20:58:43 +08:00
hardirq.h LoongArch: Fix the !CONFIG_SMP build 2022-06-08 11:00:40 +08:00
hugetlb.h LoongArch: Add memory management 2022-06-03 20:09:28 +08:00
hw_breakpoint.h LoongArch: Add hardware breakpoints/watchpoints support 2023-02-25 22:12:17 +08:00
hw_irq.h LoongArch: Add exception/interrupt handling 2022-06-03 20:09:28 +08:00
idle.h LoongArch: Add process management 2022-06-03 20:09:28 +08:00
inst.h LoongArch: Add larch_insn_gen_break() to generate break insns 2023-06-29 20:58:44 +08:00
io.h asm-generic/iomap.h: remove ARCH_HAS_IOREMAP_xx macros 2023-08-18 10:12:32 -07:00
irq_regs.h LoongArch: Add exception/interrupt handling 2022-06-03 20:09:28 +08:00
irq.h nmi_backtrace: allow excluding an arbitrary CPU 2023-08-18 10:19:00 -07:00
irqflags.h LoongArch: Remove useless header compiler.h 2022-07-29 18:22:32 +08:00
jump_label.h LoongArch: Add jump-label implementation 2023-06-29 20:58:44 +08:00
kasan.h LoongArch: Don't inline kasan_mem_to_shadow()/kasan_shadow_to_mem() 2023-09-20 14:26:29 +08:00
Kbuild LoongArch: Remove <asm/export.h> 2023-08-25 23:40:26 +08:00
kdebug.h LoongArch: Remove five DIE_* definitions in kdebug.h 2023-06-29 20:58:44 +08:00
kexec.h LoongArch: Add kexec support 2022-10-12 16:36:19 +08:00
kfence.h LoongArch: Add KFENCE (Kernel Electric-Fence) support 2023-09-06 22:54:16 +08:00
kgdb.h LoongArch: Add basic KGDB & KDB support 2023-09-06 22:53:55 +08:00
kprobes.h LoongArch: Use larch_insn_gen_break() for kprobes 2023-06-29 20:58:44 +08:00
lbt.h LoongArch: Add Loongson Binary Translation (LBT) extension support 2023-09-06 22:53:55 +08:00
linkage.h LoongArch: Add other common headers 2022-06-03 20:09:28 +08:00
local.h locking/arch: Avoid variable shadowing in local_try_cmpxchg() 2023-07-10 09:52:36 +02:00
loongarch.h LoongArch: Add Loongson Binary Translation (LBT) extension support 2023-09-06 22:53:55 +08:00
loongson.h LoongArch: Add suspend (ACPI S3) support 2022-12-14 08:41:53 +08:00
mmu_context.h LoongArch: Add process management 2022-06-03 20:09:28 +08:00
mmu.h LoongArch: Add process management 2022-06-03 20:09:28 +08:00
mmzone.h LoongArch: Use static defined zero page rather than allocated 2023-09-06 22:53:10 +08:00
module.h LoongArch: Add guard for the larch_insn_gen_xxx functions 2023-06-29 20:58:42 +08:00
module.lds.h LoongArch: module: set section addresses to 0x0 2023-04-18 19:38:58 +08:00
numa.h LoongArch: Add Non-Uniform Memory Access (NUMA) support 2022-06-03 20:09:29 +08:00
page.h LoongArch: mm: Add page table mapped mode support for virt_to_page() 2023-09-06 22:53:55 +08:00
pci.h LoongArch: Add PCI controller support 2022-08-12 13:10:11 +08:00
percpu.h LoongArch: Tweak CFLAGS for Clang compatibility 2023-06-29 20:58:43 +08:00
perf_event.h LoongArch: Add perf events support 2022-10-12 16:36:14 +08:00
pgalloc.h mm: add statistics for PUD level pagetable 2023-10-06 14:44:10 -07:00
pgtable-bits.h loongarch: implement the new page table range API 2023-08-24 16:20:21 -07:00
pgtable.h LoongArch changes for v6.6 2023-09-08 12:16:52 -07:00
prefetch.h LoongArch: Add other common headers 2022-06-03 20:09:28 +08:00
processor.h LoongArch: Add Loongson Binary Translation (LBT) extension support 2023-09-06 22:53:55 +08:00
ptrace.h LoongArch: Add identifier names to arguments of die() declaration 2023-08-25 23:40:26 +08:00
qspinlock.h LoongArch: Support dbar with different hints 2023-06-29 20:58:44 +08:00
regdef.h
seccomp.h LoongArch: Add system call support 2022-06-03 20:09:28 +08:00
serial.h LoongArch: Add other common headers 2022-06-03 20:09:28 +08:00
setup.h LoongArch: Simplify the processing of jumping new kernel for KASLR 2023-09-06 22:54:16 +08:00
shmparam.h LoongArch: Add memory management 2022-06-03 20:09:28 +08:00
smp.h LoongArch: Fix some build warnings with W=1 2023-09-20 14:26:28 +08:00
sparsemem.h LoongArch: add sparse memory vmemmap support 2022-12-11 18:12:12 -08:00
spinlock_types.h LoongArch: Add qspinlock support 2022-10-12 16:36:14 +08:00
spinlock.h LoongArch: Add qspinlock support 2022-10-12 16:36:14 +08:00
stackframe.h LoongArch: Add basic KGDB & KDB support 2023-09-06 22:53:55 +08:00
stackprotector.h LoongArch: Add basic STACKPROTECTOR support 2022-12-14 08:41:53 +08:00
stacktrace.h LoongArch: Add USER_STACKTRACE support 2022-08-12 13:10:11 +08:00
string.h LoongArch: Add KASAN (Kernel Address Sanitizer) support 2023-09-06 22:54:16 +08:00
suspend.h LoongArch: Export some arch-specific pm interfaces 2023-06-29 20:58:44 +08:00
switch_to.h LoongArch: Add Loongson Binary Translation (LBT) extension support 2023-09-06 22:53:55 +08:00
syscall.h LoongArch: Add system call support 2022-06-03 20:09:28 +08:00
thread_info.h LoongArch: Add Loongson Binary Translation (LBT) extension support 2023-09-06 22:53:55 +08:00
time.h LoongArch: Add suspend (ACPI S3) support 2022-12-14 08:41:53 +08:00
timex.h LoongArch: Remove MIPS comment about cycle counter 2022-06-08 11:00:40 +08:00
tlb.h LoongArch: Introduce hardware page table walker 2023-06-29 20:58:44 +08:00
tlbflush.h LoongArch: Add multi-processor (SMP) support 2022-06-03 20:09:29 +08:00
topology.h LoongArch: Add Non-Uniform Memory Access (NUMA) support 2022-06-03 20:09:29 +08:00
types.h LoongArch: Add other common headers 2022-06-03 20:09:28 +08:00
uaccess.h LoongArch: Use la.pcrel instead of la.abs when it's trivially possible 2023-02-25 22:12:16 +08:00
unistd.h LoongArch: Add system call support 2022-06-03 20:09:28 +08:00
unwind.h LoongArch: Add generic ex-handler unwind in prologue unwinder 2023-01-17 11:42:16 +08:00
uprobes.h LoongArch: Add uprobes support 2023-06-29 20:58:44 +08:00
vdso.h LoongArch: Add vDSO syscall __vdso_getcpu() 2022-08-12 13:10:11 +08:00
vermagic.h LoongArch: Add ELF and module support 2022-06-03 20:09:28 +08:00
vmalloc.h LoongArch: Add memory management 2022-06-03 20:09:28 +08:00
xor_simd.h LoongArch: Add SIMD-optimized XOR routines 2023-09-06 22:53:55 +08:00
xor.h LoongArch: Add SIMD-optimized XOR routines 2023-09-06 22:53:55 +08:00