Minki/linux
1
0
Fork 1
mirror of https://github.com/torvalds/linux.git synced 2024-12-25 12:21:37 +00:00
Code Issues Packages Projects Releases Wiki Activity
3403e56b41
linux/arch/arm64/include/asm
History
Alex Van Brunt 3403e56b41 arm64: mm: Don't wait for completion of TLB invalidation when page aging 
When transitioning a PTE from young to old as part of page aging, we
can avoid waiting for the TLB invalidation to complete and therefore
drop the subsequent DSB instruction. Whilst this opens up a race with
page reclaim, where a PTE in active use via a stale, young TLB entry
does not update the underlying descriptor, the worst thing that happens
is that the page is reclaimed and then immediately faulted back in.

Given that we have a DSB in our context-switch path, the window for a
spurious reclaim is fairly limited and eliding the barrier claims to
boost NVMe/SSD accesses by over 10% on some platforms.

A similar optimisation was made for x86 in commit b13b1d2d86 ("x86/mm:
In the PTE swapout page reclaim case clear the accessed bit instead of
flushing the TLB").

Signed-off-by: Alex Van Brunt <avanbrunt@nvidia.com>
Signed-off-by: Ashish Mhetre <amhetre@nvidia.com>
[will: rewrote patch]
Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-11-26 16:59:46 +00:00
..
xen arm64: xen: Use existing helper to check interrupt status 2018-10-03 16:14:11 +01:00
acenv.h
acpi.h arm64: acpi: fix alignment fault in accessing ACPI 2018-07-23 15:34:12 +01:00
alternative.h arm64: Avoid flush_icache_range() in alternatives patching code 2018-06-27 18:21:53 +01:00
arch_gicv3.h irqchip/gic-v3: Probe for SCR_EL3 being clear before resetting AP0Rn 2018-03-22 13:46:18 +00:00
arch_timer.h First batch of KVM changes for 4.15 2017-11-16 13:00:24 -08:00
arm_dsu_pmu.h perf: ARM DynamIQ Shared Unit PMU support 2018-01-02 16:43:12 +00:00
arm-cci.h
asm-bug.h arm64: asm-bug: Renumber macro local labels to avoid clashes 2017-10-25 15:57:15 +01:00
asm-offsets.h
asm-uaccess.h arm64: mm: Add additional parameter to uaccess_ttbr0_disable 2018-01-17 13:57:49 +01:00
assembler.h arm64: cpu_errata: Remove ARM64_MISMATCHED_CACHE_LINE_SIZE 2018-09-19 18:21:49 +01:00
atomic_ll_sc.h
atomic_lse.h arm64: lse: Add early clobbers to some input/output asm operands 2018-05-21 19:00:36 +01:00
atomic.h atomics/treewide: Make conditional inc/dec ops optional 2018-06-21 14:25:24 +02:00
barrier.h arm64: barrier: Implement smp_cond_load_relaxed 2018-07-05 10:05:05 +01:00
bitops.h locking/atomics/arm64, arm64/bitops: Include <asm-generic/bitops/ext2-atomic-setbit.h> 2018-06-21 12:52:12 +02:00
bitrev.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
boot.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
brk-imm.h
bug.h
cache.h arm64: cpufeature: Fix handling of CTR_EL0.IDC field 2018-10-16 11:53:31 +01:00
cacheflush.h arm64: IPI each CPU after invalidating the I-cache for kernel mappings 2018-07-05 17:24:36 +01:00
checksum.h
clocksource.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cmpxchg.h arm64: cmpwait: Clear event register before arming exclusive monitor 2018-05-16 12:21:19 +01:00
compat.h Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-10-25 11:14:36 -07:00
cpu_ops.h
cpu.h arm64/sve: Probe SVE capabilities and usable vector lengths 2017-11-03 15:24:17 +00:00
cpucaps.h arm64: arch_timer: Add workaround for ARM erratum 1188873 2018-10-01 13:38:47 +01:00
cpufeature.h KVM updates for v4.20 2018-10-25 17:57:35 -07:00
cpuidle.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cputype.h arm64: arch_timer: Add workaround for ARM erratum 1188873 2018-10-01 13:38:47 +01:00
current.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
daifflags.h arm64: daifflags: Use irqflags functions for daifflags 2018-10-03 16:08:52 +01:00
dcc.h
debug-monitors.h
device.h arm64: use the generic swiotlb_dma_ops 2018-10-19 08:53:24 +02:00
dma-mapping.h arm64: use the generic swiotlb_dma_ops 2018-10-19 08:53:24 +02:00
dmi.h
efi.h efi: Deduplicate efi_open_volume() 2018-07-22 14:13:43 +02:00
elf.h arm64: signal: Report signal frame size to userspace via auxv 2018-06-01 15:53:10 +01:00
esr.h arm64: Trap WFI executed in userspace 2018-10-01 16:52:24 +01:00
exception.h arm64: kernel: Prepare for a DISR user 2018-01-16 15:07:12 +00:00
exec.h
extable.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fb.h
fixmap.h arm64: kaslr: Put kernel vectors address in separate data page 2017-12-11 13:41:20 +00:00
fpsimd.h arm64: move sve_user_{enable,disable} to <asm/fpsimd.h> 2018-07-12 14:40:39 +01:00
fpsimdmacros.h arm64/sve: Write ZCR_EL1 on context switch only if changed 2018-05-17 18:19:53 +01:00
ftrace.h
futex.h arm64: futex: Mask __user pointers prior to dereference 2018-02-06 22:53:42 +00:00
hardirq.h
hugetlb.h hugetlb: introduce generic version of huge_ptep_get 2018-10-26 16:26:34 -07:00
hw_breakpoint.h perf/hw_breakpoint: Remove default hw_breakpoint_arch_parse() 2018-06-26 09:07:58 +02:00
hwcap.h
hypervisor.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
insn.h arm64: insn: Don't fallback on nosync path for general insn patching 2018-07-05 17:24:48 +01:00
io.h xen: don't include <xen/xen.h> from <asm/io.h> and <asm/dma-mapping.h> 2018-09-26 08:45:18 -06:00
irq_work.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
irq.h arm64: Use the new GENERIC_IRQ_MULTI_HANDLER 2018-08-03 12:14:09 +02:00
irqflags.h arm64: Move the async/fiq helpers to explicitly set process context flags 2017-11-02 15:55:41 +00:00
jump_label.h arm64/kernel: jump_label: Switch to relative references 2018-09-27 17:56:47 +02:00
kasan.h kasan: clean up KASAN_SHADOW_SCALE_SHIFT usage 2018-02-06 18:32:43 -08:00
Kbuild arm64: locking: Replace ticket lock implementation with qspinlock 2018-07-05 10:05:06 +01:00
kernel-pgtable.h arm64/mm: Separate boot-time page tables from swapper_pg_dir 2018-09-25 15:10:54 +01:00
kexec.h
kgdb.h
kprobes.h arm64/kprobes: Remove jprobe implementation 2018-06-21 12:33:07 +02:00
kvm_arm.h KVM updates for v4.20 2018-10-25 17:57:35 -07:00
kvm_asm.h KVM: arm64: Safety check PSTATE when entering guest and handle IL 2018-10-19 11:13:03 +01:00
kvm_coproc.h
kvm_emulate.h arm64/cpufeatures: Introduce ESR_ELx_SYS64_ISS_RT() 2018-09-21 11:05:25 +01:00
kvm_host.h KVM updates for v4.20 2018-10-25 17:57:35 -07:00
kvm_hyp.h kvm: arm64: Configure VTCR_EL2 per VM 2018-10-01 13:50:29 +01:00
kvm_mmio.h
kvm_mmu.h KVM updates for v4.20 2018-10-25 17:57:35 -07:00
linkage.h
lse.h arm64: lse: Include compiler_types.h and export.h for out-of-line LL/SC 2018-03-27 13:14:49 +01:00
memblock.h
memory.h arm64: KVM: Add support for Stage-2 control of memory types and cacheability 2018-07-09 11:37:41 +01:00
mmu_context.h arm64: mm: apply r/o permissions of VM areas to its linear alias as well 2018-11-20 11:38:26 +00:00
mmu.h arm64/mm: Separate boot-time page tables from swapper_pg_dir 2018-09-25 15:10:54 +01:00
mmzone.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
module.h arm64/module: use plt section indices for relocations 2018-11-20 11:38:26 +00:00
neon.h arm64: drop unused kernel_neon_begin_partial() macro 2018-07-31 10:13:50 +01:00
numa.h arm64: numa: separate out updates to percpu nodeid and NUMA node cpumap 2018-07-06 13:18:18 +01:00
page-def.h
page.h arm64: Kconfig: Remove ARCH_HAS_HOLES_MEMORYMODEL 2018-09-21 12:02:45 +01:00
paravirt.h x86/paravirt: Use a single ops structure 2018-09-03 16:50:35 +02:00
pci.h PCI: remove PCI_DMA_BUS_IS_PHYS 2018-05-07 07:15:41 +02:00
percpu.h arm64: percpu: Initialize ret in the default case 2018-09-25 13:26:48 -07:00
perf_event.h arm64/bpf: correct broken uapi for BPF_PROG_TYPE_PERF_EVENT program type 2017-12-05 15:02:41 +01:00
pgalloc.h arm64: mm: Use READ_ONCE/WRITE_ONCE when accessing page tables 2018-02-16 18:13:57 +00:00
pgtable-hwdef.h arm64: mm: Support Common Not Private translations 2018-09-18 12:02:27 +01:00
pgtable-prot.h arm64: KVM: Avoid marking pages as XN in Stage-2 if CTR_EL0.DIC is set 2018-07-09 11:37:42 +01:00
pgtable-types.h
pgtable.h arm64: mm: Don't wait for completion of TLB invalidation when page aging 2018-11-26 16:59:46 +00:00
probes.h
proc-fns.h arm64: mm: Move ASID from TTBR0 to TTBR1 2017-12-11 13:40:25 +00:00
processor.h arm64: mm: define NET_IP_ALIGN to 0 2018-11-08 17:50:26 +00:00
ptdump.h
ptrace.h KVM updates for v4.20 2018-10-25 17:57:35 -07:00
sdei.h arm64: Add stack information to on_accessible_stack 2018-07-26 11:36:07 +01:00
seccomp.h
sections.h arm64: mmu: add the entry trampolines start/end section markers into sections.h 2018-01-14 18:49:50 +00:00
shmparam.h
signal32.h
simd.h arm64: neon: Fix function may_use_simd() return error status 2018-07-11 17:02:02 +01:00
smp_plat.h
smp.h
sparsemem.h arm64: add kconfig symbol to configure physical address size 2017-12-22 17:30:33 +00:00
spinlock_types.h arm64: locking: Replace ticket lock implementation with qspinlock 2018-07-05 10:05:06 +01:00
spinlock.h arm64: locking: Replace ticket lock implementation with qspinlock 2018-07-05 10:05:06 +01:00
stack_pointer.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
stackprotector.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
stacktrace.h arm64: Add stack information to on_accessible_stack 2018-07-26 11:36:07 +01:00
stage2_pgtable.h kvm: arm64: Allow tuning the physical address size for VM 2018-10-03 11:45:20 +01:00
stat.h y2038: globally rename compat_time to old_time32 2018-08-27 14:48:48 +02:00
string.h arm64: lib: use C string functions with KASAN enabled 2018-10-26 16:25:18 -07:00
suspend.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sync_bitops.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
syscall_wrapper.h arm64: implement syscall wrappers 2018-07-12 14:49:48 +01:00
syscall.h arm64: implement syscall wrappers 2018-07-12 14:49:48 +01:00
sysreg.h arm64: Trap WFI executed in userspace 2018-10-01 16:52:24 +01:00
system_misc.h signal/arm64: Push siginfo generation into arm64_notify_die 2018-09-27 21:52:54 +02:00
thread_info.h Small update for KVM. 2018-06-12 11:34:04 -07:00
timex.h arm64: Use physical counter for in-kernel reads when booted in EL2 2017-11-06 16:23:09 +01:00
tlb.h arm64: tlb: Avoid synchronous TLBIs when freeing page tables 2018-09-11 16:49:12 +01:00
tlbflush.h arm64: mm: Don't wait for completion of TLB invalidation when page aging 2018-11-26 16:59:46 +00:00
topology.h sched/topology, arch/arm64: Rebuild the sched_domain hierarchy when the CPU capacity changes 2018-09-10 11:05:47 +02:00
traps.h signal/arm64: Add and use arm64_force_sig_ptrace_errno_trap 2018-09-27 21:55:15 +02:00
uaccess.h Revert "arm64: uaccess: implement unsafe accessors" 2018-10-10 17:52:08 +01:00
unistd32.h arm64: convert compat wrappers to C 2018-07-12 14:49:48 +01:00
unistd.h y2038: utimes: Rework #ifdef guards for compat syscalls 2018-08-29 15:42:23 +02:00
uprobes.h
vdso_datapage.h
vdso.h
virt.h arm64: capabilities: Change scope of VHE to Boot CPU feature 2018-03-26 18:01:41 +01:00
vmap_stack.h arm64: Add vmap_stack header file 2018-01-13 10:45:03 +00:00
word-at-a-time.h