leandrof/linux
1
0
Fork 0
forked from Minki/linux
Code Issues Pull Requests Packages Projects Releases Wiki Activity
f0915781bd
linux/arch/arm/include/asm
History
Will Deacon f0915781bd ARM: tlb: don't perform inner-shareable invalidation for local TLB ops 
Inner-shareable TLB invalidation is typically more expensive than local
(non-shareable) invalidation, so performing the broadcasting for
local_flush_tlb_* operations is a waste of cycles and needlessly
clobbers entries in the TLBs of other CPUs.

This patch introduces __flush_tlb_* versions for many of the TLB
invalidation functions, which only respect inner-shareable variants of
the invalidation instructions when presented with the TLB_V7_UIS_FULL
flag. The local version is also inlined to prevent SMP_ON_UP kernels
from missing flushes, where the __flush variant would be called with
the UP flags.

This gains us around 0.5% in hackbench scores for a dual-core A15, but I
would expect this to improve as more cores (and clusters) are added to
the equation.

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reported-by: Albin Tonnerre <Albin.Tonnerre@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2013-08-12 12:25:44 +01:00
..
hardware reboot: arm: change reboot_mode to use enum reboot_mode 2013-07-09 10:33:29 -07:00
mach reboot: arm: change reboot_mode to use enum reboot_mode 2013-07-09 10:33:29 -07:00
xen xen/arm and xen/arm64: implement HYPERVISOR_tmem_op 2013-07-04 11:41:12 +00:00
arch_timer.h arm: delete __cpuinit/__CPUINIT usage from all ARM users 2013-07-14 19:36:52 -04:00
asm-offsets.h
assembler.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
atomic.h ARM: 7687/1: atomics: don't use exclusives for atomic64 read/set with LPAE 2013-04-03 17:00:00 +01:00
barrier.h ARM: kill off arch_is_coherent 2012-10-02 08:58:07 +02:00
bitops.h Disintegrate asm/system.h for ARM 2012-03-28 18:30:01 +01:00
bug.h Disintegrate asm/system.h for ARM 2012-03-28 18:30:01 +01:00
bugs.h
cache.h ARM: implement support for read-mostly sections 2010-12-05 08:39:36 +00:00
cacheflush.h ARM: 7755/1: handle user space mapped pages in flush_kernel_dcache_page 2013-06-17 10:30:52 +01:00
cachetype.h ARM: 7062/1: cache: detect PIPT I-cache using CTR 2011-10-17 09:13:41 +01:00
checksum.h
clkdev.h ARM: Consolidate the clkdev header files 2011-07-19 18:09:45 +02:00
cmpxchg.h ARM: 7720/1: ARM v6/v7 cmpxchg64 shouldn't clear upper 32 bits of the old/new value 2013-05-13 23:42:24 +01:00
compiler.h Disintegrate asm/system.h for ARM 2012-03-28 18:30:01 +01:00
cp15.h Merge branch 'for-linus' of git://git.linaro.org/people/rmk/linux-arm 2013-07-03 09:46:29 -07:00
cpu.h ARM: kernel: add MIDR to per-CPU information data 2012-11-19 14:51:11 +00:00
cpuidle.h cpuidle: Add common time keeping and irq enabling 2012-03-21 01:59:40 -04:00
cputype.h ARM: 7801/1: v6: prevent gcc 4.5 from reordering extended CP15 reads above is_smp() test 2013-07-31 11:12:59 +01:00
cti.h ARM: coresight: common definition for (OS) Lock Access Register key value 2013-01-10 21:13:05 +00:00
delay.h ARM: 7685/1: delay: use private ticks_per_jiffy field for timer-based delay ops 2013-04-03 16:45:50 +01:00
device.h ARM: dma-mapping: Add macro to_dma_iommu_mapping() 2013-02-25 15:30:41 +01:00
div64.h ARM: 7705/1: use optimized do_div only for EABI 2013-05-15 19:35:53 +01:00
dma-contiguous.h ARM: integrate CMA with DMA-mapping subsystem 2012-05-21 15:09:38 +02:00
dma-iommu.h ARM: dma-mapping: Add arm_iommu_detach_device() 2013-02-25 15:30:41 +01:00
dma-mapping.h IOMMU Updates for Linux v3.8 2012-12-20 10:07:25 -08:00
dma.h ARM: disable virt_to_bus/virt_to_bus almost everywhere 2013-02-14 15:04:33 +01:00
domain.h ARM: fix set_domain() macro 2012-07-05 09:50:55 +01:00
ecard.h ARM: io: ecard: move ioaddr() inside __ecard_address 2011-08-17 08:44:16 +01:00
edac.h ARM: 7201/1: add EDAC atomic_scrub function 2011-12-11 08:35:50 +00:00
elf.h ARM: fix nommu builds with 48be69a02 (ARM: move signal handlers into a vdso-like page) 2013-08-03 10:49:01 +01:00
entry-macro-multi.S ARM: gic: consolidate PPI handling 2011-10-23 13:32:29 +01:00
exception.h ARM: 7115/4: move __exception and friends to asm/exception.h 2011-10-17 09:02:44 +01:00
fb.h
fiq.h ARM: 6940/1: fiq: Briefly document driver responsibilities for suspend/resume 2011-05-26 10:31:06 +01:00
firmware.h ARM: Add interface for registering and calling firmware-specific operations 2013-04-09 01:52:06 +09:00
fixmap.h
flat.h ARM: binfmt_flat: unused variable 'persistent' 2012-10-09 20:29:06 +02:00
floppy.h
fncpy.h ARM: 6640/1: Thumb-2: Symbol manipulation macros for function body copying 2011-01-27 11:48:58 +00:00
fpstate.h Fix common misspellings 2011-03-31 11:26:23 -03:00
ftrace.h ARM: 6319/1: ftrace: add Thumb-2 support to dynamic ftrace 2010-09-02 15:28:43 +01:00
futex.h ARM: 7425/1: extable: ensure fixup entries are 4-byte aligned 2012-06-16 16:30:25 +01:00
glue-cache.h Merge branch 'for-next' of git://git.pengutronix.de/git/ukl/linux into devel-stable 2013-05-22 10:52:24 +01:00
glue-df.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
glue-pf.h ARM: move cache/processor/fault glue to separate include files 2011-02-12 11:52:21 +00:00
glue-proc.h ARM SoC specific changes 2013-07-02 13:43:38 -07:00
glue.h Fix common misspellings 2011-03-31 11:26:23 -03:00
gpio.h ARM: make mach/gpio.h headers optional 2012-09-14 09:21:59 -05:00
hardirq.h ARM: 7536/1: smp: Formalize an IPI for wakeup 2012-09-19 21:52:08 +01:00
highmem.h ARM: 7684/1: errata: Workaround for Cortex-A15 erratum 798181 (TLBI/DSB operations) 2013-04-03 16:45:49 +01:00
hugetlb-3level.h ARM: mm: HugeTLB support for LPAE systems. 2013-06-04 16:52:37 +01:00
hugetlb.h ARM: mm: HugeTLB support for LPAE systems. 2013-06-04 16:52:37 +01:00
hw_breakpoint.h ARM: hw_breakpoint: Check function for OS Save and Restore mechanism 2013-01-10 21:13:06 +00:00
hw_irq.h arm: dove: Use proper irq accessor functions 2011-03-29 14:47:57 +02:00
hwcap.h UAPI: (Scripted) Disintegrate arch/arm/include/asm 2012-10-12 13:05:52 +01:00
hypervisor.h arm: initial Xen support 2012-09-14 13:53:39 +00:00
ide.h
idmap.h ARM: KVM: move to a KVM provided HYP idmap 2013-04-28 22:23:08 -07:00
io.h ARM: 7728/1: mm: Use phys_addr_t properly for ioremap functions 2013-05-23 00:09:44 +01:00
irq.h arm: Move the set_handle_irq and handle_arch_irq declarations to asm/irq.h 2013-03-26 16:11:20 +00:00
irqflags.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
jump_label.h ARM: 7386/1: jump_label: fixup for rename to static_key 2012-04-15 22:00:31 +01:00
Kbuild Merge branch 'for-linus' of git://git.linaro.org/people/rmk/linux-arm 2012-12-12 11:30:02 -08:00
kexec.h [ARM] add machine-specific hook to machine_kexec 2011-03-03 16:26:55 -05:00
kgdb.h kgdb,arm: fix register dump 2010-10-29 13:14:40 -05:00
kmap_types.h arm: remove km_type definitions 2012-07-24 15:27:28 +08:00
kprobes.h Kernel: Audit Support For The ARM Platform 2012-01-17 16:17:01 -05:00
kvm_arm.h ARM: KVM: get rid of S2_PGD_SIZE 2013-06-26 10:50:04 -07:00
kvm_asm.h ARM: KVM: perform save/restore of PAR 2013-06-26 10:50:04 -07:00
kvm_coproc.h KVM: ARM: User space API for getting/setting co-proc registers 2013-01-23 13:29:14 -05:00
kvm_emulate.h ARM: KVM: don't special case PC when doing an MMIO 2013-06-26 10:50:03 -07:00
kvm_host.h arm/kvm: Cleanup KVM_ARM_MAX_VCPUS logic 2013-06-26 10:50:05 -07:00
kvm_mmio.h KVM: ARM: Handle I/O aborts 2013-01-23 13:29:17 -05:00
kvm_mmu.h ARM: KVM: perform HYP initilization for hotplugged CPUs 2013-04-28 22:23:11 -07:00
kvm_psci.h KVM: ARM: Power State Coordination Interface implementation 2013-01-23 13:29:18 -05:00
limits.h
linkage.h
localtimer.h ARM: local timers: make the runtime registration interface mandatory 2012-03-13 13:45:55 +00:00
mach-types.h
mc146818rtc.h ARM: mc146818rtc: remove unnecessary include of mach/irqs.h 2012-01-25 20:37:45 -06:00
mcpm.h ARM: mcpm: provide an interface to set the SMP ops at run time 2013-04-24 10:37:03 -04:00
memblock.h ARM: Add arm_memblock_steal() to allocate memory away from the kernel 2012-01-13 15:02:35 +00:00
memory.h mm/ARM: fix stale comment about VALID_PAGE() 2013-07-03 16:07:39 -07:00
mmu_context.h ARM: 7790/1: Fix deferred mm switch on VIVT processors 2013-07-26 12:02:09 +01:00
mmu.h Merge branch 'security-fixes' into fixes 2013-08-01 20:51:13 +01:00
module.h Make most arch asm/module.h files use asm-generic/module.h 2012-09-28 14:31:03 +09:30
mpu.h ARM: mpu: protect the vectors page with an MPU region 2013-06-17 15:13:18 +01:00
mtd-xip.h
mutex.h ARM: 7495/1: mutex: use generic atomic_dec-based implementation for ARMv6+ 2012-08-25 09:22:31 +01:00
nwflash.h
opcodes-sec.h ARM: opcodes: add opcodes definitions for ARM security extensions 2013-01-10 21:10:20 +00:00
opcodes-virt.h ARM: opcodes: add __ERET/__MSR_ELR_HYP instruction encoding 2012-09-19 08:32:49 +01:00
opcodes.h ARM: opcodes: add missing include of linux/linkage.h 2013-01-10 21:10:20 +00:00
outercache.h ARM: 7639/1: cache-l2x0: add missed dummy outer_resume entry 2013-02-06 09:33:07 +00:00
page-nommu.h
page.h ARM: make vectors page inaccessible from userspace 2013-08-01 14:31:58 +01:00
pci.h PCI: collapse pcibios_resource_to_bus 2012-02-23 20:19:04 -07:00
percpu.h ARM: 7747/1: pcpu: ensure __my_cpu_offset cannot be re-ordered across barrier() 2013-06-05 23:35:56 +01:00
perf_event.h ARM: 7584/1: perf: fix link error when CONFIG_HW_PERF_EVENTS is not selected 2012-11-23 13:01:30 +00:00
pgalloc.h ARM: LPAE: Page table maintenance for the 3-level format 2011-12-08 10:30:39 +00:00
pgtable-2level-hwdef.h ARM: 7077/1: LPAE: Use a mask for physical addresses in page table entries 2011-10-06 15:40:06 +01:00
pgtable-2level-types.h ARM: 7076/1: LPAE: Add (pte|pmd)val_t type definitions as u32 2011-10-06 15:40:05 +01:00
pgtable-2level.h ARM: mm: introduce present, faulting entries for PAGE_NONE 2012-11-09 14:13:20 +00:00
pgtable-3level-hwdef.h Merge branch 'for-rmk/lpae' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into devel-stable 2013-06-18 20:11:32 +01:00
pgtable-3level-types.h ARM: LPAE: Introduce the 3-level page table format definitions 2011-12-08 10:30:39 +00:00
pgtable-3level.h Merge branch 'for-rmk/lpae' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into devel-stable 2013-06-18 20:11:32 +01:00
pgtable-hwdef.h ARM: LPAE: Introduce the 3-level page table format definitions 2011-12-08 10:30:39 +00:00
pgtable-nommu.h consolidate io_remap_pfn_range definitions 2013-06-29 12:46:35 +04:00
pgtable.h Merge branch 'for-linus' of git://git.linaro.org/people/rmk/linux-arm 2013-07-03 09:46:29 -07:00
pmu.h ARM: perf: consistently use arm_pmu->name for PMU name 2012-11-09 11:37:26 +00:00
proc-fns.h Merge branch 'for-rmk/lpae' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into devel-stable 2013-06-18 20:11:32 +01:00
processor.h ARM: 7791/1: a.out: remove partial a.out support 2013-07-26 12:02:10 +01:00
procinfo.h
prom.h Merge branch 'for-linus' of git://git.linaro.org/people/rmk/linux-arm 2012-12-12 11:30:02 -08:00
psci.h arm: introduce psci_smp_ops 2013-05-21 14:24:11 +00:00
ptrace.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
scatterlist.h ARM: Allow SoCs to enable scatterlist chaining 2011-06-02 11:16:22 +01:00
sched_clock.h sched_clock: Add temporary asm/sched_clock.h 2013-06-21 10:17:44 -07:00
seccomp.h ARM: SECCOMP support 2010-10-01 22:32:18 -04:00
setup.h UAPI: (Scripted) Disintegrate arch/arm/include/asm 2012-10-12 13:05:52 +01:00
shmparam.h
signal.h arm: switch to generic old sigaction() 2013-02-03 18:15:47 -05:00
smp_plat.h Merge branch 'devel-stable' into for-next 2013-06-29 11:44:43 +01:00
smp_scu.h ARM: scu: provide inline dummy functions when SCU is not present 2013-07-04 05:10:51 -07:00
smp_twd.h ARM: smp_twd: convert to use CLKSRC_OF init 2013-03-11 08:42:08 -05:00
smp.h ARM: mpu: add MPU initialisation for secondary cores 2013-06-07 17:02:53 +01:00
sparsemem.h
spinlock_types.h ARM: 7446/1: spinlock: use ticket algorithm for ARMv6+ locking implementation 2012-07-09 17:41:10 +01:00
spinlock.h ARM: 7749/1: spinlock: retry trylock operation if strex fails on free lock 2013-06-17 09:27:04 +01:00
stackprotector.h
stacktrace.h
string.h
suspend.h ARM: kernel: implement stack pointer save array through MPIDR hashing 2013-06-20 11:24:11 +01:00
swab.h UAPI: (Scripted) Disintegrate arch/arm/include/asm 2012-10-12 13:05:52 +01:00
switch_to.h Disintegrate asm/system.h for ARM 2012-03-28 18:30:01 +01:00
sync_bitops.h xen/arm: sync_bitops 2012-08-08 16:34:01 +00:00
syscall.h ARM: 7577/1: arch/add syscall_get_arch 2012-11-19 14:14:17 +00:00
system_info.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
system_misc.h reboot: arm: change reboot_mode to use enum reboot_mode 2013-07-09 10:33:29 -07:00
system.h the only place that needs to include asm/exec.h is linux/binfmts.h 2012-09-20 09:51:13 -04:00
tcm.h ARM: 6985/1: export functions to determine the presence of I/DTCM 2011-07-06 20:49:45 +01:00
therm.h
thread_info.h ARM: 7790/1: Fix deferred mm switch on VIVT processors 2013-07-26 12:02:09 +01:00
thread_notify.h ARM: 6867/1: Introduce THREAD_NOTIFY_COPY for copy_thread() hooks 2011-04-10 21:13:36 +01:00
timex.h Merge branch 'for-linus' of git://git.linaro.org/people/rmk/linux-arm 2012-10-07 21:20:57 +09:00
tlb.h Merge branch 'devel-stable' into for-next 2013-06-29 11:44:43 +01:00
tlbflush.h ARM: tlb: don't perform inner-shareable invalidation for local TLB ops 2013-08-12 12:25:44 +01:00
tls.h ARM: 7735/2: Preserve the user r/w register TPIDRURW on context switch and fork 2013-06-24 15:21:59 +01:00
topology.h ARM: 7182/1: ARM cpu topology: fix warning 2011-11-30 23:55:21 +00:00
traps.h ARM: earlier initialization of vectors page 2012-01-23 10:24:11 +00:00
uaccess.h ARM: warnings in arch/arm/include/asm/uaccess.h 2012-10-09 20:29:07 +02:00
ucontext.h Fix common misspellings 2011-03-31 11:26:23 -03:00
unified.h ARM: make BSYM macro assembly only 2012-01-16 08:56:25 -06:00
unistd.h consolidate cond_syscall and SYSCALL_ALIAS declarations 2013-03-03 22:55:19 -05:00
unwind.h ARM: 7187/1: fix unwinding for XIP kernels 2011-12-06 11:16:13 +00:00
user.h ARM: 6798/1: aout-core: zero thread debug registers in a.out core dump 2011-03-10 15:16:29 +00:00
v7m.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
vfp.h
vfpmacros.h ARM: 7566/1: vfp: fix save and restore when running on pre-VFPv3 and CONFIG_VFPv3 set 2012-10-29 10:04:05 +00:00
vga.h ARM: set vga memory base at run-time 2011-07-12 11:19:29 -05:00
virt.h ARM: 7787/1: virt: ensure visibility of __boot_cpu_mode 2013-07-26 12:01:17 +01:00
word-at-a-time.h ARM: 7450/1: dcache: select DCACHE_WORD_ACCESS for little-endian ARMv6+ CPUs 2012-07-09 17:41:11 +01:00
xor.h