linux_dsm_epyc7002/arch/arm/include/asm
Will Deacon f38d999c4d ARM: atomics: prefetch the destination word for write prior to strex
The cost of changing a cacheline from shared to exclusive state can be
significant, especially when this is triggered by an exclusive store,
since it may result in having to retry the transaction.

This patch prefixes our atomic access implementations with pldw
instructions (on CPUs which support them) to try and grab the line in
exclusive state from the start. Only the barrier-less functions are
updated, since memory barriers can limit the usefulness of prefetching
data.

Acked-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2013-09-30 16:42:56 +01:00
..
hardware ARM: debug: move PL01X debug include into arch/arm/include/debug/ 2013-08-25 17:11:06 +01:00
mach ARM: SoC late changes for v3.12 2013-09-09 16:35:29 -07:00
xen xen/arm and xen/arm64: implement HYPERVISOR_tmem_op 2013-07-04 11:41:12 +00:00
arch_timer.h Merge branch 'timers/clockevents-next' of git://git.linaro.org/people/dlezcano/clockevents into timers/core 2013-08-21 14:59:23 +02:00
asm-offsets.h kbuild: move asm-offsets.h to include/generated 2009-12-12 13:08:14 +01:00
assembler.h ARM: barrier: allow options to be passed to memory barrier instructions 2013-08-12 12:25:44 +01:00
atomic.h ARM: atomics: prefetch the destination word for write prior to strex 2013-09-30 16:42:56 +01:00
barrier.h ARM: barrier: allow options to be passed to memory barrier instructions 2013-08-12 12:25:44 +01:00
bitops.h Disintegrate asm/system.h for ARM 2012-03-28 18:30:01 +01:00
bug.h Disintegrate asm/system.h for ARM 2012-03-28 18:30:01 +01:00
bugs.h [ARM] move include/asm-arm to arch/arm/include/asm 2008-08-02 21:32:35 +01:00
cache.h ARM: implement support for read-mostly sections 2010-12-05 08:39:36 +00:00
cacheflush.h Merge branch 'for-rmk/cacheflush-v2' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into devel-stable 2013-08-28 18:38:40 +01:00
cachetype.h ARM: 7062/1: cache: detect PIPT I-cache using CTR 2011-10-17 09:13:41 +01:00
checksum.h [ARM] move include/asm-arm to arch/arm/include/asm 2008-08-02 21:32:35 +01:00
clkdev.h ARM: Consolidate the clkdev header files 2011-07-19 18:09:45 +02:00
cmpxchg.h ARM: 7720/1: ARM v6/v7 cmpxchg64 shouldn't clear upper 32 bits of the old/new value 2013-05-13 23:42:24 +01:00
compiler.h Disintegrate asm/system.h for ARM 2012-03-28 18:30:01 +01:00
cp15.h Merge branch 'for-linus' of git://git.linaro.org/people/rmk/linux-arm 2013-07-03 09:46:29 -07:00
cpu.h ARM: kernel: add MIDR to per-CPU information data 2012-11-19 14:51:11 +00:00
cpuidle.h cpuidle: Add common time keeping and irq enabling 2012-03-21 01:59:40 -04:00
cputype.h ARM: 7801/1: v6: prevent gcc 4.5 from reordering extended CP15 reads above is_smp() test 2013-07-31 11:12:59 +01:00
cti.h ARM: coresight: common definition for (OS) Lock Access Register key value 2013-01-10 21:13:05 +00:00
delay.h ARM: 7685/1: delay: use private ticks_per_jiffy field for timer-based delay ops 2013-04-03 16:45:50 +01:00
device.h ARM: dma-mapping: Add macro to_dma_iommu_mapping() 2013-02-25 15:30:41 +01:00
div64.h ARM: 7705/1: use optimized do_div only for EABI 2013-05-15 19:35:53 +01:00
dma-contiguous.h drivers: dma-contiguous: clean source code and prepare for device tree 2013-08-27 09:18:29 +02:00
dma-iommu.h ARM: dma-mapping: Add arm_iommu_detach_device() 2013-02-25 15:30:41 +01:00
dma-mapping.h IOMMU Updates for Linux v3.8 2012-12-20 10:07:25 -08:00
dma.h ARM: disable virt_to_bus/virt_to_bus almost everywhere 2013-02-14 15:04:33 +01:00
domain.h ARM: fix set_domain() macro 2012-07-05 09:50:55 +01:00
ecard.h ARM: io: ecard: move ioaddr() inside __ecard_address 2011-08-17 08:44:16 +01:00
edac.h ARM: 7201/1: add EDAC atomic_scrub function 2011-12-11 08:35:50 +00:00
elf.h Move the EM_ARM and EM_AARCH64 definitions to uapi/linux/elf-em.h 2013-09-02 16:35:50 +01:00
entry-macro-multi.S ARM: gic: consolidate PPI handling 2011-10-23 13:32:29 +01:00
exception.h ARM: 7115/4: move __exception and friends to asm/exception.h 2011-10-17 09:02:44 +01:00
fb.h [ARM] move include/asm-arm to arch/arm/include/asm 2008-08-02 21:32:35 +01:00
fiq.h ARM: 6940/1: fiq: Briefly document driver responsibilities for suspend/resume 2011-05-26 10:31:06 +01:00
firmware.h ARM: Add interface for registering and calling firmware-specific operations 2013-04-09 01:52:06 +09:00
fixmap.h [ARM] fixmap support 2009-03-15 21:01:20 -04:00
flat.h ARM: binfmt_flat: unused variable 'persistent' 2012-10-09 20:29:06 +02:00
floppy.h [ARM] Move include/asm-arm/arch-* to arch/arm/*/include/mach 2008-08-07 09:55:48 +01:00
fncpy.h ARM: 6640/1: Thumb-2: Symbol manipulation macros for function body copying 2011-01-27 11:48:58 +00:00
fpstate.h Fix common misspellings 2011-03-31 11:26:23 -03:00
ftrace.h ARM: 6319/1: ftrace: add Thumb-2 support to dynamic ftrace 2010-09-02 15:28:43 +01:00
futex.h ARM: 7425/1: extable: ensure fixup entries are 4-byte aligned 2012-06-16 16:30:25 +01:00
glue-cache.h Merge branch 'for-next' of git://git.pengutronix.de/git/ukl/linux into devel-stable 2013-05-22 10:52:24 +01:00
glue-df.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
glue-pf.h ARM: move cache/processor/fault glue to separate include files 2011-02-12 11:52:21 +00:00
glue-proc.h ARM SoC specific changes 2013-07-02 13:43:38 -07:00
glue.h Fix common misspellings 2011-03-31 11:26:23 -03:00
gpio.h ARM: make mach/gpio.h headers optional 2012-09-14 09:21:59 -05:00
hardirq.h ARM: 7536/1: smp: Formalize an IPI for wakeup 2012-09-19 21:52:08 +01:00
highmem.h ARM: 7684/1: errata: Workaround for Cortex-A15 erratum 798181 (TLBI/DSB operations) 2013-04-03 16:45:49 +01:00
hugetlb-3level.h ARM: mm: HugeTLB support for LPAE systems. 2013-06-04 16:52:37 +01:00
hugetlb.h ARM: mm: HugeTLB support for LPAE systems. 2013-06-04 16:52:37 +01:00
hw_breakpoint.h ARM: hw_breakpoint: Check function for OS Save and Restore mechanism 2013-01-10 21:13:06 +00:00
hw_irq.h arm: dove: Use proper irq accessor functions 2011-03-29 14:47:57 +02:00
hwcap.h UAPI: (Scripted) Disintegrate arch/arm/include/asm 2012-10-12 13:05:52 +01:00
hypervisor.h arm: initial Xen support 2012-09-14 13:53:39 +00:00
ide.h [ARM] move include/asm-arm to arch/arm/include/asm 2008-08-02 21:32:35 +01:00
idmap.h ARM: KVM: move to a KVM provided HYP idmap 2013-04-28 22:23:08 -07:00
io.h ARM: 7728/1: mm: Use phys_addr_t properly for ioremap functions 2013-05-23 00:09:44 +01:00
irq.h arm: Move the set_handle_irq and handle_arch_irq declarations to asm/irq.h 2013-03-26 16:11:20 +00:00
irqflags.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
jump_label.h ARM: 7386/1: jump_label: fixup for rename to static_key 2012-04-15 22:00:31 +01:00
Kbuild Merge branch 'for-linus' of git://git.linaro.org/people/rmk/linux-arm 2012-12-12 11:30:02 -08:00
kexec.h [ARM] add machine-specific hook to machine_kexec 2011-03-03 16:26:55 -05:00
kgdb.h kgdb,arm: fix register dump 2010-10-29 13:14:40 -05:00
kmap_types.h arm: remove km_type definitions 2012-07-24 15:27:28 +08:00
kprobes.h Kernel: Audit Support For The ARM Platform 2012-01-17 16:17:01 -05:00
kvm_arm.h ARM: KVM: get rid of S2_PGD_SIZE 2013-06-26 10:50:04 -07:00
kvm_asm.h ARM: KVM: perform save/restore of PAR 2013-06-26 10:50:04 -07:00
kvm_coproc.h KVM: ARM: User space API for getting/setting co-proc registers 2013-01-23 13:29:14 -05:00
kvm_emulate.h ARM: KVM: don't special case PC when doing an MMIO 2013-06-26 10:50:03 -07:00
kvm_host.h arm/kvm: Cleanup KVM_ARM_MAX_VCPUS logic 2013-06-26 10:50:05 -07:00
kvm_mmio.h KVM: ARM: Handle I/O aborts 2013-01-23 13:29:17 -05:00
kvm_mmu.h ARM: KVM: Fix kvm_set_pte assignment 2013-08-30 15:47:37 -07:00
kvm_psci.h KVM: ARM: Power State Coordination Interface implementation 2013-01-23 13:29:18 -05:00
limits.h [ARM] move include/asm-arm to arch/arm/include/asm 2008-08-02 21:32:35 +01:00
linkage.h [ARM] move include/asm-arm to arch/arm/include/asm 2008-08-02 21:32:35 +01:00
mach-types.h arm: move mach-types to include/generated 2009-12-12 13:08:14 +01:00
mc146818rtc.h ARM: mc146818rtc: remove unnecessary include of mach/irqs.h 2012-01-25 20:37:45 -06:00
mcpm.h ARM: mcpm: provide an interface to set the SMP ops at run time 2013-04-24 10:37:03 -04:00
memblock.h ARM: constify machine_desc structure uses 2013-07-26 14:55:59 +01:00
memory.h mm/ARM: fix stale comment about VALID_PAGE() 2013-07-03 16:07:39 -07:00
mmu_context.h ARM: 7790/1: Fix deferred mm switch on VIVT processors 2013-07-26 12:02:09 +01:00
mmu.h Merge branch 'security-fixes' into fixes 2013-08-01 20:51:13 +01:00
module.h ARM: 7829/1: Add ".text.unlikely" and ".text.hot" to arm unwind tables 2013-09-02 13:49:47 +01:00
mpu.h ARM: mpu: protect the vectors page with an MPU region 2013-06-17 15:13:18 +01:00
mtd-xip.h [ARM] move asm/xip.h's mach/hardware.h include to mach/xip.h 2008-12-14 13:22:51 +00:00
mutex.h ARM: 7495/1: mutex: use generic atomic_dec-based implementation for ARMv6+ 2012-08-25 09:22:31 +01:00
neon.h ARM: add support for kernel mode NEON 2013-07-08 22:08:51 +01:00
nwflash.h [ARM] move include/asm-arm to arch/arm/include/asm 2008-08-02 21:32:35 +01:00
opcodes-sec.h ARM: opcodes: add opcodes definitions for ARM security extensions 2013-01-10 21:10:20 +00:00
opcodes-virt.h ARM: opcodes: add __ERET/__MSR_ELR_HYP instruction encoding 2012-09-19 08:32:49 +01:00
opcodes.h ARM: opcodes: add missing include of linux/linkage.h 2013-01-10 21:10:20 +00:00
outercache.h ARM: move outer_cache declaration out of ifdef 2013-08-22 20:48:41 -05:00
page-nommu.h nommu: Remove the memory_start/end variables from ARM page-nommu.h 2009-07-24 12:35:01 +01:00
page.h ARM: make vectors page inaccessible from userspace 2013-08-01 14:31:58 +01:00
pci.h PCI: collapse pcibios_resource_to_bus 2012-02-23 20:19:04 -07:00
percpu.h ARM: 7747/1: pcpu: ensure __my_cpu_offset cannot be re-ordered across barrier() 2013-06-05 23:35:56 +01:00
perf_event.h ARM: 7584/1: perf: fix link error when CONFIG_HW_PERF_EVENTS is not selected 2012-11-23 13:01:30 +00:00
pgalloc.h ARM: LPAE: Page table maintenance for the 3-level format 2011-12-08 10:30:39 +00:00
pgtable-2level-hwdef.h ARM: 7077/1: LPAE: Use a mask for physical addresses in page table entries 2011-10-06 15:40:06 +01:00
pgtable-2level-types.h ARM: 7076/1: LPAE: Add (pte|pmd)val_t type definitions as u32 2011-10-06 15:40:05 +01:00
pgtable-2level.h ARM: mm: introduce present, faulting entries for PAGE_NONE 2012-11-09 14:13:20 +00:00
pgtable-3level-hwdef.h Merge branch 'for-rmk/lpae' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into devel-stable 2013-06-18 20:11:32 +01:00
pgtable-3level-types.h ARM: LPAE: Introduce the 3-level page table format definitions 2011-12-08 10:30:39 +00:00
pgtable-3level.h Merge branch 'for-rmk/lpae' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into devel-stable 2013-06-18 20:11:32 +01:00
pgtable-hwdef.h ARM: LPAE: Introduce the 3-level page table format definitions 2011-12-08 10:30:39 +00:00
pgtable-nommu.h consolidate io_remap_pfn_range definitions 2013-06-29 12:46:35 +04:00
pgtable.h ARM: 7808/1: KVM: mm: Get rid of L_PTE_USER ref from PAGE_S2_DEVICE 2013-08-13 20:25:06 +01:00
pmu.h ARM: perf: consistently use arm_pmu->name for PMU name 2012-11-09 11:37:26 +00:00
proc-fns.h Merge branch 'for-rmk/lpae' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into devel-stable 2013-06-18 20:11:32 +01:00
processor.h ARM: prefetch: add support for prefetchw using pldw on SMP ARMv7+ CPUs 2013-09-30 16:42:55 +01:00
procinfo.h [ARM] move include/asm-arm to arch/arm/include/asm 2008-08-02 21:32:35 +01:00
prom.h ARM: constify machine_desc structure uses 2013-07-26 14:55:59 +01:00
psci.h arm: introduce psci_smp_ops 2013-05-21 14:24:11 +00:00
ptrace.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
scatterlist.h ARM: Allow SoCs to enable scatterlist chaining 2011-06-02 11:16:22 +01:00
sched_clock.h sched_clock: Add temporary asm/sched_clock.h 2013-06-21 10:17:44 -07:00
seccomp.h ARM: SECCOMP support 2010-10-01 22:32:18 -04:00
setup.h UAPI: (Scripted) Disintegrate arch/arm/include/asm 2012-10-12 13:05:52 +01:00
shmparam.h [ARM] move include/asm-arm to arch/arm/include/asm 2008-08-02 21:32:35 +01:00
signal.h arm: switch to generic old sigaction() 2013-02-03 18:15:47 -05:00
smp_plat.h ARM: 7807/1: kexec: validate CPU hotplug support 2013-08-13 20:23:18 +01:00
smp_scu.h ARM: scu: provide inline dummy functions when SCU is not present 2013-07-04 05:10:51 -07:00
smp_twd.h ARM: smp_twd: convert to use CLKSRC_OF init 2013-03-11 08:42:08 -05:00
smp.h ARM: mpu: add MPU initialisation for secondary cores 2013-06-07 17:02:53 +01:00
sparsemem.h [ARM] mm: enable sparsemem on clps7500 and RiscPC 2008-10-01 17:24:04 +01:00
spinlock_types.h ARM: locks: prefetch the destination word for write prior to strex 2013-09-30 16:42:55 +01:00
spinlock.h ARM: locks: prefetch the destination word for write prior to strex 2013-09-30 16:42:55 +01:00
stackprotector.h ARM: initial stack protector (-fstack-protector) support 2010-06-14 21:31:00 -04:00
stacktrace.h [ARM] 5382/1: unwind: Reorganise the stacktrace support 2009-02-12 13:21:17 +00:00
string.h [ARM] remove memzero() 2008-11-27 12:37:59 +00:00
suspend.h ARM: kernel: implement stack pointer save array through MPIDR hashing 2013-06-20 11:24:11 +01:00
swab.h UAPI: (Scripted) Disintegrate arch/arm/include/asm 2012-10-12 13:05:52 +01:00
switch_to.h ARM: spinlock: use inner-shareable dsb variant prior to sev instruction 2013-08-12 12:25:45 +01:00
sync_bitops.h xen/arm: sync_bitops 2012-08-08 16:34:01 +00:00
syscall.h ARM: 7577/1: arch/add syscall_get_arch 2012-11-19 14:14:17 +00:00
system_info.h ARM: Add base support for ARMv7-M 2013-04-17 21:38:10 +02:00
system_misc.h reboot: arm: change reboot_mode to use enum reboot_mode 2013-07-09 10:33:29 -07:00
system.h the only place that needs to include asm/exec.h is linux/binfmts.h 2012-09-20 09:51:13 -04:00
tcm.h ARM: 6985/1: export functions to determine the presence of I/DTCM 2011-07-06 20:49:45 +01:00
therm.h [ARM] move include/asm-arm to arch/arm/include/asm 2008-08-02 21:32:35 +01:00
thread_info.h Merge branch 'for-rmk/cacheflush-v2' of git://git.kernel.org/pub/scm/linux/kernel/git/will/linux into devel-stable 2013-08-28 18:38:40 +01:00
thread_notify.h ARM: 6867/1: Introduce THREAD_NOTIFY_COPY for copy_thread() hooks 2011-04-10 21:13:36 +01:00
timex.h Merge branch 'for-linus' of git://git.linaro.org/people/rmk/linux-arm 2012-10-07 21:20:57 +09:00
tlb.h Fix TLB gather virtual address range invalidation corner cases 2013-08-16 08:52:46 -07:00
tlbflush.h ARM: tlb: reduce scope of barrier domains for TLB invalidation 2013-08-12 12:25:45 +01:00
tls.h ARM: 7735/2: Preserve the user r/w register TPIDRURW on context switch and fork 2013-06-24 15:21:59 +01:00
topology.h ARM: 7182/1: ARM cpu topology: fix warning 2011-11-30 23:55:21 +00:00
traps.h ARM: earlier initialization of vectors page 2012-01-23 10:24:11 +00:00
types.h ARM: 7822/1: add workaround for ambiguous C99 stdint.h types 2013-08-25 17:09:46 +01:00
uaccess.h ARM: 7836/1: add __get_user_unaligned/__put_user_unaligned 2013-09-18 10:58:23 +01:00
ucontext.h Fix common misspellings 2011-03-31 11:26:23 -03:00
unified.h ARM: smp_on_up: move inline asm ALT_SMP patching macro out of spinlock.h 2013-09-30 16:42:55 +01:00
unistd.h consolidate cond_syscall and SYSCALL_ALIAS declarations 2013-03-03 22:55:19 -05:00
unwind.h ARM: 7187/1: fix unwinding for XIP kernels 2011-12-06 11:16:13 +00:00
user.h ARM: 6798/1: aout-core: zero thread debug registers in a.out core dump 2011-03-10 15:16:29 +00:00
v7m.h ARM: 7828/1: ARMv7-M: implement restart routine common to all v7-M machines 2013-09-02 13:49:29 +01:00
vfp.h [ARM] move include/asm-arm to arch/arm/include/asm 2008-08-02 21:32:35 +01:00
vfpmacros.h ARM: 7566/1: vfp: fix save and restore when running on pre-VFPv3 and CONFIG_VFPv3 set 2012-10-29 10:04:05 +00:00
vga.h ARM: set vga memory base at run-time 2011-07-12 11:19:29 -05:00
virt.h ARM: 7787/1: virt: ensure visibility of __boot_cpu_mode 2013-07-26 12:01:17 +01:00
word-at-a-time.h ARM: 7450/1: dcache: select DCACHE_WORD_ACCESS for little-endian ARMv6+ CPUs 2012-07-09 17:41:11 +01:00
xor.h ARM: crypto: add NEON accelerated XOR implementation 2013-07-08 22:09:06 +01:00