linux_dsm_epyc7002/arch/arm64/include/asm
Will Deacon 33309ecda0 arm64: Fix minor issues with the dcache_by_line_op macro
The dcache_by_line_op macro suffers from a couple of small problems:

First, the GAS directives that are currently being used rely on
assembler behavior that is not documented, and probably not guaranteed
to produce the correct behavior going forward. As a result, we end up
with some undefined symbols in cache.o:

$ nm arch/arm64/mm/cache.o
         ...
         U civac
         ...
         U cvac
         U cvap
         U cvau

This is due to the fact that the comparisons used to select the
operation type in the dcache_by_line_op macro are comparing symbols
not strings, and even though it seems that GAS is doing the right
thing here (undefined symbols by the same name are equal to each
other), it seems unwise to rely on this.

Second, when patching in a DC CVAP instruction on CPUs that support it,
the fallback path consists of a DC CVAU instruction which may be
affected by CPU errata that require ARM64_WORKAROUND_CLEAN_CACHE.

Solve these issues by unrolling the various maintenance routines and
using the conditional directives that are documented as operating on
strings. To avoid the complexity of nested alternatives, we move the
DC CVAP patching to __clean_dcache_area_pop, falling back to a branch
to __clean_dcache_area_poc if DCPOP is not supported by the CPU.

Reported-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Suggested-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2018-12-10 15:03:51 +00:00
..
xen arm64: xen: Use existing helper to check interrupt status 2018-10-03 16:14:11 +01:00
acenv.h
acpi.h arm64: acpi: Prepare for longer MADTs 2018-11-27 18:00:14 +00:00
alternative.h arm64: Avoid flush_icache_range() in alternatives patching code 2018-06-27 18:21:53 +01:00
arch_gicv3.h irqchip/gic-v3: Probe for SCR_EL3 being clear before resetting AP0Rn 2018-03-22 13:46:18 +00:00
arch_timer.h First batch of KVM changes for 4.15 2017-11-16 13:00:24 -08:00
arm_dsu_pmu.h perf: ARM DynamIQ Shared Unit PMU support 2018-01-02 16:43:12 +00:00
arm-cci.h
asm-bug.h arm64: asm-bug: Renumber macro local labels to avoid clashes 2017-10-25 15:57:15 +01:00
asm-offsets.h
asm-uaccess.h arm64: mm: Add additional parameter to uaccess_ttbr0_disable 2018-01-17 13:57:49 +01:00
assembler.h arm64: Fix minor issues with the dcache_by_line_op macro 2018-12-10 15:03:51 +00:00
atomic_ll_sc.h arm64: cmpxchg: Use "K" instead of "L" for ll/sc immediate constraint 2018-12-07 17:28:13 +00:00
atomic_lse.h arm64: Avoid masking "old" for LSE cmpxchg() implementation 2018-12-07 17:28:01 +00:00
atomic.h atomics/treewide: Make conditional inc/dec ops optional 2018-06-21 14:25:24 +02:00
barrier.h arm64: Add support for SB barrier and patch in over DSB; ISB sequences 2018-12-06 16:47:04 +00:00
bitops.h locking/atomics/arm64, arm64/bitops: Include <asm-generic/bitops/ext2-atomic-setbit.h> 2018-06-21 12:52:12 +02:00
bitrev.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
boot.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
brk-imm.h
bug.h arm64: Add ASM_BUG() 2017-08-08 16:28:13 +01:00
cache.h arm64: cpufeature: Fix handling of CTR_EL0.IDC field 2018-10-16 11:53:31 +01:00
cacheflush.h arm64: IPI each CPU after invalidating the I-cache for kernel mappings 2018-07-05 17:24:36 +01:00
checksum.h arm64: add missing conversion to __wsum in ip_fast_csum() 2017-06-29 16:32:43 +01:00
clocksource.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cmpxchg.h arm64: Avoid masking "old" for LSE cmpxchg() implementation 2018-12-07 17:28:01 +00:00
compat.h Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-10-25 11:14:36 -07:00
cpu_ops.h
cpu.h arm64/sve: Probe SVE capabilities and usable vector lengths 2017-11-03 15:24:17 +00:00
cpucaps.h arm64: Add support for SB barrier and patch in over DSB; ISB sequences 2018-12-06 16:47:04 +00:00
cpufeature.h arm64: capabilities: Batch cpu_enable callbacks 2018-12-06 15:12:26 +00:00
cpuidle.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cputype.h arm64: capabilities: Merge duplicate Cavium erratum entries 2018-12-06 11:47:44 +00:00
current.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
daifflags.h arm64: daifflags: Use irqflags functions for daifflags 2018-10-03 16:08:52 +01:00
dcc.h
debug-monitors.h
device.h arm64: use the generic swiotlb_dma_ops 2018-10-19 08:53:24 +02:00
dma-mapping.h arm64: use the generic swiotlb_dma_ops 2018-10-19 08:53:24 +02:00
dmi.h
efi.h efi: Deduplicate efi_open_volume() 2018-07-22 14:13:43 +02:00
elf.h arm64: signal: Report signal frame size to userspace via auxv 2018-06-01 15:53:10 +01:00
esr.h arm64: Trap WFI executed in userspace 2018-10-01 16:52:24 +01:00
exception.h arm64: kernel: Prepare for a DISR user 2018-01-16 15:07:12 +00:00
exec.h
extable.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fb.h
fixmap.h arm64: kaslr: Put kernel vectors address in separate data page 2017-12-11 13:41:20 +00:00
fpsimd.h arm64: move sve_user_{enable,disable} to <asm/fpsimd.h> 2018-07-12 14:40:39 +01:00
fpsimdmacros.h arm64/sve: Write ZCR_EL1 on context switch only if changed 2018-05-17 18:19:53 +01:00
ftrace.h arm64: ftrace: enable graph FP test 2018-11-30 13:29:04 +00:00
futex.h arm64: futex: Mask __user pointers prior to dereference 2018-02-06 22:53:42 +00:00
hardirq.h
hugetlb.h hugetlb: introduce generic version of huge_ptep_get 2018-10-26 16:26:34 -07:00
hw_breakpoint.h perf/hw_breakpoint: Remove default hw_breakpoint_arch_parse() 2018-06-26 09:07:58 +02:00
hwcap.h
hypervisor.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
insn.h arm64/insn: add support for emitting ADR/ADRP instructions 2018-11-27 18:47:33 +00:00
io.h arm64: io: Ensure value passed to __iormb() is held in a 64-bit register 2018-11-29 16:36:18 +00:00
irq_work.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
irq.h arm64: Use the new GENERIC_IRQ_MULTI_HANDLER 2018-08-03 12:14:09 +02:00
irqflags.h arm64: Move the async/fiq helpers to explicitly set process context flags 2017-11-02 15:55:41 +00:00
jump_label.h arm64/kernel: jump_label: Switch to relative references 2018-09-27 17:56:47 +02:00
kasan.h kasan: clean up KASAN_SHADOW_SCALE_SHIFT usage 2018-02-06 18:32:43 -08:00
Kbuild arm64: preempt: Provide our own implementation of asm/preempt.h 2018-12-07 12:35:53 +00:00
kernel-pgtable.h arm64/mm: Separate boot-time page tables from swapper_pg_dir 2018-09-25 15:10:54 +01:00
kexec.h
kgdb.h
kprobes.h arm64/kprobes: Remove jprobe implementation 2018-06-21 12:33:07 +02:00
kvm_arm.h KVM updates for v4.20 2018-10-25 17:57:35 -07:00
kvm_asm.h KVM: arm64: Safety check PSTATE when entering guest and handle IL 2018-10-19 11:13:03 +01:00
kvm_coproc.h
kvm_emulate.h arm64/cpufeatures: Introduce ESR_ELx_SYS64_ISS_RT() 2018-09-21 11:05:25 +01:00
kvm_host.h KVM updates for v4.20 2018-10-25 17:57:35 -07:00
kvm_hyp.h kvm: arm64: Configure VTCR_EL2 per VM 2018-10-01 13:50:29 +01:00
kvm_mmio.h
kvm_mmu.h KVM updates for v4.20 2018-10-25 17:57:35 -07:00
linkage.h arm64: relax assembly code alignment from 16 byte to 4 byte 2017-09-18 11:20:19 +01:00
lse.h arm64: lse: Include compiler_types.h and export.h for out-of-line LL/SC 2018-03-27 13:14:49 +01:00
memblock.h
memory.h arm64: KVM: Add support for Stage-2 control of memory types and cacheability 2018-07-09 11:37:41 +01:00
mmu_context.h arm64: mm: apply r/o permissions of VM areas to its linear alias as well 2018-11-20 11:38:26 +00:00
mmu.h arm64/mm: Separate boot-time page tables from swapper_pg_dir 2018-09-25 15:10:54 +01:00
mmzone.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
module.h arm64/module: switch to ADRP/ADD sequences for PLT entries 2018-11-27 19:00:45 +00:00
neon-intrinsics.h arm64/neon: add workaround for ambiguous C99 stdint.h types 2018-12-06 16:47:05 +00:00
neon.h arm64: drop unused kernel_neon_begin_partial() macro 2018-07-31 10:13:50 +01:00
numa.h arm64: numa: separate out updates to percpu nodeid and NUMA node cpumap 2018-07-06 13:18:18 +01:00
page-def.h arm64: factor out PAGE_* and CONT_* definitions 2017-08-15 18:35:00 +01:00
page.h arm64: Kconfig: Remove ARCH_HAS_HOLES_MEMORYMODEL 2018-09-21 12:02:45 +01:00
paravirt.h x86/paravirt: Use a single ops structure 2018-09-03 16:50:35 +02:00
pci.h PCI: remove PCI_DMA_BUS_IS_PHYS 2018-05-07 07:15:41 +02:00
percpu.h arm64: percpu: Rewrite per-cpu ops to allow use of LSE atomics 2018-12-07 17:28:06 +00:00
perf_event.h arm64/bpf: correct broken uapi for BPF_PROG_TYPE_PERF_EVENT program type 2017-12-05 15:02:41 +01:00
pgalloc.h arm64: mm: Use READ_ONCE/WRITE_ONCE when accessing page tables 2018-02-16 18:13:57 +00:00
pgtable-hwdef.h arm64: mm: Support Common Not Private translations 2018-09-18 12:02:27 +01:00
pgtable-prot.h arm64: KVM: Avoid marking pages as XN in Stage-2 if CTR_EL0.DIC is set 2018-07-09 11:37:42 +01:00
pgtable-types.h
pgtable.h arm64: mm: Don't wait for completion of TLB invalidation when page aging 2018-11-26 16:59:46 +00:00
preempt.h arm64: preempt: Provide our own implementation of asm/preempt.h 2018-12-07 12:35:53 +00:00
probes.h
proc-fns.h arm64: mm: Move ASID from TTBR0 to TTBR1 2017-12-11 13:40:25 +00:00
processor.h arm64: mm: define NET_IP_ALIGN to 0 2018-11-08 17:50:26 +00:00
ptdump.h
ptrace.h KVM updates for v4.20 2018-10-25 17:57:35 -07:00
sdei.h arm64: Add stack information to on_accessible_stack 2018-07-26 11:36:07 +01:00
seccomp.h
sections.h arm64: mmu: add the entry trampolines start/end section markers into sections.h 2018-01-14 18:49:50 +00:00
shmparam.h
signal32.h arm64: compat: Remove leftover variable declaration 2017-08-10 18:17:32 +01:00
simd.h arm64: neon: Fix function may_use_simd() return error status 2018-07-11 17:02:02 +01:00
smp_plat.h
smp.h arm64: kexec: have own crash_smp_send_stop() for crash dump for nonpanic cores 2017-08-21 18:01:04 +01:00
sparsemem.h arm64: add kconfig symbol to configure physical address size 2017-12-22 17:30:33 +00:00
spinlock_types.h arm64: locking: Replace ticket lock implementation with qspinlock 2018-07-05 10:05:06 +01:00
spinlock.h arm64: locking: Replace ticket lock implementation with qspinlock 2018-07-05 10:05:06 +01:00
stack_pointer.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
stackprotector.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
stacktrace.h arm64: Add stack information to on_accessible_stack 2018-07-26 11:36:07 +01:00
stage2_pgtable.h kvm: arm64: Allow tuning the physical address size for VM 2018-10-03 11:45:20 +01:00
stat.h y2038: globally rename compat_time to old_time32 2018-08-27 14:48:48 +02:00
string.h arm64: lib: use C string functions with KASAN enabled 2018-10-26 16:25:18 -07:00
suspend.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sync_bitops.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
syscall_wrapper.h arm64: implement syscall wrappers 2018-07-12 14:49:48 +01:00
syscall.h arm64: implement syscall wrappers 2018-07-12 14:49:48 +01:00
sysreg.h arm64: Add support for SB barrier and patch in over DSB; ISB sequences 2018-12-06 16:47:04 +00:00
system_misc.h signal/arm64: Push siginfo generation into arm64_notify_die 2018-09-27 21:52:54 +02:00
thread_info.h arm64: preempt: Provide our own implementation of asm/preempt.h 2018-12-07 12:35:53 +00:00
timex.h arm64: Use physical counter for in-kernel reads when booted in EL2 2017-11-06 16:23:09 +01:00
tlb.h arm64: tlb: Avoid synchronous TLBIs when freeing page tables 2018-09-11 16:49:12 +01:00
tlbflush.h arm64: tlbi: Set MAX_TLBI_OPS to PTRS_PER_PTE 2018-11-27 19:01:21 +00:00
topology.h sched/topology, arch/arm64: Rebuild the sched_domain hierarchy when the CPU capacity changes 2018-09-10 11:05:47 +02:00
traps.h signal/arm64: Add and use arm64_force_sig_ptrace_errno_trap 2018-09-27 21:55:15 +02:00
uaccess.h arm64: Add support for SB barrier and patch in over DSB; ISB sequences 2018-12-06 16:47:04 +00:00
unistd32.h arm64: convert compat wrappers to C 2018-07-12 14:49:48 +01:00
unistd.h y2038: utimes: Rework #ifdef guards for compat syscalls 2018-08-29 15:42:23 +02:00
uprobes.h
vdso_datapage.h
vdso.h
virt.h arm64: capabilities: Change scope of VHE to Boot CPU feature 2018-03-26 18:01:41 +01:00
vmap_stack.h arm64: Add vmap_stack header file 2018-01-13 10:45:03 +00:00
word-at-a-time.h
xor.h arm64: crypto: add NEON accelerated XOR implementation 2018-12-06 16:47:06 +00:00