linux_dsm_epyc7002/arch/x86/include/asm
Dan Williams 52db400fcd pmem, dax: clean up clear_pmem()
To date, we have implemented two I/O usage models for persistent memory,
PMEM (a persistent "ram disk") and DAX (mmap persistent memory into
userspace).  This series adds a third, DAX-GUP, that allows DAX mappings
to be the target of direct-i/o.  It allows userspace to coordinate
DMA/RDMA from/to persistent memory.

The implementation leverages the ZONE_DEVICE mm-zone that went into
4.3-rc1 (also discussed at kernel summit) to flag pages that are owned
and dynamically mapped by a device driver.  The pmem driver, after
mapping a persistent memory range into the system memmap via
devm_memremap_pages(), arranges for DAX to distinguish pfn-only versus
page-backed pmem-pfns via flags in the new pfn_t type.

The DAX code, upon seeing a PFN_DEV+PFN_MAP flagged pfn, flags the
resulting pte(s) inserted into the process page tables with a new
_PAGE_DEVMAP flag.  Later, when get_user_pages() is walking ptes it keys
off _PAGE_DEVMAP to pin the device hosting the page range active.
Finally, get_page() and put_page() are modified to take references
against the device driver established page mapping.

Finally, this need for "struct page" for persistent memory requires
memory capacity to store the memmap array.  Given the memmap array for a
large pool of persistent may exhaust available DRAM introduce a
mechanism to allocate the memmap from persistent memory.  The new
"struct vmem_altmap *" parameter to devm_memremap_pages() enables
arch_add_memory() to use reserved pmem capacity rather than the page
allocator.

This patch (of 25):

Both __dax_pmd_fault, and clear_pmem() were taking special steps to
clear memory a page at a time to take advantage of non-temporal
clear_page() implementations.  However, x86_64 does not use non-temporal
instructions for clear_page(), and arch_clear_pmem() was always
incurring the cost of __arch_wb_cache_pmem().

Clean up the assumption that doing clear_pmem() a page at a time is more
performant.

Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Reported-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Reviewed-by: Jeff Moyer <jmoyer@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Christoffer Dall <christoffer.dall@linaro.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: David Airlie <airlied@linux.ie>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jan Kara <jack@suse.com>
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Jens Axboe <axboe@fb.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Logan Gunthorpe <logang@deltatee.com>
Cc: Matthew Wilcox <willy@linux.intel.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Richard Weinberger <richard@nod.at>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Toshi Kani <toshi.kani@hpe.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-01-15 17:56:32 -08:00
..
crypto
fpu x86/fpu: Disable AVX when eagerfpu is off 2016-01-12 11:51:21 +01:00
numachip
trace
uv
xen xen: rename dom0_op to platform_op 2015-12-21 14:40:55 +00:00
a.out-core.h
acenv.h
acpi.h
agp.h
alternative-asm.h
alternative.h
amd_nb.h
apb_timer.h
apic_flat_64.h
apic.h x86/apic: Introduce apic_extnmi command line parameter 2015-12-19 11:07:01 +01:00
apicdef.h
apm.h
arch_hweight.h
archrandom.h
asm-offsets.h
asm.h
atomic64_32.h x86/headers: Don't include asm/processor.h in asm/atomic.h 2015-12-06 12:56:03 +01:00
atomic64_64.h
atomic.h x86/headers: Don't include asm/processor.h in asm/atomic.h 2015-12-06 12:56:03 +01:00
barrier.h
bios_ebda.h
bitops.h
boot.h x86/boot: Double BOOT_HEAP_SIZE to 64KB 2016-01-11 12:30:50 +01:00
bootparam_utils.h
bug.h
bugs.h
cache.h
cacheflush.h
calgary.h x86/platform/calgary: Constify cal_chipset_ops structures 2015-11-29 08:50:58 +01:00
ce4100.h
checksum_32.h
checksum_64.h
checksum.h
clocksource.h
cmdline.h
cmpxchg_32.h x86/cpufeature: Remove unused and seldomly used cpu_has_xx macros 2015-12-19 11:49:55 +01:00
cmpxchg_64.h x86/cpufeature: Remove unused and seldomly used cpu_has_xx macros 2015-12-19 11:49:55 +01:00
cmpxchg.h
compat.h
cpu_device_id.h
cpu.h x86/cpu: Unify CPU family, model, stepping calculation 2015-11-24 09:15:54 +01:00
cpufeature.h Merge branch 'x86-cleanups-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2016-01-11 16:26:03 -08:00
cpumask.h
crash.h
current.h
debugreg.h
delay.h
desc_defs.h
desc.h
device.h
disabled-features.h
div64.h
dma-mapping.h
dma.h
dmi.h
dwarf2.h
e820.h
edac.h
efi.h
elf.h
emergency-restart.h
entry_arch.h
espfix.h
exec.h
fb.h
fixmap.h x86/vdso: Remove pvclock fixmap machinery 2015-12-11 08:56:03 +01:00
floppy.h
frame.h
ftrace.h
futex.h
gart.h
genapic.h
geode.h
gpio.h
hardirq.h
highmem.h kmap_atomic_to_page() has no users, remove it 2015-11-09 15:11:24 -08:00
hpet.h
hugetlb.h
hw_breakpoint.h
hw_irq.h
hypertransport.h
hypervisor.h
i8259.h x86/irq: Probe for PIC presence before allocating descs for legacy IRQs 2015-11-07 10:37:37 +01:00
ia32_unistd.h
ia32.h
idle.h
imr.h
inat_types.h
inat.h
init.h
insn.h
inst.h
intel_mid_vrtc.h
intel_pmc_ipc.h
intel_pt.h perf/x86/intel/pt: Add interface to stop Intel PT logging 2015-11-23 09:58:26 +01:00
intel_scu_ipc.h
intel-mid.h
io_apic.h
io.h
iomap.h
iommu_table.h
iommu.h
iosf_mbi.h ACPI / LPSS: override power state for LPSS DMA device 2016-01-07 14:11:32 +01:00
ipi.h x86/apic: Provide default send single IPI wrapper 2015-11-05 13:07:53 +01:00
irq_regs.h
irq_remapping.h
irq_vectors.h
irq_work.h
irq.h
irqdomain.h
irqflags.h
ist.h
jump_label.h x86/asm: Add asm macros for static keys/jump labels 2015-11-24 09:56:44 +01:00
kasan.h
kbdleds.h
Kbuild
kdebug.h
kexec-bzimage64.h
kexec.h
kgdb.h
kmap_types.h
kmemcheck.h
kprobes.h
kvm_emulate.h KVM: x86: add read_phys to x86_emulate_ops 2015-11-04 16:24:31 +01:00
kvm_guest.h
kvm_host.h KVM: move architecture-dependent requests to arch/ 2016-01-08 19:04:36 +01:00
kvm_para.h
lguest_hcall.h
lguest.h lguest: Map switcher text R/O 2016-01-12 12:17:28 +01:00
linkage.h
livepatch.h
local64.h
local.h
mach_timer.h
mach_traps.h
math_emu.h
mc146818rtc.h
mce.h
microcode_amd.h
microcode_intel.h
microcode.h x86/cpu: Unify CPU family, model, stepping calculation 2015-11-24 09:15:54 +01:00
misc.h
mmconfig.h
mmu_context.h x86/mm: Improve switch_mm() barrier comments 2016-01-13 10:42:49 +01:00
mmu.h
mmx.h
mmzone_32.h
mmzone_64.h
mmzone.h
module.h
mpspec_def.h
mpspec.h
mpx.h
mshyperv.h
msi.h x86/irq: Export functions to allow MSI domains in modules 2015-12-20 12:40:49 +01:00
msidef.h
msr-index.h x86/cpu/amd, kvm: Satisfy guest kernel reads of IC_CFG MSR 2015-11-24 09:15:54 +01:00
msr-trace.h x86, tracing, perf: Add trace point for MSR accesses 2015-12-06 12:56:10 +01:00
msr.h Merge branch 'x86-platform-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2016-01-11 17:45:32 -08:00
mtrr.h
mutex_32.h
mutex_64.h
mutex.h
mwait.h
nmi.h
nops.h
numa_32.h
numa.h
olpc_ofw.h
olpc.h
page_32_types.h
page_32.h
page_64_types.h
page_64.h
page_types.h x86/mm: Align macro defines 2015-12-19 11:53:40 +01:00
page.h
paravirt_types.h Merge branch 'x86-cleanups-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2016-01-11 16:26:03 -08:00
paravirt.h Merge branch 'x86-cleanups-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2016-01-11 16:26:03 -08:00
parport.h
pat.h
pci_64.h
pci_x86.h
pci-direct.h
pci-functions.h
pci.h
percpu.h
perf_event_p4.h
perf_event.h
pgalloc.h
pgtable_32_types.h
pgtable_32.h
pgtable_64_types.h
pgtable_64.h
pgtable_types.h x86, thp: remove infrastructure for handling splitting PMDs 2016-01-15 17:56:32 -08:00
pgtable-2level_types.h
pgtable-2level.h
pgtable-3level_types.h
pgtable-3level.h
pgtable.h arch/x86/include/asm/pgtable.h: add pmd_[dirty|mkclean] for THP 2016-01-15 17:56:32 -08:00
platform_sst_audio.h
pm-trace.h
pmc_atom.h
pmem.h pmem, dax: clean up clear_pmem() 2016-01-15 17:56:32 -08:00
posix_types.h
preempt.h
probe_roms.h
processor-cyrix.h
processor-flags.h
processor.h x86/paravirt: Prevent rtc_cmos platform device init on PV guests 2015-12-19 21:35:13 +01:00
prom.h
proto.h
ptrace.h
pvclock-abi.h
pvclock.h x86/vsdo: Fix build on PARAVIRT_CLOCK=y, KVM_GUEST=n 2016-01-06 10:49:53 +01:00
qrwlock.h
qspinlock_paravirt.h locking/pvqspinlock, x86: Optimize the PV unlock code path 2015-11-23 10:02:02 +01:00
qspinlock.h
realmode.h
reboot_fixups.h
reboot.h x86/nmi: Save regs in crash dump on external NMI 2015-12-19 11:07:01 +01:00
required-features.h
rio.h
rmwcc.h
rtc.h
rwsem.h
seccomp.h
sections.h
segment.h
serial.h
setup_arch.h
setup.h
shmparam.h
sigcontext.h
sigframe.h
sighandling.h
signal.h
simd.h
smap.h
smp.h x86/paravirt: Remove unused pv_apic_ops structure 2015-11-19 11:03:58 +01:00
sparsemem.h
special_insns.h
spinlock_types.h
spinlock.h
sta2x11.h
stackprotector.h
stacktrace.h
string_32.h
string_64.h
string.h
suspend_32.h x86/pm: Introduce quirk framework to save/restore extra MSR registers around suspend/resume 2015-11-26 10:04:53 +01:00
suspend_64.h x86/pm: Introduce quirk framework to save/restore extra MSR registers around suspend/resume 2015-11-26 10:04:53 +01:00
suspend.h
svm.h
swiotlb.h
switch_to.h
sync_bitops.h
sys_ia32.h
syscall.h
syscalls.h
sysfb.h
tce.h
thread_info.h
time.h
timer.h
timex.h
tlb.h
tlbflush.h
topology.h
trace_clock.h
traps.h
tsc.h
uaccess_32.h
uaccess_64.h
uaccess.h x86: Add an inlined __copy_from_user_nmi() variant 2015-11-23 09:58:24 +01:00
unaligned.h
unistd.h
uprobes.h
user32.h
user_32.h
user_64.h
user.h
vdso.h x86/vdso: Get pvclock data from the vvar VMA instead of the fixmap 2015-12-11 08:56:03 +01:00
vga.h
vgtod.h
virtext.h
vm86.h
vmx.h KVM: VMX: Enable and initialize VMX TSC scaling 2015-11-10 12:06:19 +01:00
vsyscall.h
vvar.h
word-at-a-time.h
x2apic.h
x86_init.h Merge branch 'linus' into x86/cleanups 2015-12-19 11:49:13 +01:00
xor_32.h x86/cpufeature: Remove unused and seldomly used cpu_has_xx macros 2015-12-19 11:49:55 +01:00
xor_64.h
xor_avx.h
xor.h