AuxXxilium/linux_dsm_epyc7002
1
0
Fork 0
mirror of https://github.com/AuxXxilium/linux_dsm_epyc7002.git synced 2024-12-28 11:18:45 +07:00
Code Issues Actions Packages Projects Releases Wiki Activity
1fe565517b
linux_dsm_epyc7002/arch/x86/include/asm
History
Rik van Riel 2084140594 mm: fix TLB flush race between migration, and change_protection_range 
There are a few subtle races, between change_protection_range (used by
mprotect and change_prot_numa) on one side, and NUMA page migration and
compaction on the other side.

The basic race is that there is a time window between when the PTE gets
made non-present (PROT_NONE or NUMA), and the TLB is flushed.

During that time, a CPU may continue writing to the page.

This is fine most of the time, however compaction or the NUMA migration
code may come in, and migrate the page away.

When that happens, the CPU may continue writing, through the cached
translation, to what is no longer the current memory location of the
process.

This only affects x86, which has a somewhat optimistic pte_accessible.
All other architectures appear to be safe, and will either always flush,
or flush whenever there is a valid mapping, even with no permissions
(SPARC).

The basic race looks like this:

CPU A			CPU B			CPU C

						load TLB entry
make entry PTE/PMD_NUMA
			fault on entry
						read/write old page
			start migrating page
			change PTE/PMD to new page
						read/write old page [*]
flush TLB
						reload TLB from new entry
						read/write new page
						lose data

[*] the old page may belong to a new user at this point!

The obvious fix is to flush remote TLB entries, by making sure that
pte_accessible aware of the fact that PROT_NONE and PROT_NUMA memory may
still be accessible if there is a TLB flush pending for the mm.

This should fix both NUMA migration and compaction.

[mgorman@suse.de: fix build]
Signed-off-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Cc: Alex Thorlton <athorlton@sgi.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-12-18 19:04:51 -08:00
..
crypto crypto: move x86 to the generic version of ablk_helper 2013-09-24 06:02:24 +10:00
numachip x86/PCI: Add NumaChip remote PCI support 2012-12-07 14:24:32 -07:00
trace ftrace, perf: Avoid infinite event generation loop 2013-11-19 16:57:40 +01:00
uv Revert "x86/UV: Add uvtrace support" 2013-11-11 19:53:42 +01:00
visws
xen xen: introduce xen_dma_map/unmap_page and xen_dma_sync_single_for_cpu/device 2013-10-25 10:39:49 +00:00
a.out-core.h
acpi.h ACPI / x86: Increase override tables number limit 2013-09-25 16:59:39 +02:00
agp.h
alternative-asm.h
alternative.h kprobes/x86: Call out into INT3 handler directly instead of using notifier 2013-07-23 10:12:57 +02:00
amd_nb.h x86, AMD, NB: Add multi-domain support 2013-01-10 16:17:58 +01:00
apb_timer.h
apic_flat_64.h
apic.h x86/ioapic/kcrash: Prevent crash_kexec() from deadlocking on ioapic_lock 2013-08-20 09:26:33 +02:00
apicdef.h
apm.h
arch_hweight.h
archrandom.h
asm-offsets.h
asm.h x86, asm: Extend definitions of _ASM_* with a raw format 2013-08-29 13:26:32 -07:00
atomic64_32.h
atomic64_64.h x86, bitops: Correct the assembly constraints to testing bitops 2013-12-04 14:31:28 -08:00
atomic.h x86, bitops: Correct the assembly constraints to testing bitops 2013-12-04 14:31:28 -08:00
barrier.h
bios_ebda.h
bitops.h x86, bitops: Correct the assembly constraints to testing bitops 2013-12-04 14:31:28 -08:00
boot.h UAPI: (Scripted) Disintegrate arch/x86/include/asm 2012-12-14 22:37:13 +00:00
bootparam_utils.h x86: Don't clear olpc_ofw_header when sentinel is detected 2013-08-09 15:29:48 -07:00
bug.h dump_stack: unify debug information printed by show_regs() 2013-04-30 17:04:02 -07:00
bugs.h
cache.h
cacheflush.h
calgary.h
calling.h sched, x86: Optimize the preempt_schedule() call 2013-09-25 14:23:07 +02:00
ce4100.h
checksum_32.h x86, smap: Handle csum_partial_copy_*_user() 2013-09-01 14:09:48 -07:00
checksum_64.h x86, asmlinkage: Make 64bit checksum functions visible 2013-08-06 14:20:59 -07:00
checksum.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
clocksource.h x86: pvclock: generic pvclock vsyscall initialization 2012-11-27 23:29:09 -02:00
cmpxchg_32.h x86, 386 removal: Remove CONFIG_CMPXCHG 2012-11-29 13:23:01 -08:00
cmpxchg_64.h
cmpxchg.h x86 cmpxchg.h: fix wrong comment 2013-04-25 10:39:04 +02:00
compat.h compat: move compat_siginfo_t definition to asm/compat.h 2012-10-06 03:05:16 +09:00
context_tracking.h context_tracking: Move exception handling to generic code 2013-03-07 17:09:25 +01:00
cpu_device_id.h
cpu.h x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
cpufeature.h compiler/gcc4: Add quirk for 'asm goto' miscompilation bug 2013-10-11 07:39:14 +02:00
cpumask.h
cputime.h
current.h
debugreg.h UAPI: (Scripted) Disintegrate arch/x86/include/asm 2012-12-14 22:37:13 +00:00
delay.h
desc_defs.h
desc.h x86, trace: Delete __trace_alloc_intr_gate() 2013-11-08 14:15:47 -08:00
device.h driver core / ACPI: Move ACPI support to core device and driver types 2012-11-15 00:28:00 +01:00
div64.h
dma-contiguous.h drivers: dma-contiguous: clean source code and prepare for device tree 2013-08-27 09:18:29 +02:00
dma-mapping.h dma-debug: New interfaces to debug dma mapping errors 2012-10-24 17:06:43 +02:00
dma.h
dmi.h
dwarf2.h
e820.h x86: avoid remapping data in parse_setup_data() 2013-08-13 23:29:19 -07:00
edac.h
efi.h x86/efi: Add EFI framebuffer earlyprintk support 2013-10-28 18:09:58 +00:00
elf.h mm: use vm_unmapped_area() on x86_64 architecture 2012-12-11 17:22:25 -08:00
emergency-restart.h reboot: move arch/x86 reboot= handling to generic kernel 2013-07-09 10:33:29 -07:00
entry_arch.h x86, trace: Add irq vector tracepoints 2013-06-20 22:25:34 -07:00
exec.h
fb.h
fixmap.h ix86: Don't waste fixmap entries 2013-05-31 13:08:18 +02:00
floppy.h
fpu-internal.h x86: move fpu_counter into ARCH specific thread_struct 2013-11-13 12:09:13 +09:00
frame.h
ftrace.h tracing/syscalls: Allow archs to ignore tracing compat syscalls 2013-02-12 17:46:28 -05:00
futex.h x86, 386 removal: Remove CONFIG_BSWAP 2012-11-29 13:23:02 -08:00
gart.h
genapic.h
geode.h
gpio.h
hardirq.h KVM: VMX: Register a new IPI for posted interrupt 2013-04-16 16:32:39 -03:00
highmem.h
hpet.h x86, hpet: Introduce x86_msi_ops.setup_hpet_msi 2013-01-28 10:48:30 +01:00
hugetlb.h mm/hugetlb: add more arch-defined huge_pte functions 2013-04-29 15:54:33 -07:00
hw_breakpoint.h UAPI: (Scripted) Disintegrate arch/x86/include/asm 2012-12-14 22:37:13 +00:00
hw_irq.h x86, trace: Register exception handler to trace IDT 2013-11-08 14:15:45 -08:00
hypertransport.h
hypervisor.h x86: Correctly detect hypervisor 2013-08-05 06:35:33 -07:00
i387.h
i8259.h
ia32_unistd.h
ia32.h x86: switch to generic old sigaction 2013-02-03 18:16:27 -05:00
idle.h
inat_types.h
inat.h
init.h x86, 64bit, mm: Add generic kernel/ident mapping helper 2013-01-29 15:12:25 -08:00
insn.h
inst.h x86, crc32-pclmul: Fix build with older binutils 2013-05-30 16:36:23 -07:00
intel_mid_vrtc.h intel_mid: Renamed *mrst* to *intel_mid* 2013-10-17 16:40:36 -07:00
intel_scu_ipc.h
intel-mid.h intel_mid: Move platform device setups to their own platform_<device>.* files 2013-10-17 16:41:50 -07:00
io_apic.h x86, io_apic: Introduce eoi_ioapic_pin call-back 2013-01-28 12:51:52 +01:00
io.h Add arch_phys_wc_{add, del} to manipulate WC MTRRs if needed 2013-05-31 13:02:52 +10:00
iomap.h
iommu_table.h
iommu.h
ipi.h
irq_regs.h
irq_remapping.h iommu: Fix compile warnings with forward declarations 2013-04-19 20:34:55 +02:00
irq_vectors.h KVM: VMX: Register a new IPI for posted interrupt 2013-04-16 16:32:39 -03:00
irq.h x86, asmlinkage: Make all interrupt handlers asmlinkage / __visible 2013-08-06 14:18:23 -07:00
irqflags.h
ist.h UAPI: (Scripted) Disintegrate arch/x86/include/asm 2012-12-14 22:37:13 +00:00
jump_label.h compiler/gcc4: Add quirk for 'asm goto' miscompilation bug 2013-10-11 07:39:14 +02:00
kbdleds.h
Kbuild sched, x86: Provide a per-cpu preempt_count implementation 2013-09-25 14:07:57 +02:00
kdebug.h x86/dumpstack: Fix printk_address for direct addresses 2013-11-12 21:06:06 +01:00
kexec.h x86, kexec: Remove 1024G limitation for kexec buffer on 64bit 2013-01-29 15:26:23 -08:00
kgdb.h
kmap_types.h
kmemcheck.h
kprobes.h x86, asmlinkage: Make kprobes code visible and fix assembler code 2013-08-06 14:19:48 -07:00
kvm_emulate.h kvm, emulator: Rename VendorSpecific flag 2013-10-30 18:54:40 +01:00
kvm_guest.h x86: kvm guest: pvclock vsyscall support 2012-11-27 23:29:10 -02:00
kvm_host.h kvm: Delete prototype for non-existent function kvm_check_iopl 2013-11-06 11:07:09 +02:00
kvm_para.h Merge branch 'x86-spinlocks-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2013-09-04 11:55:10 -07:00
lguest_hcall.h
lguest.h lguest: map Switcher below fixmap. 2013-04-22 15:45:03 +09:30
linkage.h ix86: Tighten asmlinkage_protect() constraints 2013-01-24 11:25:59 +01:00
local64.h
local.h x86, bitops: Correct the assembly constraints to testing bitops 2013-12-04 14:31:28 -08:00
mach_timer.h
mach_traps.h
math_emu.h
mc146818rtc.h x86: Increase precision of x86_platform.get/set_wallclock() 2013-05-28 14:00:59 -07:00
mce.h ACPI, x86: Extended error log driver for x86 platform 2013-10-23 10:09:07 -07:00
microcode_amd.h x86, microcode, AMD: Fix early microcode loading 2013-08-12 18:32:45 +02:00
microcode_intel.h x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
microcode.h x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
misc.h x86: Improve the printout of the SMP bootup CPU table 2013-09-28 10:10:26 +02:00
mmconfig.h x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
mmu_context.h sched/x86: Optimize switch_mm() for multi-threaded workloads 2013-08-01 09:10:26 +02:00
mmu.h
mmx.h
mmzone_32.h x86-32, mm: Remove reference to resume_map_numa_kva() 2013-01-31 14:12:30 -08:00
mmzone_64.h
mmzone.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
module.h x86, 386 removal: Remove CONFIG_M386 from Kconfig 2012-11-29 13:23:01 -08:00
mpspec_def.h
mpspec.h x86 / ACPI: simplify _acpi_map_lsapic() 2013-09-24 01:39:40 +02:00
mshyperv.h x86, trace: Add irq vector tracepoints 2013-06-20 22:25:34 -07:00
msidef.h
msr.h x86 / msr: add 64bit _on_cpu access functions 2013-10-17 00:36:06 +02:00
mtrr.h Add arch_phys_wc_{add, del} to manipulate WC MTRRs if needed 2013-05-31 13:02:52 +10:00
mutex_32.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
mutex_64.h compiler/gcc4: Add quirk for 'asm goto' miscompilation bug 2013-10-11 07:39:14 +02:00
mutex.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
mwait.h intel_idle: remove assumption of one C-state per MWAIT flag 2013-02-08 19:29:16 -05:00
nmi.h x86: Fix trigger_all_cpu_backtrace() implementation 2013-06-20 14:00:21 +02:00
nops.h
numa_32.h
numa.h x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
numaq.h
olpc_ofw.h
olpc.h
page_32_types.h x86, relocs: Move ELF relocation handling to C 2013-08-07 21:00:04 -07:00
page_32.h x86: Make it so that __pa_symbol can only process kernel symbols on x86_64 2012-11-16 16:42:09 -08:00
page_64_types.h x86, relocs: Move ELF relocation handling to C 2013-08-07 21:00:04 -07:00
page_64.h x86: Make it so that __pa_symbol can only process kernel symbols on x86_64 2012-11-16 16:42:09 -08:00
page_types.h x86, relocs: Move ELF relocation handling to C 2013-08-07 21:00:04 -07:00
page.h Merge remote-tracking branch 'origin/x86/mm' into x86/mm2 2013-02-01 02:28:36 -08:00
paravirt_types.h Merge branch 'x86-spinlocks-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2013-09-04 11:55:10 -07:00
paravirt.h x86, ticketlock: Add slowpath logic 2013-08-09 07:54:00 -07:00
parport.h X86: drivers: remove __dev* attributes. 2013-01-03 15:57:04 -08:00
pat.h
pci_64.h
pci_x86.h PCI changes for the v3.9 merge window: 2013-02-25 21:18:18 -08:00
pci-direct.h
pci-functions.h
pci.h ACPI / driver core: Store an ACPI device pointer in struct acpi_dev_node 2013-11-14 23:14:43 +01:00
percpu.h Merge branch 'for-3.13' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu 2013-11-13 15:17:16 +09:00
perf_event_p4.h perf/x86/intel/P4: Robistify P4 PMU types 2013-04-26 09:31:41 +02:00
perf_event.h perf/x86/intel: Add simple Haswell PMU support 2013-06-19 14:43:33 +02:00
pgalloc.h x86, mm: enable split page table lock for PMD level 2013-11-15 09:32:15 +09:00
pgtable_32_types.h
pgtable_32.h x86/mm: Convert update_mmu_cache() and update_mmu_cache_pmd() to functions 2013-01-24 16:12:13 +01:00
pgtable_64_types.h x86, 64bit: Use a #PF handler to materialize early mappings on demand 2013-01-29 15:20:06 -08:00
pgtable_64.h Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2013-02-21 18:06:55 -08:00
pgtable_types.h mm: make sure _PAGE_SWP_SOFT_DIRTY bit is not set on present pte 2013-09-11 15:58:06 -07:00
pgtable-2level_types.h
pgtable-2level.h mm: save soft-dirty bits on file pages 2013-08-13 17:57:48 -07:00
pgtable-3level_types.h
pgtable-3level.h mm: save soft-dirty bits on file pages 2013-08-13 17:57:48 -07:00
pgtable.h mm: fix TLB flush race between migration, and change_protection_range 2013-12-18 19:04:51 -08:00
posix_types.h UAPI: (Scripted) Disintegrate arch/x86/include/asm 2012-12-14 22:37:13 +00:00
preempt.h sched: Remove PREEMPT_NEED_RESCHED from generic code 2013-12-11 15:52:32 +01:00
probe_roms.h
processor-cyrix.h
processor-flags.h UAPI: (Scripted) Disintegrate arch/x86/include/asm 2012-12-14 22:37:13 +00:00
processor.h x86: move fpu_counter into ARCH specific thread_struct 2013-11-13 12:09:13 +09:00
prom.h of: remove HAVE_ARCH_DEVTREE_FIXUPS 2013-10-09 20:04:08 -05:00
proto.h x86-64: don't set the early IDT to point directly to 'early_idt_handler' 2013-02-22 13:09:51 -08:00
ptrace.h Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/signal 2012-12-20 18:05:28 -08:00
pvclock-abi.h
pvclock.h pvclock: detect watchdog reset at pvclock read 2013-11-06 09:48:43 +02:00
realmode.h x86, realmode: Separate real_mode reserve and setup 2013-01-29 15:13:24 -08:00
reboot_fixups.h
reboot.h
required-features.h x86: Require MOVBE feature in cpuid when we use it 2013-01-28 16:59:55 -08:00
resume-trace.h
rio.h
rmwcc.h x86, bitops: Correct the assembly constraints to testing bitops 2013-12-04 14:31:28 -08:00
rtc.h
rwlock.h
rwsem.h x86 rwsem: avoid taking slow path when stealing write lock 2013-05-07 07:20:17 -07:00
scatterlist.h
seccomp_32.h
seccomp_64.h
seccomp.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
sections.h
segment.h x86, trace: Register exception handler to trace IDT 2013-11-08 14:15:45 -08:00
serial.h
setup_arch.h
setup.h intel_mid: Renamed *mrst* to *intel_mid* 2013-10-17 16:40:47 -07:00
shmparam.h
sigcontext.h UAPI: (Scripted) Disintegrate arch/x86/include/asm 2012-12-14 22:37:13 +00:00
sigframe.h
sighandling.h x86/signals: Propagate RF EFLAGS bit through the signal restore call 2013-05-28 08:46:50 +02:00
signal.h x86: switch to generic old sigaction 2013-02-03 18:16:27 -05:00
simd.h crypto: move x86 to the generic version of ablk_helper 2013-09-24 06:02:24 +10:00
smap.h
smp.h x86: delete __cpuinit usage from all x86 files 2013-07-14 19:36:56 -04:00
smpboot_hooks.h
sparsemem.h
special_insns.h x86, asmlinkage: Make several variables used from assembler/linker script visible 2013-08-06 14:20:13 -07:00
spinlock_types.h x86, ticketlock: Add slowpath logic 2013-08-09 07:54:00 -07:00
spinlock.h Linux 3.11-rc7 2013-09-09 12:05:37 -04:00
sta2x11.h
stackprotector.h
stacktrace.h
string_32.h
string_64.h
string.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
suspend_32.h x86, gdt, hibernate: Store/load GDT for hibernate path. 2013-05-02 11:27:35 -07:00
suspend_64.h x86, gdt, hibernate: Store/load GDT for hibernate path. 2013-05-02 11:27:35 -07:00
suspend.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
svm.h UAPI: (Scripted) Disintegrate arch/x86/include/asm 2012-12-14 22:37:13 +00:00
swiotlb.h
switch_to.h x86, asmlinkage: Make 32bit/64bit __switch_to visible 2013-08-06 14:18:30 -07:00
sync_bitops.h x86, bitops: Change bitops to be native operand size 2013-07-16 15:24:04 -07:00
sys_ia32.h unify compat fanotify_mark(2), switch to COMPAT_SYSCALL_DEFINE 2013-05-09 13:46:38 -04:00
syscall.h x86: Fix sys_call_table type in asm/syscall.h 2013-08-06 14:18:08 -07:00
syscalls.h x86, asmlinkage: Make various syscalls asmlinkage 2013-08-06 14:18:33 -07:00
sysfb.h x86: sysfb: move EFI quirks from efifb to sysfb 2013-08-02 16:17:47 -07:00
tce.h
thread_info.h preempt: Make PREEMPT_ACTIVE generic 2013-11-13 20:21:47 +01:00
time.h
timer.h
timex.h
tlb.h x86-32: Fix possible incomplete TLB invalidate with PAE pagetables 2013-04-12 16:56:47 -07:00
tlbflush.h mm: vmstats: track TLB flush stats on UP too 2013-09-11 15:57:09 -07:00
topology.h x86 / cpu topology: remove the stale macro arch_provides_topology_pointers 2013-07-29 13:12:45 -07:00
trace_clock.h tracing: Format non-nanosec times from tsc clock without a decimal point. 2012-11-13 15:48:40 -05:00
traps.h x86, trace: Register exception handler to trace IDT 2013-11-08 14:15:45 -08:00
tsc.h perf/x86: Add ability to calculate TSC from perf sample timestamps 2013-07-23 12:17:45 +02:00
uaccess_32.h x86: Unify copy_to_user() and add size checking to it 2013-10-26 12:27:37 +02:00
uaccess_64.h Merge branch 'x86-uaccess-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2013-11-12 11:46:06 +09:00
uaccess.h x86: Unify copy_to_user() and add size checking to it 2013-10-26 12:27:37 +02:00
unaligned.h
unistd.h consolidate cond_syscall and SYSCALL_ALIAS declarations 2013-03-03 22:55:19 -05:00
uprobes.h uprobes: Introduce arch_uprobe->ixol 2013-11-06 20:00:05 +01:00
user32.h
user_32.h
user_64.h
user.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
vdso.h
vga.h
vgtod.h
virtext.h
vm86.h UAPI: (Scripted) Disintegrate arch/x86/include/asm 2012-12-14 22:37:13 +00:00
vmx.h nEPT: Nested INVEPT 2013-08-07 15:57:42 +02:00
vsyscall.h UAPI: (Scripted) Disintegrate arch/x86/include/asm 2012-12-14 22:37:13 +00:00
vvar.h x86, asmlinkage, vdso: Mark vdso variables __visible 2013-08-06 14:21:08 -07:00
word-at-a-time.h
x2apic.h
x86_init.h PCI: Add x86_msi.msi_mask_irq() and msix_mask_irq() 2013-11-06 16:32:19 -07:00
xcr.h
xor_32.h x86/xor: Add alternative SSE implementation only prefetching once per 64-byte line 2013-01-25 09:23:50 +01:00
xor_64.h x86/xor: Add alternative SSE implementation only prefetching once per 64-byte line 2013-01-25 09:23:50 +01:00
xor_avx.h crypto: xor - Check for osxsave as well as avx in crypto/xor 2013-08-21 21:08:35 +10:00
xor.h x86/xor: Add alternative SSE implementation only prefetching once per 64-byte line 2013-01-25 09:23:50 +01:00
xsave.h