linux_dsm_epyc7002/arch/x86/kernel
Thomas Gleixner 4c71a2b6fd x86/speculation: Prepare for conditional IBPB in switch_mm()
The IBPB speculation barrier is issued from switch_mm() when the kernel
switches to a user space task with a different mm than the user space task
which ran last on the same CPU.

An additional optimization is to avoid IBPB when the incoming task can be
ptraced by the outgoing task. This optimization only works when switching
directly between two user space tasks. When switching from a kernel task to
a user space task the optimization fails because the previous task cannot
be accessed anymore. So for quite some scenarios the optimization is just
adding overhead.

The upcoming conditional IBPB support will issue IBPB only for user space
tasks which have the TIF_SPEC_IB bit set. This requires to handle the
following cases:

  1) Switch from a user space task (potential attacker) which has
     TIF_SPEC_IB set to a user space task (potential victim) which has
     TIF_SPEC_IB not set.

  2) Switch from a user space task (potential attacker) which has
     TIF_SPEC_IB not set to a user space task (potential victim) which has
     TIF_SPEC_IB set.

This needs to be optimized for the case where the IBPB can be avoided when
only kernel threads ran in between user space tasks which belong to the
same process.

The current check whether two tasks belong to the same context is using the
tasks context id. While correct, it's simpler to use the mm pointer because
it allows to mangle the TIF_SPEC_IB bit into it. The context id based
mechanism requires extra storage, which creates worse code.

When a task is scheduled out its TIF_SPEC_IB bit is mangled as bit 0 into
the per CPU storage which is used to track the last user space mm which was
running on a CPU. This bit can be used together with the TIF_SPEC_IB bit of
the incoming task to make the decision whether IBPB needs to be issued or
not to cover the two cases above.

As conditional IBPB is going to be the default, remove the dubious ptrace
check for the IBPB always case and simply issue IBPB always when the
process changes.

Move the storage to a different place in the struct as the original one
created a hole.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Jiri Kosina <jkosina@suse.cz>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: David Woodhouse <dwmw@amazon.co.uk>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Casey Schaufler <casey.schaufler@intel.com>
Cc: Asit Mallick <asit.k.mallick@intel.com>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Jon Masters <jcm@redhat.com>
Cc: Waiman Long <longman9394@gmail.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: Dave Stewart <david.c.stewart@intel.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20181125185005.466447057@linutronix.de
2018-11-28 11:57:11 +01:00
..
acpi memblock: stop using implicit alignment to SMP_CACHE_BYTES 2018-10-31 08:54:16 -07:00
apic memblock: stop using implicit alignment to SMP_CACHE_BYTES 2018-10-31 08:54:16 -07:00
cpu x86/speculation: Prepare for conditional IBPB in switch_mm() 2018-11-28 11:57:11 +01:00
fpu x86/fpu: Remove second definition of fpu in __fpu__restore_sig() 2018-10-17 12:30:31 +02:00
kprobes Merge branch 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-10-23 18:43:04 +01:00
.gitignore
alternative.c Merge branch 'x86-paravirt-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-10-23 17:54:58 +01:00
amd_gart_64.c dma-mapping: merge direct and noncoherent ops 2018-09-20 09:01:15 +02:00
amd_nb.c x86/pci, x86/amd_nb: Add Hygon Dhyana support to PCI and northbridge 2018-09-27 18:28:58 +02:00
apb_timer.c
aperture_64.c
apm_32.c x86/APM: Fix build warning when PROC_FS is not enabled 2018-09-15 10:16:25 +02:00
asm-offsets_32.c x86/entry/32: Load task stack from x86_tss.sp1 in SYSENTER handler 2018-07-20 01:11:36 +02:00
asm-offsets_64.c x86/paravirt: Move the pv_irq_ops under the PARAVIRT_XXL umbrella 2018-09-03 16:50:36 +02:00
asm-offsets.c Merge branch 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-10-23 18:43:04 +01:00
audit_64.c
bootflag.c
check.c x86/corruption-check: Use pr_*() instead of printk() 2018-09-10 14:47:33 +02:00
cpuid.c x86/cpuid: Allow cpuid_read() to schedule 2018-03-27 12:01:48 +02:00
crash_dump_32.c
crash_dump_64.c kdump, proc/vmcore: Enable kdumping encrypted memory with SME enabled 2018-10-06 12:09:26 +02:00
crash.c kexec_file, x86: move re-factored code to generic side 2018-04-13 17:10:27 -07:00
devicetree.c x86: DT: use for_each_of_cpu_node iterator 2018-09-28 14:25:58 -05:00
doublefault.c
dumpstack_32.c
dumpstack_64.c
dumpstack.c x86/process: Don't mix user/kernel regs in 64bit __show_regs() 2018-09-06 14:33:12 +02:00
e820.c memblock: stop using implicit alignment to SMP_CACHE_BYTES 2018-10-31 08:54:16 -07:00
early_printk.c x86/earlyprintk: Add a force option for pciserial device 2018-10-02 21:02:47 +02:00
early-quirks.c On GEM side: 2018-07-20 12:29:24 +10:00
ebda.c
eisa.c x86/EISA: Don't probe EISA bus for Xen PV guests 2018-09-11 23:36:50 +02:00
espfix_64.c x86/espfix: Document use of _PAGE_GLOBAL 2018-04-09 18:27:33 +02:00
ftrace_32.S
ftrace_64.S
ftrace.c
head32.c x86/boot: Add ACPI RSDP address to setup_header 2018-10-10 10:44:22 +02:00
head64.c x86: Clean up 'sizeof x' => 'sizeof(x)' 2018-10-29 07:13:28 +01:00
head_32.S x86/pgtable/32: Allocate 8k page-tables when PTI is enabled 2018-07-20 01:11:41 +02:00
head_64.S Merge branch 'x86-paravirt-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-10-23 17:54:58 +01:00
hpet.c x86: Don't include linux/irq.h from asm/hardirq.h 2018-08-05 09:53:13 +02:00
hw_breakpoint.c perf/arch/x86: Implement hw_breakpoint_arch_parse() 2018-06-26 09:07:55 +02:00
i8237.c x86/i8237: Register device based on FADT legacy boot flag 2018-04-27 16:44:29 +02:00
i8253.c
i8259.c x86: Don't include linux/irq.h from asm/hardirq.h 2018-08-05 09:53:13 +02:00
idt.c x86: Don't include linux/irq.h from asm/hardirq.h 2018-08-05 09:53:13 +02:00
io_delay.c
ioport.c x86/ioport: add ksys_ioperm() helper; remove in-kernel calls to sys_ioperm() 2018-04-02 20:16:12 +02:00
irq_32.c x86: Don't include linux/irq.h from asm/hardirq.h 2018-08-05 09:53:13 +02:00
irq_64.c x86: Don't include linux/irq.h from asm/hardirq.h 2018-08-05 09:53:13 +02:00
irq_work.c
irq.c x86: Don't include linux/irq.h from asm/hardirq.h 2018-08-05 09:53:13 +02:00
irqflags.S x86/paravirt: Make native_save_fl() extern inline 2018-07-03 10:56:27 +02:00
irqinit.c x86: Don't include linux/irq.h from asm/hardirq.h 2018-08-05 09:53:13 +02:00
itmt.c
jailhouse.c x86: Convert x86_platform_ops to timespec64 2018-05-19 14:03:14 +02:00
jump_label.c x86/jump_label: Switch to jump_entry accessors 2018-09-27 17:56:48 +02:00
kdebugfs.c
kexec-bzimage64.c Fix kexec forbidding kernels signed with keys in the secondary keyring to boot 2018-08-16 09:57:20 -07:00
kgdb.c
ksysfs.c treewide: kmalloc() -> kmalloc_array() 2018-06-12 16:19:22 -07:00
kvm.c x86/paravirt: Use a single ops structure 2018-09-03 16:50:35 +02:00
kvmclock.c Merge branch 'x86-paravirt-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-10-23 17:54:58 +01:00
ldt.c x86/ldt: Remove unused variable in map_ldt_struct() 2018-11-06 21:35:11 +01:00
livepatch.c
machine_kexec_32.c x86/kexec: Allocate 8k PGDs for PTI 2018-07-30 13:53:48 +02:00
machine_kexec_64.c x86/mm: Stop pretending pgtable_l5_enabled is a variable 2018-05-19 11:56:57 +02:00
macros.S x86/jump-labels: Macrofy inline assembly code to work around GCC inlining bugs 2018-10-06 15:52:17 +02:00
Makefile x86/paravirt: Make native_save_fl() extern inline 2018-07-03 10:56:27 +02:00
mmconf-fam10h_64.c
module.c x86: Add support for 64-bit place relative relocations 2018-09-27 17:56:47 +02:00
mpparse.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
msr.c x86: Clean up 'sizeof x' => 'sizeof(x)' 2018-10-29 07:13:28 +01:00
nmi_selftest.c
nmi.c
paravirt_patch_32.c x86/paravirt: Remove unused _paravirt_ident_32 2018-10-30 09:55:31 +01:00
paravirt_patch_64.c x86/paravirt: Remove unused _paravirt_ident_32 2018-10-30 09:55:31 +01:00
paravirt-spinlocks.c x86/paravirt: Use a single ops structure 2018-09-03 16:50:35 +02:00
paravirt.c x86/paravirt: Remove unused _paravirt_ident_32 2018-10-30 09:55:31 +01:00
pci-calgary_64.c
pci-dma.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
pci-iommu_table.c x86/iommu: Use NULL instead of 0 2018-08-02 14:33:19 +02:00
pci-swiotlb.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
pcspeaker.c x86/platform/pcspeaker: Use PTR_ERR_OR_ZERO() to fix ptr_ret.cocci warning 2018-07-24 09:46:42 +02:00
perf_regs.c perf/x86: Store user space frame-pointer value on a sample 2018-05-25 08:11:12 +02:00
platform-quirks.c x86/i8237: Register device based on FADT legacy boot flag 2018-04-27 16:44:29 +02:00
pmem.c
probe_roms.c
process_32.c x86/process: Consolidate and simplify switch_to_xtra() code 2018-11-28 11:57:11 +01:00
process_64.c x86/process: Consolidate and simplify switch_to_xtra() code 2018-11-28 11:57:11 +01:00
process.c x86/process: Consolidate and simplify switch_to_xtra() code 2018-11-28 11:57:11 +01:00
process.h x86/speculation: Avoid __switch_to_xtra() calls 2018-11-28 11:57:11 +01:00
ptrace.c Merge branch 'siginfo-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2018-10-24 11:22:39 +01:00
pvclock.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
quirks.c x86/mce: Check for alternate indication of machine check recovery on Skylake 2018-06-07 22:22:12 +02:00
reboot_fixups_32.c
reboot.c
relocate_kernel_32.S
relocate_kernel_64.S
resource.c
rtc.c x86: Convert x86_platform_ops to timespec64 2018-05-19 14:03:14 +02:00
setup_percpu.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
setup.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
signal_compat.c signal: Add TRAP_UNK si_code for undiagnosted trap exceptions 2018-04-25 10:40:56 -05:00
signal.c rseq: Avoid infinite recursion when delivering SIGSEGV 2018-06-22 19:04:22 +02:00
smp.c x86/irq: Let interrupt handlers set kvm_cpu_l1tf_flush_l1d 2018-08-05 09:53:13 +02:00
smpboot.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
stacktrace.c x86/stacktrace: Do not fail for ORC with regs on stack 2018-06-21 16:34:56 +02:00
step.c
sys_x86_64.c x86/compat: Adjust in_compat_syscall() to generic code under !COMPAT 2018-11-01 12:59:25 +01:00
sysfb_efi.c
sysfb_simplefb.c
sysfb.c
tboot.c
tce_64.c mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
time.c Merge branch 'x86-vdso-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-10-23 19:07:25 +01:00
tls.c
tls.h
topology.c x86/xen: Disable CPU0 hotplug for Xen PV 2018-09-12 21:15:02 +02:00
trace_clock.c
tracepoint.c
traps.c x86/traps: Use format string with panic() call 2018-10-29 07:19:26 +01:00
tsc_msr.c x86/cpu: Sanitize FAM6_ATOM naming 2018-10-02 10:14:32 +02:00
tsc_sync.c
tsc.c Merge branch 'x86-paravirt-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-10-23 17:54:58 +01:00
umip.c signal/x86: Use force_sig_fault where appropriate 2018-09-21 15:30:54 +02:00
unwind_frame.c
unwind_guess.c
unwind_orc.c x86/unwind/orc: Detect the end of the stack 2018-06-21 16:34:56 +02:00
uprobes.c signal: Properly deliver SIGSEGV from x86 uprobes 2018-09-11 21:18:53 +02:00
verify_cpu.S
vm86_32.c x86/mm/tlb: Add freed_tables argument to flush_tlb_mm_range 2018-10-09 16:51:12 +02:00
vmlinux.lds.S Merge branch 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-10-23 18:43:04 +01:00
vsmp_64.c x86/vsmp: Remove dependency on pv_irq_ops 2018-11-06 21:35:11 +01:00
x86_init.c x86/acpi, x86/boot: Take RSDP address for boot params if available 2018-10-10 10:44:22 +02:00