linux_dsm_epyc7002/arch/x86
David Woodhouse 928a4c3948 KVM/x86: Reduce retpoline performance impact in slot_handle_level_range(), by always inlining iterator helper methods
With retpoline, tight loops of "call this function for every XXX" are
very much pessimised by taking a prediction miss *every* time. This one
is by far the biggest contributor to the guest launch time with retpoline.

By marking the iterator slot_handle_…() functions always_inline, we can
ensure that the indirect function call can be optimised away into a
direct call and it actually generates slightly smaller code because
some of the other conditionals can get optimised away too.

Performance is now pretty close to what we see with nospectre_v2 on
the command line.

Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Tested-by: Filippo Sironi <sironi@amazon.de>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Filippo Sironi <sironi@amazon.de>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: arjan.van.de.ven@intel.com
Cc: dave.hansen@intel.com
Cc: jmattson@google.com
Cc: karahmed@amazon.de
Cc: kvm@vger.kernel.org
Cc: rkrcmar@redhat.com
Link: http://lkml.kernel.org/r/1518305967-31356-4-git-send-email-dwmw@amazon.co.uk
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2018-02-13 08:59:45 +01:00
..
boot Merge branch 'x86/urgent' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2017-12-31 13:13:56 -08:00
configs x86/unwind: Rename unwinder config options to 'CONFIG_UNWINDER_*' 2017-10-14 10:12:12 +02:00
crypto Merge branch 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-01-14 09:51:25 -08:00
entry x86/mm/pti: Fix PTI comment in entry_SYSCALL_64() 2018-02-10 12:44:51 +01:00
events Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-01-28 12:19:23 -08:00
hyperv Char/Misc patches for 4.15-rc1 2017-11-16 09:10:59 -08:00
ia32 License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
include Revert "x86/speculation: Simplify indirect_branch_prediction_barrier()" 2018-02-13 08:59:00 +01:00
kernel Revert "x86/speculation: Simplify indirect_branch_prediction_barrier()" 2018-02-13 08:59:00 +01:00
kvm KVM/x86: Reduce retpoline performance impact in slot_handle_level_range(), by always inlining iterator helper methods 2018-02-13 08:59:45 +01:00
lib x86/get_user: Use pointer masking to limit speculation 2018-01-30 21:54:31 +01:00
math-emu License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
mm x86/speculation: Use Indirect Branch Prediction Barrier in context switch 2018-01-30 23:09:21 +01:00
net bpf: fix bpf_tail_call() x64 JIT 2017-10-03 16:04:44 -07:00
oprofile Modules updates for v4.15 2017-11-15 13:46:33 -08:00
pci x86/PCI: Enable AMD 64-bit window on resume 2018-01-16 11:45:50 -06:00
platform Merge branch 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-01-14 09:51:25 -08:00
power Merge branch 'WIP.x86-pti.entry-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2017-12-18 08:59:15 -08:00
purgatory License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ras License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
realmode x86/realmode: Don't decrypt trampoline area under SEV 2017-11-07 15:35:55 +01:00
tools License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
um Merge commit 'upstream-x86-entry' into WIP.x86/mm 2017-12-17 12:58:53 +01:00
video
xen xen: fixes for 4.15-rc8 2018-01-12 10:00:15 -08:00
.gitignore
Kbuild Merge branch 'x86-platform-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2017-09-07 09:25:15 -07:00
Kconfig Merge branch 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-01-14 09:51:25 -08:00
Kconfig.cpu License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
Kconfig.debug Merge commit 'upstream-x86-entry' into WIP.x86/mm 2017-12-17 12:58:53 +01:00
Makefile Merge branch 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-01-14 15:30:02 -08:00
Makefile_32.cpu License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
Makefile.um License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00