linux_dsm_epyc7002/arch
Steven Rostedt 0c54dd341f ftrace: Remove memory barriers from NMI code when not needed
The code in stop_machine that modifies the kernel text has a bit
of logic to handle the case of NMIs. stop_machine does not prevent
NMIs from executing, and if an NMI were to trigger on another CPU
as the modifying CPU is changing the NMI text, a GPF could result.

To prevent the GPF, the NMI calls ftrace_nmi_enter() which may
modify the code first, then any other NMIs will just change the
text to the same content which will do no harm. The code that
stop_machine called must wait for NMIs to finish while it changes
each location in the kernel. That code may also change the text
to what the NMI changed it to. The key is that the text will never
change content while another CPU is executing it.

To make the above work, the call to ftrace_nmi_enter() must also
do a smp_mb() as well as atomic_inc().  But for applications like
perf that require a high number of NMIs for profiling, this can have
a dramatic effect on the system. Not only is it doing a full memory
barrier on both nmi_enter() as well as nmi_exit() it is also
modifying a global variable with an atomic operation. This kills
performance on large SMP machines.

Since the memory barriers are only needed when ftrace is in the
process of modifying the text (which is seldom), this patch
adds a "modifying_code" variable that gets set before stop machine
is executed and cleared afterwards.

The NMIs will check this variable and store it in a per CPU
"save_modifying_code" variable that it will use to check if it
needs to do the memory barriers and atomic dec on NMI exit.

Acked-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2010-02-25 08:42:06 -05:00
..
alpha alpha: Convert BUG() to use unreachable() 2009-12-18 01:14:52 -05:00
arm ARM: Merge next-simtec 2010-01-04 09:32:37 +09:00
avr32 Merge branch 'for-33' of git://repo.or.cz/linux-kbuild 2009-12-17 07:23:42 -08:00
blackfin Merge branch 'for-33' of git://repo.or.cz/linux-kbuild 2009-12-17 07:23:42 -08:00
cris Merge branch 'for-33' of git://repo.or.cz/linux-kbuild 2009-12-17 07:23:42 -08:00
frv Merge branch 'for-33' of git://repo.or.cz/linux-kbuild 2009-12-17 07:23:42 -08:00
h8300 Merge branch 'for-33' of git://repo.or.cz/linux-kbuild 2009-12-17 07:23:42 -08:00
ia64 KVM: ia64: fix build breakage due to host spinlock change 2009-12-27 13:36:33 -02:00
m32r elf: kill USE_ELF_CORE_DUMP 2009-12-16 07:20:12 -08:00
m68k Merge branch 'for-33' of git://repo.or.cz/linux-kbuild 2009-12-17 07:23:42 -08:00
m68knommu Unify sys_mmap* 2009-12-11 06:44:29 -05:00
microblaze Merge branch 'for-33' of git://repo.or.cz/linux-kbuild 2009-12-17 07:23:42 -08:00
mips Merge branch 'upstream' of git://ftp.linux-mips.org/pub/scm/upstream-linus 2009-12-17 16:38:06 -08:00
mn10300 Merge branch 'for-33' of git://repo.or.cz/linux-kbuild 2009-12-17 07:23:42 -08:00
parisc Merge branch 'for-33' of git://repo.or.cz/linux-kbuild 2009-12-17 07:23:42 -08:00
powerpc Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jbarnes/pci-2.6 2009-12-30 13:13:24 -08:00
s390 [S390] Update default configuration. 2010-01-04 09:05:58 +01:00
score Merge branch 'for-linus' of git://gitserver.sunplusct.com/linux-2.6-score 2009-12-17 08:08:57 -08:00
sh sh: update defconfigs. 2010-01-04 15:38:50 +09:00
sparc sparc64: Fix Niagara2 perf event handling. 2010-01-04 23:16:03 -08:00
um Merge branch 'for-33' of git://repo.or.cz/linux-kbuild 2009-12-17 07:23:42 -08:00
x86 ftrace: Remove memory barriers from NMI code when not needed 2010-02-25 08:42:06 -05:00
xtensa Merge branch 'for-33' of git://repo.or.cz/linux-kbuild 2009-12-17 07:23:42 -08:00
.gitignore
Kconfig hw-breakpoints: Fix hardware breakpoints -> perf events dependency 2009-12-18 13:11:51 +01:00