Teach buildid-cache how to add, remove, and update binary objects from
other mount namespaces. Allow probe events tracing binaries in
different namespaces to add their objects to the probe and build-id
caches too. As a handy side effect, this also lets us access SDT probes
in binaries from alternate mount namespaces.
Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
Tested-by: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas-Mich Richter <tmricht@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/1499305693-1599-5-git-send-email-kjlx@templeofstupid.com
[ Add util/namespaces.c to tools/perf/util/python-ext-sources, to fix the python binding 'perf test' ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Teaches perf how to place a uprobe on a file that's in a different mount
namespace. The user must add the probe using the --target-ns argument
to perf probe. Once it has been placed, it may be recorded against
without further namespace-specific commands.
Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
[ PPC build fixed by Ravi: ]
Link: http://lkml.kernel.org/r/1500287542-6219-1-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Cc: Thomas-Mich Richter <tmricht@linux.vnet.ibm.com>
[ Fix !HAVE_DWARF_SUPPORT build ]
Link: http://lkml.kernel.org/r/1499305693-1599-4-git-send-email-kjlx@templeofstupid.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
In commit 613f050d68 ("perf probe: Fix to probe on gcc generated
functions in modules"), the offset from symbol is, incorrectly, added
to the trace point address. This leads to incorrect probe trace points
for inlined functions and when using relative line number on symbols.
Prior this patch:
$ perf probe -m nf_nat -D in_range
p:probe/in_range nf_nat:in_range.isra.9+0
$ perf probe -m i40e -D i40e_clean_rx_irq
p:probe/i40e_clean_rx_irq i40e:i40e_napi_poll+2212
$ perf probe -m i40e -D i40e_clean_rx_irq:16
p:probe/i40e_clean_rx_irq i40e:i40e_lan_xmit_frame+626
After:
$ perf probe -m nf_nat -D in_range
p:probe/in_range nf_nat:in_range.isra.9+0
$ perf probe -m i40e -D i40e_clean_rx_irq
p:probe/i40e_clean_rx_irq i40e:i40e_napi_poll+1106
$ perf probe -m i40e -D i40e_clean_rx_irq:16
p:probe/i40e_clean_rx_irq i40e:i40e_napi_poll+2665
Committer testing:
Using 'pfunct', a tool found in the 'dwarves' package [1], one can ask what are
the functions that while not being explicitely marked as inline, were inlined
by the compiler:
# pfunct --cc_inlined /lib/modules/4.12.0-rc4+/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko | head
__ew32
e1000_regdump
e1000e_dump_ps_pages
e1000_desc_unused
e1000e_systim_to_hwtstamp
e1000e_rx_hwtstamp
e1000e_update_rdt_wa
e1000e_update_tdt_wa
e1000_put_txbuf
e1000_consume_page
Then ask 'perf probe' to produce the kprobe_tracer probe definitions for two of
them:
# perf probe -m e1000e -D e1000e_rx_hwtstamp
p:probe/e1000e_rx_hwtstamp e1000e:e1000_receive_skb+74
# perf probe -m e1000e -D e1000_consume_page
p:probe/e1000_consume_page e1000e:e1000_clean_jumbo_rx_irq+876
p:probe/e1000_consume_page_1 e1000e:e1000_clean_jumbo_rx_irq+1506
p:probe/e1000_consume_page_2 e1000e:e1000_clean_rx_irq_ps+1074
Now lets concentrate on the 'e1000_consume_page' one, that was inlined twice in
e1000_clean_jumbo_rx_irq(), lets see what readelf says about the DWARF tags for
that function:
$ readelf -wi /lib/modules/4.12.0-rc4+/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko
<SNIP>
<1><13e27b>: Abbrev Number: 121 (DW_TAG_subprogram)
<13e27c> DW_AT_name : (indirect string, offset: 0xa8945): e1000_clean_jumbo_rx_irq
<13e287> DW_AT_low_pc : 0x17a30
<3><13e6ef>: Abbrev Number: 119 (DW_TAG_inlined_subroutine)
<13e6f0> DW_AT_abstract_origin: <0x13ed2c>
<13e6f4> DW_AT_low_pc : 0x17be6
<SNIP>
<1><13ed2c>: Abbrev Number: 142 (DW_TAG_subprogram)
<13ed2e> DW_AT_name : (indirect string, offset: 0xa54c3): e1000_consume_page
So, the first time in e1000_clean_jumbo_rx_irq() where e1000_consume_page() is
inlined is at PC 0x17be6, which subtracted from e1000_clean_jumbo_rx_irq()'s
address, gives us the offset we should use in the probe definition:
0x17be6 - 0x17a30 = 438
but above we have 876, which is twice as much.
Lets see the second inline expansion of e1000_consume_page() in
e1000_clean_jumbo_rx_irq():
<3><13e86e>: Abbrev Number: 119 (DW_TAG_inlined_subroutine)
<13e86f> DW_AT_abstract_origin: <0x13ed2c>
<13e873> DW_AT_low_pc : 0x17d21
0x17d21 - 0x17a30 = 753
So we where adding it at twice the offset from the containing function as we
should.
And then after this patch:
# perf probe -m e1000e -D e1000e_rx_hwtstamp
p:probe/e1000e_rx_hwtstamp e1000e:e1000_receive_skb+37
# perf probe -m e1000e -D e1000_consume_page
p:probe/e1000_consume_page e1000e:e1000_clean_jumbo_rx_irq+438
p:probe/e1000_consume_page_1 e1000e:e1000_clean_jumbo_rx_irq+753
p:probe/e1000_consume_page_2 e1000e:e1000_clean_jumbo_rx_irq+1353
#
Which matches the two first expansions and shows that because we were
doubling the offset it would spill over the next function:
readelf -sw /lib/modules/4.12.0-rc4+/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko
673: 0000000000017a30 1626 FUNC LOCAL DEFAULT 2 e1000_clean_jumbo_rx_irq
674: 0000000000018090 2013 FUNC LOCAL DEFAULT 2 e1000_clean_rx_irq_ps
This is the 3rd inline expansion of e1000_consume_page() in
e1000_clean_jumbo_rx_irq():
<3><13ec77>: Abbrev Number: 119 (DW_TAG_inlined_subroutine)
<13ec78> DW_AT_abstract_origin: <0x13ed2c>
<13ec7c> DW_AT_low_pc : 0x17f79
0x17f79 - 0x17a30 = 1353
So:
0x17a30 + 2 * 1353 = 0x184c2
And:
0x184c2 - 0x18090 = 1074
Which explains the bogus third expansion for e1000_consume_page() to end up at:
p:probe/e1000_consume_page_2 e1000e:e1000_clean_rx_irq_ps+1074
All fixed now :-)
[1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/
Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Acked-by: Magnus Karlsson <magnus.karlsson@intel.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: stable@vger.kernel.org
Fixes: 613f050d68 ("perf probe: Fix to probe on gcc generated functions in modules")
Link: http://lkml.kernel.org/r/20170621164134.5701-1-bjorn.topel@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
There are places where we just need a forward declaration, and others
were we need to include strlist.h and/or strfilter.h, reducing the
impact of changes in headers on the build time, do it.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-zab42gbiki88y9k0csorxekb@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Moving them from util.h, where they don't belong. Since libc already
have string.h, name it slightly differently, as string2.h.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-eh3vz5sqxsrdd8lodoro4jrw@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
More stuff that came from git, out of the hodge-podge that is util.h
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-e3lana4gctz3ub4hn4y29hkw@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Needed to use the PRI[xu](32,64) formatting macros.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-wkbho8kaw24q67dd11q0j39f@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
There are many SDT markers in powerpc whose uprobe definition goes
beyond current MAX_CMDLEN, especially when target filename is long and
sdt marker has long list of arguments. For example, definition of sdt
marker
method__compile__end: 8@17 8@9 8@10 -4@8 8@7 -4@6 8@5 -4@4 1@37(28)
from file
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.91-2.b14.fc22.ppc64/jre/lib/ppc64/server/libjvm.so
is
p:sdt_hotspot/method__compile__end /usr/lib/jvm/java-1.8.0-openjdk-\
1.8.0.91-2.b14.fc22.ppc64/jre/lib/ppc64/server/libjvm.so:0x4c4e00\
arg1=%gpr17:u64 arg2=%gpr9:u64 arg3=%gpr10:u64 arg4=%gpr8:s32\
arg5=%gpr7:u64 arg6=%gpr6:s32 arg7=%gpr5:u64 arg8=%gpr4:s32\
arg9=+37(%gpr28):u8
'perf probe' fails with segfault for such markers. As the uprobe_events
file accepts definitions up to 4094 characters(4096 - 2 (\n\0)),
increase value of MAX_CMDLEN match that.
Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Alexis Berlemont <alexis.berlemont@gmail.com>
Cc: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20170207054547.3690-1-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
'*ntevs' contains number of elements present in 'tevs' array. If there
are no elements in array, 'tevs2' can be directly assigned to 'tevs'
without allocating more space. So the condition should be '*ntevs == 0'
not 'ntevs == 0'.
Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Fixes: 42bba263eb ("perf probe: Allow wildcard for cached events")
Link: http://lkml.kernel.org/r/20170308065908.4128-1-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We indicate support for accepting sym+offset with kretprobes through a
line in ftrace README. Parse the same to identify support and choose the
appropriate format for kprobe_events.
As an example, without this perf patch, but with the ftrace changes:
naveen@ubuntu:~/linux/tools/perf$ sudo cat /sys/kernel/debug/tracing/README | grep kretprobe
place (kretprobe): [<module>:]<symbol>[+<offset>]|<memaddr>
naveen@ubuntu:~/linux/tools/perf$
naveen@ubuntu:~/linux/tools/perf$ sudo ./perf probe -v do_open%return
probe-definition(0): do_open%return
symbol:do_open file:(null) line:0 offset:0 return:1 lazy:(null)
0 arguments
Looking at the vmlinux_path (8 entries long)
Using /boot/vmlinux for symbols
Open Debuginfo file: /boot/vmlinux
Try to find probe point from debuginfo.
Matched function: do_open [2d0c7d8]
Probe point found: do_open+0
Matched function: do_open [35d76b5]
found inline addr: 0xc0000000004ba984
Failed to find "do_open%return",
because do_open is an inlined function and has no return point.
An error occurred in debuginfo analysis (-22).
Trying to use symbols.
Opening /sys/kernel/debug/tracing//kprobe_events write=1
Writing event: r:probe/do_open do_open+0
Writing event: r:probe/do_open_1 do_open+0
Added new events:
probe:do_open (on do_open%return)
probe:do_open_1 (on do_open%return)
You can now use it in all perf tools, such as:
perf record -e probe:do_open_1 -aR sleep 1
naveen@ubuntu:~/linux/tools/perf$ sudo cat /sys/kernel/debug/kprobes/list
c000000000041370 k kretprobe_trampoline+0x0 [OPTIMIZED]
c0000000004433d0 r do_open+0x0 [DISABLED]
c0000000004433d0 r do_open+0x0 [DISABLED]
And after this patch (and the subsequent powerpc patch):
naveen@ubuntu:~/linux/tools/perf$ sudo ./perf probe -v do_open%return
probe-definition(0): do_open%return
symbol:do_open file:(null) line:0 offset:0 return:1 lazy:(null)
0 arguments
Looking at the vmlinux_path (8 entries long)
Using /boot/vmlinux for symbols
Open Debuginfo file: /boot/vmlinux
Try to find probe point from debuginfo.
Matched function: do_open [2d0c7d8]
Probe point found: do_open+0
Matched function: do_open [35d76b5]
found inline addr: 0xc0000000004ba984
Failed to find "do_open%return",
because do_open is an inlined function and has no return point.
An error occurred in debuginfo analysis (-22).
Trying to use symbols.
Opening /sys/kernel/debug/tracing//README write=0
Opening /sys/kernel/debug/tracing//kprobe_events write=1
Writing event: r:probe/do_open _text+4469712
Writing event: r:probe/do_open_1 _text+4956248
Added new events:
probe:do_open (on do_open%return)
probe:do_open_1 (on do_open%return)
You can now use it in all perf tools, such as:
perf record -e probe:do_open_1 -aR sleep 1
naveen@ubuntu:~/linux/tools/perf$ sudo cat /sys/kernel/debug/kprobes/list
c000000000041370 k kretprobe_trampoline+0x0 [OPTIMIZED]
c0000000004433d0 r do_open+0x0 [DISABLED]
c0000000004ba058 r do_open+0x8 [DISABLED]
Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: linuxppc-dev@lists.ozlabs.org
Link: http://lkml.kernel.org/r/496ef9f33c1ab16286ece9dd62aa672807aef91c.1488961018.git.naveen.n.rao@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It now can have negative value to suppress the message entirely. So it
needs to check it being positive.
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: kernel-team@lge.com
Link: http://lkml.kernel.org/r/20170217081742.17417-3-namhyung@kernel.org
[ Adjust fuzz on tools/perf/util/pmu.c, add > 0 checks in many other places ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Genuine problem detected with clang, the warnings are spot on:
util/probe-event.c:2079:7: error: variable 'map' is used uninitialized whenever 'if' condition is false [-Werror,-Wsometimes-uninitialized]
if (addr) {
^~~~
util/probe-event.c:2094:6: note: uninitialized use occurs here
if (map && !is_kprobe) {
^~~
util/probe-event.c:2079:3: note: remove the 'if' if its condition is always true
if (addr) {
^~~~~~~~~~
util/probe-event.c:2075:8: error: variable 'map' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (kernel_get_symbol_address_by_name(tp->symbol,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
util/probe-event.c:2094:6: note: uninitialized use occurs here
if (map && !is_kprobe) {
^~~
util/probe-event.c:2075:4: note: remove the 'if' if its condition is always false
if (kernel_get_symbol_address_by_name(tp->symbol,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
util/probe-event.c:2064:17: note: initialize the variable 'map' to silence this warning
struct map *map;
^
= NULL
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-m3501el55i10hctfbmi2qxzr@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
- Introduce 'perf ftrace' a perf front end to the kernel's ftrace
function and function_graph tracer, defaulting to the "function_graph"
tracer, more work will be done in reviving this effort, forward porting
it from its initial patch submission (Namhyung Kim)
- Add 'e' and 'c' hotkeys to expand/collapse call chains for a single
hist entry in the 'perf report' and 'perf top' TUI (Jiri Olsa)
Fixes:
- Fix wrong register name for arm64, used in 'perf probe' (He Kuang)
- Fix map offsets in relocation in libbpf (Joe Stringer)
- Fix looking up dwarf unwind stack info (Matija Glavinic Pecotic)
Infrastructure:
- libbpf prog functions sync with what is exported via uapi (Joe Stringer)
Trivial:
- Remove unnecessary checks and assignments in 'perf probe's
try_to_find_absolute_address() (Markus Elfring)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQIcBAABCAAGBQJYig7UAAoJENZQFvNTUqpAhJQP/iI0T7A8TNekPGLv7j20c302
89N9+9TAFtVqjgr1hIzqQgGOqbOdAW1tU3VTPW92nNDBn9JV5qwuF9YWEiDaAVv2
0bmV5hLnrNlymddm3pdg/PbD1TVlwk2NFxtrkPxuf/vx0ZhEGqsSrRUCR/xGXbtQ
TcMg3rQquspV9JNv4HzFdQC9nsG1CGNotZKsE1avRw70pWAqCtF81B0m8teb6OWo
5qnN+AMJlYcC+OGffROemUksuehkMvi5L8v1e/6RO/lU1qt9Jrc/2sT9cqvjVFNR
k4c76cUgWOCYzDEotENMpU4bc6e/24DE2ydFeovihdXw8Qs4ajEA9LXKM4yW+ZoE
MZE3GS153a8n+CvTfkB9Ow1QJ8rgmR/L0BuhmGb6bYW/MtuTRTShhSduZwOrIyap
9KckHYti4p3oN3CKFYGO9PN3DRUdx+Xqg/miwrgjkPo09QFp+lzfFFOk0P2/Zqw2
yfvdWeHxkkrwoWQIyMHVKp/E9jQPuyYqwnKdp68LCN+DgNiFpPpSA8id5e47RQDE
otqrK8U/82ktakfrBijSPBI6EEqFg7ltip2KT/xlDMfnP9HtxgFhzrk52dyi6pM/
jkBhJaTQhVZTyaFvUXuaLmBSdPpcaaGM4KJ+2iAayA2r0KLiDj6IdzD5ROCRFOvJ
SFA472mIxNxUjpQEUTtc
=tYKN
-----END PGP SIGNATURE-----
Merge tag 'perf-core-for-mingo-4.11-20170126' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core
Pull the latest perf/core updates from Arnaldo Carvalho de Melo:
New features:
- Introduce 'perf ftrace' a perf front end to the kernel's ftrace
function and function_graph tracer, defaulting to the "function_graph"
tracer, more work will be done in reviving this effort, forward porting
it from its initial patch submission (Namhyung Kim)
- Add 'e' and 'c' hotkeys to expand/collapse call chains for a single
hist entry in the 'perf report' and 'perf top' TUI (Jiri Olsa)
Fixes:
- Fix wrong register name for arm64, used in 'perf probe' (He Kuang)
- Fix map offsets in relocation in libbpf (Joe Stringer)
- Fix looking up dwarf unwind stack info (Matija Glavinic Pecotic)
Infrastructure changes:
- libbpf prog functions sync with what is exported via uapi (Joe Stringer)
Trivial changes:
- Remove unnecessary checks and assignments in 'perf probe's
try_to_find_absolute_address() (Markus Elfring)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Remove an error code assignment which is redundant in an if branch for
the handling of a memory allocation failure because the same value was
set for the local variable "err" before.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Milian Wolff <milian.wolff@kdab.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Cc: Wang Nan <wangnan0@huawei.com>
Cc: kernel-janitors@vger.kernel.org
Link: http://lkml.kernel.org/r/0ede09ec-79b6-c8bd-5b20-02c63ed98aab@users.sourceforge.net
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Remove a condition check which is unnecessary at the end
because this source code place should usually only be reached
with a non-zero pointer.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: He Kuang <hekuang@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Milian Wolff <milian.wolff@kdab.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Cc: Wang Nan <wangnan0@huawei.com>
Cc: kernel-janitors@vger.kernel.org
Link: http://lkml.kernel.org/r/a3f2473b-6383-a326-bce0-b826423608b8@users.sourceforge.net
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fix to probe on gcc generated functions on modules. Since
probing on a module is based on its symbol name, it should
be adjusted on actual symbols.
E.g. without this fix, perf probe shows probe definition
on non-exist symbol as below.
$ perf probe -m build-x86_64/net/netfilter/nf_nat.ko -F in_range*
in_range.isra.12
$ perf probe -m build-x86_64/net/netfilter/nf_nat.ko -D in_range
p:probe/in_range nf_nat:in_range+0
With this fix, perf probe correctly shows a probe on
gcc-generated symbol.
$ perf probe -m build-x86_64/net/netfilter/nf_nat.ko -D in_range
p:probe/in_range nf_nat:in_range.isra.12+0
This also fixes same problem on online module as below.
$ perf probe -m i915 -D assert_plane
p:probe/assert_plane i915:assert_plane.constprop.134+0
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/148411450673.9978.14905987549651656075.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add error check codes on post processing and improve it for offline
probe events as:
- post processing fails if no matched symbol found in map(-ENOENT)
or strdup() failed(-ENOMEM).
- Even if the symbol name is the same, it updates symbol address
and offset.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/148411443738.9978.4617979132625405545.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fix perf-probe to show probe definition on gcc generated symbols for
offline kernel (including cross-arch kernel image).
gcc sometimes optimizes functions and generate new symbols with suffixes
such as ".constprop.N" or ".isra.N" etc. Since those symbol names are
not recorded in DWARF, we have to find correct generated symbols from
offline ELF binary to probe on it (kallsyms doesn't correct it). For
online kernel or uprobes we don't need it because those are rebased on
_text, or a section relative address.
E.g. Without this:
$ perf probe -k build-arm/vmlinux -F __slab_alloc*
__slab_alloc.constprop.9
$ perf probe -k build-arm/vmlinux -D __slab_alloc
p:probe/__slab_alloc __slab_alloc+0
If you put above definition on target machine, it should fail
because there is no __slab_alloc in kallsyms.
With this fix, perf probe shows correct probe definition on
__slab_alloc.constprop.9:
$ perf probe -k build-arm/vmlinux -D __slab_alloc
p:probe/__slab_alloc __slab_alloc.constprop.9+0
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/148350060434.19001.11864836288580083501.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fix --funcs (-F) option to show correct symbols for offline module.
Since previous perf-probe uses machine__findnew_module_map() for offline
module, even if user passes a module file (with full path) which is for
other architecture, perf-probe always tries to load symbol map for
current kernel module.
This fix uses dso__new_map() to load the map from given binary as same
as a map for user applications.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/148350053478.19001.15435255244512631545.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since 'perf probe' supports cross-arch probes, it is possible to analyze
different arch kernel image which has different bits-per-long.
In that case, it fails to get the module name because it uses the
MOD_NAME_OFFSET macro based on the host machine bits-per-long, instead
of the target arch bits-per-long.
This fixes above issue by changing modname-offset based on the target
archs bit width. This is ok because linux kernel uses LP64 model on
64bit arch.
E.g. without this (on x86_64, and target module is arm32):
$ perf probe -m build-arm/fs/configfs/configfs.ko -D configfs_lookup
p:probe/configfs_lookup :configfs_lookup+0
^-Here is an empty module name.
With this fix, you can see correct module name:
$ perf probe -m build-arm/fs/configfs/configfs.ko -D configfs_lookup
p:probe/configfs_lookup configfs:configfs_lookup+0
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/148337043836.6752.383495516397005695.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Static anaylsis with cppcheck[1] detected an incorrect comparison:
[tools/perf/util/probe-event.c:216]: (warning) Char literal compared
with pointer 'ptr2'. Did you intend to dereference it?
Dereference ptr2 for the comparison to fix this.
1: https://sourceforge.net/p/cppcheck/wiki/Home/
Signed-off-by: Colin King <colin.king@canonical.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Cc: Wang Nan <wangnan0@huawei.com>
Fixes: 35726d3a4c ("perf probe: Fix to cut off incompatible chars from group name")
Link: http://lkml.kernel.org/r/20161003103431.18534-1-colin.king@canonical.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cut off the characters which can not use for group name of uprobes
when making it based on executable filename.
For example, if the exec name is libstdc++.so, without this fix
perf probe generates "probe_libstdc++" as the group name, but
it is failed to set because '+' can not be used for group name.
With this fix perf accepts only alphabet, number or '_' for group
name, thus perf generates "probe_libstdc" as the group name.
E.g. with this fix, you can see the event name has no "+".
----
$ ./perf probe -x /usr/lib64/libstdc++.so.6 -D is_open
p:probe_libstdc/is_open /usr/lib64/libstdc++.so.6.0.22:0x8ca80
p:probe_libstdc/is_open_1 /usr/lib64/libstdc++.so.6.0.22:0x8ca70
p:probe_libstdc/is_open_2 /usr/lib64/libstdc++.so.6.0.22:0x8ca60
p:probe_libstdc/is_open_3 /usr/lib64/libstdc++.so.6.0.22:0xb0ad0
p:probe_libstdc/is_open_4 /usr/lib64/libstdc++.so.6.0.22:0xecca9
----
Committer note:
Before this fix:
# perf probe -x /usr/lib64/libstdc++.so.6 is_open
Failed to write event: Invalid argument
Error: Failed to add events.
#
After the fix:
# perf probe -x /usr/lib64/libstdc++.so.6 is_open
Added new events:
probe_libstdc:is_open (on is_open in /usr/lib64/libstdc++.so.6.0.22)
probe_libstdc:is_open_1 (on is_open in /usr/lib64/libstdc++.so.6.0.22)
probe_libstdc:is_open_2 (on is_open in /usr/lib64/libstdc++.so.6.0.22)
probe_libstdc:is_open_3 (on is_open in /usr/lib64/libstdc++.so.6.0.22)
probe_libstdc:is_open_4 (on is_open in /usr/lib64/libstdc++.so.6.0.22)
You can now use it in all perf tools, such as:
perf record -e probe_libstdc:is_open_4 -aR sleep 1
# perf probe -l probe_libstdc:*
probe_libstdc:is_open (on is_open@libstdc++-v3/include/fstream in /usr/lib64/libstdc++.so.6.0.22)
probe_libstdc:is_open_1 (on is_open@libstdc++-v3/include/fstream in /usr/lib64/libstdc++.so.6.0.22)
probe_libstdc:is_open_2 (on is_open@libstdc++-v3/include/fstream in /usr/lib64/libstdc++.so.6.0.22)
probe_libstdc:is_open_3 (on is_open@src/c++98/basic_file.cc in /usr/lib64/libstdc++.so.6.0.22)
probe_libstdc:is_open_4 (on stdio_filebuf:5@include/ext/stdio_filebuf.h in /usr/lib64/libstdc++.so.6.0.22)
#
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Jiri Olsa <jolsa@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/147464491667.29804.9553638175441827970.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We're not using it anymore, few users were, but we really could do
without it, simplify lots of functions by removing it.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-1zng8wdznn00iiz08bb7q3vn@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Introduce helper function instead of inline code and replace hardcoded
strings "$vars" and "$params" with their corresponding macros.
perf_probe_with_var() is not declared as static since it will be called
from different file in subsequent patch.
Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1470214725-5023-1-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Ignore the buildid of running kernel when both of --definition and
--vmlinux is given because that kernel should be off-line.
This also skips post-processing of kprobe event for relocating symbol
and checking blacklist, because it can not be done on off-line kernel.
E.g. without this fix perf shows an error as below
----
$ perf probe --vmlinux=./vmlinux-arm --definition do_sys_open
./vmlinux-arm with build id 7a1f76dd56e9c4da707cd3d6333f50748141434b not found, continuing without symbols
Failed to find symbol do_sys_open in kernel
Error: Failed to add events.
----
with this fix, we can get the definition
----
$ perf probe --vmlinux=./vmlinux-arm --definition do_sys_open
p:probe/do_sys_open do_sys_open+0
----
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/147214228193.23638.12581984840822162131.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add --definition/-D option for showing the trace-event definition in
stdout. This can be useful in debugging or combined with a shell script.
e.g.
----
# perf probe --definition 'do_sys_open $params'
p:probe/do_sys_open _text+2261728 dfd=%di:s32 filename=%si:u64 flags=%dx:s32 mode=%cx:u16
----
Suggested-and-Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/147214226712.23638.2240534040014013658.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It is simpler to just do the loop, no need for globals and the last user
of such facility disappears.
Testing:
# perf probe -F [a-z]*recvmsg
aead_recvmsg
compat_SyS_recvmsg
compat_sys_recvmsg
hash_recvmsg
inet_recvmsg
kernel_recvmsg
netlink_recvmsg
packet_recvmsg
ping_recvmsg
raw_recvmsg
rawv6_recvmsg
rng_recvmsg
security_socket_recvmsg
selinux_socket_recvmsg
skcipher_recvmsg
sock_common_recvmsg
sock_no_recvmsg
sock_recvmsg
sys_recvmsg
tcp_recvmsg
udp_recvmsg
udpv6_recvmsg
unix_dgram_recvmsg
unix_seqpacket_recvmsg
unix_stream_recvmsg
#
Without filters:
# perf probe -F | tail -5
zswap_pool_create
zswap_pool_current
zswap_update_total_size
zswap_writeback_entry
zswap_zpool_param_set
#
# perf probe -F | wc -l
33311
#
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/20160831130427.GA13095@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Powerpc has Global Entry Point and Local Entry Point for functions. LEP
catches call from both the GEP and the LEP. Symbol table of ELF contains
GEP and Offset from which we can calculate LEP, but debuginfo does not
have LEP info.
Currently, perf prioritize symbol table over dwarf to probe on LEP for
ppc64le. But when user tries to probe with function parameter, we fall
back to using dwarf(i.e. GEP) and when function called via LEP, probe
will never hit.
For example:
$ objdump -d vmlinux
...
do_sys_open():
c0000000002eb4a0: e8 00 4c 3c addis r2,r12,232
c0000000002eb4a4: 60 00 42 38 addi r2,r2,96
c0000000002eb4a8: a6 02 08 7c mflr r0
c0000000002eb4ac: d0 ff 41 fb std r26,-48(r1)
$ sudo ./perf probe do_sys_open
$ sudo cat /sys/kernel/debug/tracing/kprobe_events
p:probe/do_sys_open _text+3060904
$ sudo ./perf probe 'do_sys_open filename:string'
$ sudo cat /sys/kernel/debug/tracing/kprobe_events
p:probe/do_sys_open _text+3060896 filename_string=+0(%gpr4):string
For second case, perf probed on GEP. So when function will be called via
LEP, probe won't hit.
$ sudo ./perf record -a -e probe:do_sys_open ls
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.195 MB perf.data ]
To resolve this issue, let's not prioritize symbol table, let perf
decide what it wants to use. Perf is already converting GEP to LEP when
it uses symbol table. When perf uses debuginfo, let it find LEP offset
form symbol table. This way we fall back to probe on LEP for all cases.
After patch:
$ sudo ./perf probe 'do_sys_open filename:string'
$ sudo cat /sys/kernel/debug/tracing/kprobe_events
p:probe/do_sys_open _text+3060904 filename_string=+0(%gpr4):string
$ sudo ./perf record -a -e probe:do_sys_open ls
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.197 MB perf.data (11 samples) ]
Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Balbir Singh <bsingharora@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1470723805-5081-2-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Instead of inline code, introduce function to post process kernel
probe trace events.
Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Balbir Singh <bsingharora@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1470723805-5081-1-git-send-email-ravi.bangoria@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
If module is "module" then dso->short_name is "[module]". Substring
comparing is't enough: "raid10" matches to "[raid1]". This patch also
checks terminating zero in module name.
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Acked-by: Masami Hiramatsu <mhiramat@kernel.org>
Link: http://lkml.kernel.org/r/147039975648.715620.12985971832789032159.stgit@buzz
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Adjust map->reloc offset for the unmapped address when finding
alternative symbol address from map, because KASLR can relocate the
kernel symbol address.
The same adjustment has been done when finding appropriate kernel symbol
address from map which was introduced by commit f90acac757 ("perf
probe: Find given address from offline dwarf")
Reported-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu@linaro.org>
Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/20160806192948.e366f3fbc4b194de600f8326@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Warn unmatched function filter correctly instead of warning
"symbol-loading error", since that can be a filter issue.
From the technical point of view, this adds a filter chech in map__load
and if there is a filter, it returns -2 (filter-out), instead of -1
(error), and perf-probe checks it and change message.
E.g. without this fix:
# perf probe -F rt_sp*
no symbols found in [kernel.kallsyms], maybe install a debug package?
Failed to load symbols in kernel
With this fix:
# perf probe -F rt_sp*
no symbols passed the given filter.
Failed to find symbols matched to "rt_sp*"
Error: Failed to show functions.
Reported-and-Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146885835596.16106.2293540792775552481.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Support a special SDT probe format which can omit the '%' prefix only if
the SDT group name starts with "sdt_". So, for example both of
"%sdt_libc:setjump" and "sdt_libc:setjump" are acceptable for perf probe
--add.
E.g. without this:
# perf probe -a sdt_libc:setjmp
Semantic error :There is non-digit char in line number.
...
With this:
# perf probe -a sdt_libc:setjmp
Added new event:
sdt_libc:setjmp (on %setjmp in /usr/lib64/libc-2.20.so)
You can now use it in all perf tools, such as:
perf record -e sdt_libc:setjmp -aR sleep 1
Suggested-by: Brendan Gregg <brendan.d.gregg@gmail.com>
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831794674.17065.13359473252168740430.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Support @BUILDID or @FILE suffix for SDT events. This allows perf to add
probes on SDTs/pre-cached events on given FILE or the file which has
given BUILDID (also, this complements BUILDID.)
For example, both gcc and libstdc++ has same SDTs as below. If you
would like to add a probe on sdt_libstdcxx:catch on gcc, you can do as
below.
----
# perf list sdt | tail -n 6
sdt_libstdcxx:catch@/usr/bin/gcc(0cc207fc4b27) [SDT event]
sdt_libstdcxx:catch@/usr/lib64/libstdc++.so.6.0.20(91c7a88fdf49)
sdt_libstdcxx:rethrow@/usr/bin/gcc(0cc207fc4b27) [SDT event]
sdt_libstdcxx:rethrow@/usr/lib64/libstdc++.so.6.0.20(91c7a88fdf49)
sdt_libstdcxx:throw@/usr/bin/gcc(0cc207fc4b27) [SDT event]
sdt_libstdcxx:throw@/usr/lib64/libstdc++.so.6.0.20(91c7a88fdf49)
# perf probe -a %sdt_libstdcxx:catch@0cc
Added new event:
sdt_libstdcxx:catch (on %catch in /usr/bin/gcc)
You can now use it in all perf tools, such as:
perf record -e sdt_libstdcxx:catch -aR sleep 1
----
Committer note:
Doing the full sequence of steps to get the results above:
With a clean build-id cache:
[root@jouet ~]# rm -rf ~/.debug/
[root@jouet ~]# perf list sdt
List of pre-defined events (to be used in -e):
[root@jouet ~]#
No events whatsoever, then, we can add all events in gcc to the build-id
cache, doing a --add + --dry-run:
[root@jouet ~]# perf probe --dry-run --cache -x /usr/bin/gcc --add %sdt_libstdcxx:\*
Added new events:
sdt_libstdcxx:throw (on %* in /usr/bin/gcc)
sdt_libstdcxx:rethrow (on %* in /usr/bin/gcc)
sdt_libstdcxx:catch (on %* in /usr/bin/gcc)
You can now use it in all perf tools, such as:
perf record -e sdt_libstdcxx:catch -aR sleep 1
[root@jouet ~]#
It really didn't add any events, it just cached them:
[root@jouet ~]# perf probe -l
[root@jouet ~]#
We can see that it was cached as:
[root@jouet ~]# ls -la ~/.debug/usr/bin/gcc/9a0730e2bcc6d2a2003d21ac46807e8ee6bcb7c2/
total 976
drwxr-xr-x. 2 root root 4096 Jul 13 21:47 .
drwxr-xr-x. 3 root root 4096 Jul 13 21:47 ..
-rwxr-xr-x. 4 root root 985912 Jun 22 18:52 elf
-rw-r--r--. 1 root root 303 Jul 13 21:47 probes
[root@jouet ~]# file ~/.debug/usr/bin/gcc/9a0730e2bcc6d2a2003d21ac46807e8ee6bcb7c2/elf
/root/.debug/usr/bin/gcc/9a0730e2bcc6d2a2003d21ac46807e8ee6bcb7c2/elf: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=9a0730e2bcc6d2a2003d21ac46807e8ee6bcb7c2, stripped
[root@jouet ~]# cat ~/.debug/usr/bin/gcc/9a0730e2bcc6d2a2003d21ac46807e8ee6bcb7c2/probes
%sdt_libstdcxx:throw=throw
p:sdt_libstdcxx/throw /usr/bin/gcc:0x71ffd
%sdt_libstdcxx:rethrow=rethrow
p:sdt_libstdcxx/rethrow /usr/bin/gcc:0x720b8
%sdt_libstdcxx:catch=catch
p:sdt_libstdcxx/catch /usr/bin/gcc:0x7307f
%sdt_libgcc:unwind=unwind
p:sdt_libgcc/unwind /usr/bin/gcc:0x7eec0
#sdt_libstdcxx:*=%*
[root@jouet ~]#
Ok, now we can use 'perf probe' to refer to those cached entries as:
Humm, nope, doing as above we end up with:
[root@jouet ~]# perf probe -a %sdt_libstdcxx:catch
Semantic error :* is bad for event name -it must follow C symbol-naming rule.
Error: Failed to add events.
[root@jouet ~]#
But it worked at some point, lets try not using --dry-run:
Resetting everything:
# rm -rf ~/.debug/
# perf probe -d *:*
# perf probe -l
# perf list sdt
List of pre-defined events (to be used in -e):
#
Ok, now it cached everything, even things we haven't asked it to
(sdt_libgcc:unwind):
[root@jouet ~]# perf probe -x /usr/bin/gcc --add %sdt_libstdcxx:\*
Added new events:
sdt_libstdcxx:throw (on %* in /usr/bin/gcc)
sdt_libstdcxx:rethrow (on %* in /usr/bin/gcc)
sdt_libstdcxx:catch (on %* in /usr/bin/gcc)
You can now use it in all perf tools, such as:
perf record -e sdt_libstdcxx:catch -aR sleep 1
[root@jouet ~]# perf list sdt
List of pre-defined events (to be used in -e):
sdt_libgcc:unwind [SDT event]
sdt_libstdcxx:catch [SDT event]
sdt_libstdcxx:rethrow [SDT event]
sdt_libstdcxx:throw [SDT event]
[root@jouet ~]#
And we have the events in place:
[root@jouet ~]# perf probe -l
sdt_libstdcxx:catch (on execute_cfa_program+1551@../../../libgcc/unwind-dw2.c in /usr/bin/gcc)
sdt_libstdcxx:rethrow (on d_print_subexpr+280@libsupc++/cp-demangle.c in /usr/bin/gcc)
sdt_libstdcxx:throw (on d_print_subexpr+93@libsupc++/cp-demangle.c in /usr/bin/gcc)
[root@jouet ~]#
And trying to use them at least has 'perf trace --event sdt*:*' working.
Then, if we try to add the ones in libstdc++:
[root@jouet ~]# perf probe -x /usr/lib64/libstdc++.so.6 -a %sdt_libstdcxx:\*
Error: event "catch" already exists.
Hint: Remove existing event by 'perf probe -d'
or force duplicates by 'perf probe -f'
or set 'force=yes' in BPF source.
Error: Failed to add events.
[root@jouet ~]#
Doesn't work, dups, but at least this served to, unbeknownst to the user, add
the SDT probes in /usr/lib64/libstdc++.so.6!
[root@jouet ~]# perf list sdt
List of pre-defined events (to be used in -e):
sdt_libgcc:unwind [SDT event]
sdt_libstdcxx:catch@/usr/bin/gcc(9a0730e2bcc6) [SDT event]
sdt_libstdcxx:catch@/usr/lib64/libstdc++.so.6.0.22(ef2b7066559a) [SDT event]
sdt_libstdcxx:rethrow@/usr/bin/gcc(9a0730e2bcc6) [SDT event]
sdt_libstdcxx:rethrow@/usr/lib64/libstdc++.so.6.0.22(ef2b7066559a) [SDT event]
sdt_libstdcxx:throw@/usr/bin/gcc(9a0730e2bcc6) [SDT event]
sdt_libstdcxx:throw@/usr/lib64/libstdc++.so.6.0.22(ef2b7066559a) [SDT event]
[root@jouet ~]#
Now we should be able to get to the original cset comment, if we remove all
SDTs events in place, not from the cache, from the kernel, where it was set up as:
[root@jouet ~]# ls -la /sys/kernel/debug/tracing/events/sdt_libstdcxx/
total 0
drwxr-xr-x. 5 root root 0 Jul 13 22:00 .
drwxr-xr-x. 80 root root 0 Jul 13 21:56 ..
drwxr-xr-x. 2 root root 0 Jul 13 22:00 catch
-rw-r--r--. 1 root root 0 Jul 13 22:00 enable
-rw-r--r--. 1 root root 0 Jul 13 22:00 filter
drwxr-xr-x. 2 root root 0 Jul 13 22:00 rethrow
drwxr-xr-x. 2 root root 0 Jul 13 22:00 throw
[root@jouet ~]#
[root@jouet ~]# head -2 /sys/kernel/debug/tracing/events/sdt_libstdcxx/throw/format
name: throw
ID: 2059
[root@jouet ~]#
Now to remove it:
[root@jouet ~]# perf probe -d sdt_libstdc*:*
Removed event: sdt_libstdcxx:catch
Removed event: sdt_libstdcxx:rethrow
Removed event: sdt_libstdcxx:throw
[root@jouet ~]#
Which caused:
[root@jouet ~]# ls -la /sys/kernel/debug/tracing/events/sdt_libstdcxx/
ls: cannot access '/sys/kernel/debug/tracing/events/sdt_libstdcxx/': No such file or directory
[root@jouet ~]#
Ok, now we can do:
[root@jouet ~]# perf list sdt_libstdcxx:catch
List of pre-defined events (to be used in -e):
sdt_libstdcxx:catch@/usr/bin/gcc(9a0730e2bcc6) [SDT event]
sdt_libstdcxx:catch@/usr/lib64/libstdc++.so.6.0.22(ef2b7066559a) [SDT event]
[root@jouet ~]#
So, these are not really 'pre-defined events', i.e. we can't use them with
'perf record --event':
[root@jouet ~]# perf record --event sdt_libstdcxx:catch*
event syntax error: 'sdt_libstdcxx:catch*'
\___ unknown tracepoint
Error: File /sys/kernel/debug/tracing/events/sdt_libstdcxx/catch* not found.
Hint: Perhaps this kernel misses some CONFIG_ setting to enable this feature?.
<SNIP>
[root@jouet ~]#
To have it really pre-defined we must use perf probe to get its definition from
the cache and set it up in the kernel, creating the tracepoint to _then_ use it
with 'perf record --event':
[root@jouet ~]# perf probe -a sdt_libstdcxx:catch
Semantic error :There is non-digit char in line number.
<SNIP>
Oops, there is another gotcha here, we need that pesky '%' character:
[root@jouet ~]# perf probe -a %sdt_libstdcxx:catch
Added new events:
sdt_libstdcxx:catch (on %catch in /usr/bin/gcc)
sdt_libstdcxx:catch_1 (on %catch in /usr/lib64/libstdc++.so.6.0.22)
You can now use it in all perf tools, such as:
perf record -e sdt_libstdcxx:catch_1 -aR sleep 1
[root@jouet ~]#
But then we added _two_ events, one with the name we expected, the other one
with a _ added, when doing the analysis we need to pay attention to who maps to
who.
And here is where we get to the point of this patch, which is to be able to
disambiguate those definitions for 'catch' in the build-id cache, but first we need
remove those events we just added:
[root@jouet ~]# perf probe -d %sdt_libstdcxx:catch
Oops, that didn't remove anything, we need to _remove_ that % char in this case:
[root@jouet ~]# perf probe -d sdt_libstdcxx:catch
Removed event: sdt_libstdcxx:catch
And we need to remove the other event added, i.e. I forgot to add a * at the end:
[root@jouet ~]# perf probe -d sdt_libstdcxx:catch*
Removed event: sdt_libstdcxx:catch_1
[root@jouet ~]#
Ok, disambiguating it using what is in this patch:
[root@jouet ~]# perf list sdt_libstdcxx:catch
List of pre-defined events (to be used in -e):
sdt_libstdcxx:catch@/usr/bin/gcc(9a0730e2bcc6) [SDT event]
sdt_libstdcxx:catch@/usr/lib64/libstdc++.so.6.0.22(ef2b7066559a) [SDT event]
[root@jouet ~]#
[root@jouet ~]# perf probe -a %sdt_libstdcxx:catch@9a07
Added new event:
sdt_libstdcxx:catch (on %catch in /usr/bin/gcc)
You can now use it in all perf tools, such as:
perf record -e sdt_libstdcxx:catch -aR sleep 1
[root@jouet ~]# perf probe -l
sdt_libstdcxx:catch (on execute_cfa_program+1551@../../../libgcc/unwind-dw2.c in /usr/bin/gcc)
[root@jouet ~]#
Yeah, it works! But we need to try and simplify this :-)
Update: Some aspects of this simplification take place in the following
patches.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831793746.17065.13065062753978236612.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Search SDT/cached event from all probe caches if user doesn't pass any
binary. With this, we don't have to specify target binary for SDT and
named cached events (which start with %).
E.g. without this, a target binary must be passed with -x.
# perf probe -x /usr/lib64/libc-2.20.so -a %sdt_libc:\*
With this change, we don't need it anymore.
# perf probe -a %sdt_libc:\*
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831792812.17065.2353705982669445313.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Allo glob wildcard for reusing cached/SDT events. E.g.
# perf probe -x /usr/lib64/libc-2.20.so -a %sdt_libc:\*
This example adds probes for all SDT in libc.
Note that the SDTs must have been scanned by perf buildid-cache.
Committer note:
Using it to check what of those SDT probes would take place when doing
a cargo run (rust):
# trace --no-sys --event sdt_libc:* cargo run
0.000 sdt_libc:setjmp:(7f326b69c4d1))
28.423 sdt_libc:setjmp:(7f4b0a5364d1))
29.000 sdt_libc:setjmp:(7f4b0a5364d1))
88.597 sdt_libc:setjmp:(7fc01fd414d1))
89.220 sdt_libc:setjmp:(7fc01fd414d1))
95.501 sdt_libc:setjmp:(7f326b69c4d1))
Running `target/debug/hello_world`
97.110 sdt_libc:setjmp:(7f95e09234d1))
Hello, world!
#
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831791813.17065.17846564230840594888.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
To improve usability, support %[PROVIDER:]SDTEVENT format to add new
probes on SDT and cached events.
e.g.
----
# perf probe -x /lib/libc-2.17.so %lll_lock_wait_private
Added new event:
sdt_libc:lll_lock_wait_private (on %lll_lock_wait_private in /usr/lib/libc-2.17.so)
You can now use it in all perf tools, such as:
perf record -e sdt_libc:lll_lock_wait_private -aR sleep 1
# perf probe -l | more
sdt_libc:lll_lock_wait_private (on __lll_lock_wait_private+21 in /usr/lib/libc-2.17.so)
----
Note that this is not only for SDT events, but also normal
events with event-name.
e.g. define "myevent" on cache (-n doesn't add the real probe)
----
# perf probe -x ./perf --cache -n --add 'myevent=dso__load $params'
----
Reuse the "myevent" from cache as below.
----
# perf probe -x ./perf %myevent
----
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831788372.17065.3645054540325909346.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fix to show correct error messages for $vars and $params because
those special variables requires debug information to find the
real variables or function parameters.
E.g. without this fix;
----
# perf probe -x /lib64/libc-2.23.so getaddrinfo \$params
Failed to write event: Invalid argument
Please upgrade your kernel to at least 3.14 to have access to feature $params
Error: Failed to add events.
----
Perf ends up with an error, but the message is not correct. With this
fix, perf shows correct error message as below.
----
# perf probe -x /lib64/libc-2.23.so getaddrinfo \$params
The /usr/lib64/libc-2.23.so file has no debug information.
Rebuild with -g, or install an appropriate debuginfo package.
Error: Failed to add events.
----
Reported-and-Tested-by: Arnaldo Carvalho de Melo <acme@kernel.org>
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146831787438.17065.6152436996780110699.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The tools so far have been using the strerror_r() GNU variant, that
returns a string, be it the buffer passed or something else.
But that, besides being tricky in cases where we expect that the
function using strerror_r() returns the error formatted in a provided
buffer (we have to check if it returned something else and copy that
instead), breaks the build on systems not using glibc, like Alpine
Linux, where musl libc is used.
So, introduce yet another wrapper, str_error_r(), that has the GNU
interface, but uses the portable XSI variant of strerror_r(), so that
users rest asured that the provided buffer is used and it is what is
returned.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-d4t42fnf48ytlk8rjxs822tf@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Allow user to set group name for adding new event. Note that user must
ensure that the group name doesn't conflict with existing group name
carefully.
E.g. Existing group name can conflict with other events. Especially,
using the group name reserved for kernel modules can hide kernel
embedded events when loading modules.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146736024091.27797.9471545190066268995.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Before analyzing debuginfo, try to find a corresponding entry from probe
cache always. This does not depend on --cache, the --cache enables to
store/update cache, but looking up the cache is always enabled.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/146736019226.27797.16366402884098398857.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
To match the semantics for list.h in the kernel, that are the
interface we use in them.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Milian Wolff <milian.wolff@kdab.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Taeung Song <treeze.taeung@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-mdp1heu9xjjc12zebh91232l@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
To match the semantics for list.h in the kernel, that are the
interface we use in them.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Milian Wolff <milian.wolff@kdab.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Taeung Song <treeze.taeung@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-0b5i2ki9c3di6706fxpticsb@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
And do nothing, just like free(), to avoid having to test it in callers,
usually in error paths.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-q42gj3b3znhho9z1mrbo4jce@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add --cache option to cache the probe definitions. This just saves the
result of the dwarf analysis to probe cache.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20160615032840.31330.44412.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Uncomment and export synthesize_perf_probe_point() which had once
introduced but has been disabled for a long time. This renews the code
and re-enable it.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20160608092949.3116.21958.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add perf_probe_event__copy() to copy perf_probe_event data structure and
sub data structures under given source perf_probe_event.
Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Ananth N Mavinakayanahalli <ananth@linux.vnet.ibm.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Hemant Kumar <hemant@linux.vnet.ibm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20160608092940.3116.18034.stgit@devbox
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
