linux/tools/perf/Documentation
Arnaldo Carvalho de Melo d4097f1937 perf trace: Introduce --filter for tracepoint events
Similar to what is in 'perf record', works just like there:

  # perf trace -e msr:*
   328.297 :0/0 msr:write_msr(msr: FS_BASE, val: 140240388381888)
   328.302 :0/0 msr:write_msr(msr: FS_BASE, val: 140240388381888)
   328.306 :0/0 msr:write_msr(msr: FS_BASE, val: 140240388381888)
   328.317 :0/0 msr:write_msr(msr: FS_BASE, val: 140240388381888)
   328.322 :0/0 msr:write_msr(msr: FS_BASE, val: 140240388381888)
   328.327 :0/0 msr:write_msr(msr: FS_BASE, val: 140240388381888)
   328.331 :0/0 msr:write_msr(msr: FS_BASE, val: 140240388381888)
   328.336 :0/0 msr:write_msr(msr: FS_BASE, val: 140240388381888)
   328.340 :0/0 ^Cmsr:write_msr(msr: FS_BASE, val: 140240388381888)
  #

So, for a system wide trace session looking at the write_msr tracepoint
we see a flood of MSR_FS_BASE, we need to get the number for that:

  # grep FS_BASE /tmp/build/perf/trace/beauty/generated/x86_arch_MSRs_array.c
	[0xc0000100 - x86_64_specific_MSRs_offset] = "FS_BASE",
  #

And then use it in a filter:

  # perf trace -e msr:* --filter="msr!=0xc0000100"
  <SNIP>
   942.177 :0/0 msr:write_msr(msr: IA32_TSC_DEADLINE, val: 3056931068232)
   942.199 :0/0 msr:write_msr(msr: IA32_TSC_DEADLINE, val: 3057135655252)
   942.203 :0/0 msr:write_msr(msr: IA32_TSC_DEADLINE, val: 3056931068222)
   942.231 :0/0 msr:write_msr(msr: IA32_TSC_DEADLINE, val: 3056998373022)
   942.241 :0/0 msr:write_msr(msr: IA32_TSC_DEADLINE, val: 3056931068236)
  <SNIP>
  #

Ok, lets filter that too, too noisy:

  # grep TSC_DEADLINE /tmp/build/perf/trace/beauty/generated/x86_arch_MSRs_array.c
	[0x000006E0] = "IA32_TSC_DEADLINE",
  #

  # perf trace -e msr:* --filter="msr!=0xc0000100 && msr!=0x6e0" -a sleep 0.1
     0.000 :0/0 msr:read_msr(msr: IA32_TSC_ADJUST)
     0.066 CPU 0/KVM/4895 msr:write_msr(msr: IA32_SPEC_CTRL, val: 6)
     0.070 CPU 0/KVM/4895 msr:write_msr(msr: 0x830, val: 34359740667)
     0.099 CPU 0/KVM/4895 msr:read_msr(msr: IA32_SYSENTER_ESP, val: -2199021993472)
     0.100 CPU 0/KVM/4895 msr:read_msr(msr: IA32_APICBASE, val: 4276096000)
     0.101 CPU 0/KVM/4895 msr:read_msr(msr: IA32_DEBUGCTLMSR)
     0.109 :0/0 msr:write_msr(msr: IA32_SPEC_CTRL)
     1.000 :0/0 msr:write_msr(msr: 0x830, val: 17179871485)
    18.893 :0/0 msr:write_msr(msr: 0x83f, val: 246)
    28.810 :0/0 msr:write_msr(msr: 0x830, val: 68719479037)
    40.117 CPU 0/KVM/4895 msr:write_msr(msr: IA32_SPEC_CTRL, val: 6)
    40.127 CPU 0/KVM/4895 msr:read_msr(msr: IA32_DEBUGCTLMSR)
    40.139 CPU 0/KVM/4895 msr:write_msr(msr: LSTAR, val: -2130661312)
    40.141 CPU 0/KVM/4895 msr:write_msr(msr: SYSCALL_MASK, val: 14080)
    40.142 CPU 0/KVM/4895 msr:write_msr(msr: TSC_AUX)
    40.144 CPU 0/KVM/4895 msr:write_msr(msr: KERNEL_GS_BASE)
    40.147 CPU 0/KVM/4895 msr:write_msr(msr: IA32_SPEC_CTRL)
    40.148 CPU 0/KVM/4895 msr:write_msr(msr: IA32_FLUSH_CMD, val: 1)
    40.151 CPU 0/KVM/4895 msr:write_msr(msr: IA32_SPEC_CTRL, val: 6)
  ^C
  #

One can combine that with filtering pids as well:

  # perf trace -e msr:* --filter="msr!=0xc0000100 && msr!=0x6e0" --filter-pids 4895 -a sleep 0.09
     0.000 :0/0 msr:write_msr(msr: 0x830, val: 4294969597)
     0.291 gnome-terminal/2790 msr:write_msr(msr: SYSCALL_MASK, val: 292608)
     0.294 gnome-terminal/2790 msr:write_msr(msr: LSTAR, val: -1935671280)
     0.295 gnome-terminal/2790 msr:write_msr(msr: TSC_AUX, val: 6)
    10.940 gnome-terminal/2790 msr:write_msr(msr: 0x830, val: 4294969597)
    15.943 gnome-shell/2096 msr:write_msr(msr: 0x830, val: 4294969597)
    16.975 :0/0 msr:write_msr(msr: 0x830, val: 4294969597)
    19.560 :0/0 msr:write_msr(msr: 0x83f, val: 246)
    25.162 :0/0 msr:read_msr(msr: IA32_TSC_ADJUST)
    25.807 JS Watchdog/3635 msr:write_msr(msr: IA32_SPEC_CTRL, val: 6)
    25.820 :0/0 msr:write_msr(msr: IA32_SPEC_CTRL)
    25.941 gnome-terminal/2790 msr:write_msr(msr: 0x830, val: 4294969597)
    26.941 gnome-terminal/2790 msr:write_msr(msr: 0x830, val: 4294969597)
    29.942 gnome-terminal/2790 msr:write_msr(msr: 0x830, val: 4294969597)
    45.313 :0/0 msr:write_msr(msr: 0x83f, val: 246)
    56.945 gnome-terminal/2790 msr:write_msr(msr: 0x830, val: 4294969597)
    60.946 gnome-terminal/2790 msr:write_msr(msr: 0x830, val: 4294969597)
    74.096 JS Watchdog/8971 msr:write_msr(msr: IA32_SPEC_CTRL, val: 6)
    74.130 :0/0 msr:write_msr(msr: IA32_SPEC_CTRL)
    79.673 :0/0 msr:write_msr(msr: 0x83f, val: 246)
    79.947 gnome-terminal/2790 msr:write_msr(msr: 0x830, val: 17179871485)
  #

Or for just a pid, with callchains:

  # grep SYSCALL_MAS /tmp/build/perf/trace/beauty/generated/x86_arch_MSRs_array.c
	[0xc0000084 - x86_64_specific_MSRs_offset] = "SYSCALL_MASK",
  # perf trace -e msr:* --filter="msr==0xc0000084" --pid 2790 --call-graph=dwarf

     0.000 gnome-terminal/2790 msr:write_msr(msr: SYSCALL_MASK, val: 292608)
                                       do_trace_write_msr ([kernel.kallsyms])
                                       do_trace_write_msr ([kernel.kallsyms])
                                       kvm_on_user_return ([kvm])
                                       fire_user_return_notifiers ([kernel.kallsyms])
                                       exit_to_usermode_loop ([kernel.kallsyms])
                                       do_syscall_64 ([kernel.kallsyms])
                                       entry_SYSCALL_64 ([kernel.kallsyms])
                                       __GI___poll (inlined)
  9299.073 gnome-terminal/2790 msr:write_msr(msr: SYSCALL_MASK, val: 292608)
                                       do_trace_write_msr ([kernel.kallsyms])
                                       do_trace_write_msr ([kernel.kallsyms])
                                       kvm_on_user_return ([kvm])
                                       fire_user_return_notifiers ([kernel.kallsyms])
                                       exit_to_usermode_loop ([kernel.kallsyms])
                                       do_syscall_64 ([kernel.kallsyms])
                                       entry_SYSCALL_64 ([kernel.kallsyms])
                                       __GI___poll (inlined)
  9348.374 gnome-terminal/2790 msr:write_msr(msr: SYSCALL_MASK, val: 292608)
                                       do_trace_write_msr ([kernel.kallsyms])
                                       do_trace_write_msr ([kernel.kallsyms])
                                       kvm_on_user_return ([kvm])
                                       fire_user_return_notifiers ([kernel.kallsyms])
                                       exit_to_usermode_loop ([kernel.kallsyms])
                                       do_syscall_64 ([kernel.kallsyms])
                                       entry_SYSCALL_64 ([kernel.kallsyms])
                                       __GI___poll (inlined)
  <SNIP>
  #

Ok, just another form of KVM to emit MSRs :-)

Next step: elliminate those greps by getting the filter expression,
looking for arg names, then for the arrays associated with it to do a
reverse lookup.

Also allow those filters to be associated with strace-like syscall
names.

After that: augment the 'val' arg for 'msr:write_msr' based on the first
arg, 'msr'.

Then, do that with eBPF too, not just with tracepoint filters.

Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Brendan Gregg <brendan.d.gregg@gmail.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Luis Cláudio Gonçalves <lclaudio@redhat.com>
Cc: Marcelo Tosatti <mtosatti@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: https://lkml.kernel.org/n/tip-95bfe5d4tzy5f66bx49d05rj@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-10-09 11:23:52 -03:00
..
android.txt perf tools: Update android build documentation 2016-07-04 20:27:27 -03:00
asciidoc.conf perf docs: Allow man page date to be specified 2019-09-27 09:26:14 -03:00
asciidoctor-extensions.rb perf Documentation: Support for asciidoctor 2018-04-26 13:47:10 -03:00
build-xed.txt perf script: Add --insn-trace for instruction decoding 2018-10-24 15:29:50 -03:00
Build.txt perf tools: Add doc about how to build perf with Asan and UBSan 2019-03-19 16:52:04 -03:00
callchain-overhead-calculation.txt perf tools: Document --children option in more detail 2015-04-29 10:38:06 -03:00
db-export.txt perf db-export: Add brief documentation 2019-06-05 09:47:57 -03:00
examples.txt perf record: Remove -f/--force option 2013-07-08 17:37:25 -03:00
intel-bts.txt perf tools: Add Intel BTS support 2015-08-21 11:34:10 -03:00
intel-pt.txt perf intel-pt: Add brief documentation for PEBS via Intel PT 2019-08-14 10:59:59 -03:00
itrace.txt perf tools: Add itrace option 'o' to synthesize aux-output events 2019-08-14 10:59:59 -03:00
jit-interface.txt perf symbols: Add description of JIT interface 2012-08-13 14:55:02 -03:00
jitdump-specification.txt perf docs: Correct and clarify jitdump spec 2019-09-30 17:29:51 -03:00
Makefile perf tools: Fix a typo in a variable name in the Documentation Makefile 2019-08-08 15:41:10 -03:00
manpage-1.72.xsl
manpage-base.xsl
manpage-bold-literal.xsl
manpage-normal.xsl
manpage-suppress-sp.xsl
perf-annotate.txt perf annotate: Add --percent-type option 2018-08-08 15:55:53 -03:00
perf-archive.txt perf archive: Remove duplicated 'runs' in man page 2013-12-09 15:21:45 -03:00
perf-bench.txt perf bench: Add epoll_ctl(2) benchmark 2018-11-21 22:39:55 -03:00
perf-buildid-cache.txt perf buildid-cache: Support --purge-all option 2018-04-26 09:30:26 -03:00
perf-buildid-list.txt perf report: Accept fifos as input file 2011-12-23 17:01:03 -02:00
perf-c2c.txt perf mem/c2c: Fix perf_mem_events to support powerpc 2019-02-04 11:32:14 -03:00
perf-config.txt perf trace: Allow choosing how to augment the tracepoint arguments 2019-10-07 12:22:18 -03:00
perf-data.txt perf tools: Correct title markers for asciidoctor 2018-03-07 10:26:32 -03:00
perf-diff.txt perf diff: Documentation -c cycles option 2019-07-02 13:20:51 -03:00
perf-evlist.txt perf evlist: Document missing --force option 2017-11-16 14:50:07 -03:00
perf-ftrace.txt perf tools: Correct title markers for asciidoctor 2018-03-07 10:26:32 -03:00
perf-help.txt
perf-inject.txt perf inject: Document missing options 2017-11-16 14:50:05 -03:00
perf-kallsyms.txt perf tools: Correct title markers for asciidoctor 2018-03-07 10:26:32 -03:00
perf-kmem.txt perf kmem: Document a missing option & an argument 2018-02-16 14:55:42 -03:00
perf-kvm.txt perf tools: Configurable per thread proc map processing time out 2015-06-19 18:27:13 -03:00
perf-list.txt perf tools: Add a 'percore' event qualifier 2019-05-16 14:17:24 -03:00
perf-lock.txt perf lock: Document missing options 2017-11-16 14:50:04 -03:00
perf-mem.txt perf mem/c2c: Fix perf_mem_events to support powerpc 2019-02-04 11:32:14 -03:00
perf-probe.txt perf-probe: Add user memory access attribute support 2019-05-25 23:04:42 -04:00
perf-record.txt perf tools: Add aux-output config term 2019-08-14 10:59:59 -03:00
perf-report.txt perf report: Add --switch-on/--switch-off events 2019-08-16 12:14:33 -03:00
perf-sched.txt perf sched: Fix documentation for timehist 2018-04-12 10:33:36 -03:00
perf-script-perl.txt perf tools: Correct title markers for asciidoctor 2018-03-07 10:26:32 -03:00
perf-script-python.txt perf script python: Add dict fields introduction to Documentation 2018-06-06 15:40:10 -03:00
perf-script.txt perf script: Allow specifying event to switch off processing of other events 2019-08-15 12:24:16 -03:00
perf-stat.txt perf stat: Support per-die aggregation 2019-06-10 16:19:59 -03:00
perf-test.txt perf test: Add -F/--dont-fork option 2016-06-30 18:27:45 -03:00
perf-timechart.txt perf timechart: Document missing --force option 2017-11-16 14:50:06 -03:00
perf-top.txt perf top: Add --switch-on/--switch-off events 2019-08-15 16:03:26 -03:00
perf-trace.txt perf trace: Introduce --filter for tracepoint events 2019-10-09 11:23:52 -03:00
perf-version.txt perf version: Add man page 2018-04-02 13:52:23 -03:00
perf.data-file-format.txt perf.data documentation: Clarify HEADER_SAMPLE_TOPOLOGY format 2019-08-14 10:59:59 -03:00
perf.txt perf docs: Add description for stderr 2019-05-16 14:17:24 -03:00
perfconfig.example perf config: Show default report configuration in example and docs 2016-09-01 09:44:13 -03:00
tips.txt perf tools: Fix typos / broken sentences 2019-07-02 16:08:16 -03:00