Fix machine.vmlinux_maps to make sure to clear the old one if it is
renewal. This can leak the previous maps on the vmlinux_maps because
those are just overwritten.
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20151209021133.10245.93730.stgit@localhost.localdomain
[ Simplified the memset, same end result ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since hists__init doesn't set the destructor of hists_evsel (which is an
extended evsel structure), when hists_evsel is released, the extended
part of the hists_evsel is not deleted (note that the hists_evsel object
itself is freed).
This fixes it to add a destructor for hists__evsel and to set it up.
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20151209021129.10245.28710.stgit@localhost.localdomain
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fix cmd_stat() to release cpu_map objects (aggr_map and
cpus_aggr_map) afterwards.
refcnt debugger shows that the cmd_stat initializes cpu_map
but not puts it.
----
# ./perf stat -v ls
....
REFCNT: BUG: Unreclaimed objects found.
==== [0] ====
Unreclaimed cpu_map@0x29339c0
Refcount +1 => 1 at
./perf(cpu_map__empty_new+0x6d) [0x4e64bd]
./perf(cmd_stat+0x5fe) [0x43594e]
./perf() [0x47b785]
./perf(main+0x617) [0x422587]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f2dff420af5]
./perf() [0x4226fd]
REFCNT: Total 1 objects are not reclaimed.
"cpu_map" leaks 1 objects
----
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20151209021127.10245.93697.stgit@localhost.localdomain
[ Remove NULL checks before calling the put operation, it checks it already ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Boris reported that 'perf top' is unusable on his default 'black on
white' terminal, which uses (eye friendly) light-grey as a background
color.
The reason is that the TUI cursor for the current selection line uses
HE_COLORSET_SELECTED, and that has a default background color of
'lightgrey' - which is a common terminal background choice and thus
the colors conflict.
Use yellow as the background color instead: that should be an uncommon
terminal background, yet it's still ergonomic on both black and
white/grey terminals.
[ It would be a better solution to straight out detect color
collisions and resolve them reasonably by converting them to RGB and
calculating color space distances, but I was unable to find
proper documentation for SLtt_get_color_object() to recover the
current color scheme so I gave up ... Yellow works well enough. ]
Reported-and-Tested-by: Borislav Petkov <bp@alien8.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Binderman <dcb314@hotmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/20150305103213.GA23046@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
User visible:
- Fixes and improvements for supporting annotating ARM binaries, support ARM
call and jump instructions, more work needed to have arch specific stuff
separated into tools/perf/arch/*/annotate/ (Russell King)
- Fix several 'perf test' entries broken by recent perf/core changes (Jiri Olsa)
Infrastructure:
- Consolidate perf_ev{list,sel}__{enable,disable}() calls (Jiri Olsa)
- Pass correct string to dso__adjust_kmod_long_name (Wang Nan)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJWZgMhAAoJENZQFvNTUqpAunUP/2x5II8HDGVTpjsV5jriyDNU
vj1XnYaVrEoBQG2EXGRFnQYbrCogB8b5TjMWk01XibOoNPzNUJbTIfCpi0aPFNGl
YrAUN3paz5lQliUlRV56913dnixammfjsy+c837Kk3HzaKpIdTpo1Unkz94Bjv5i
4PVMk0YMM4fvBQF3Urg2x4BKhk7sZkDTJvuRtrCDr6WE8X2Eon7TXvYo2QEIUQ5e
9totlruQFAvpbu08F1gd6RhqtIC3uDcJNX1eEebUJtf4EhJmTAyJ7QhtaRzcmaOk
IusfXtCvhkLjprjOXLGt4nnhUw3Lnaxno6/Izm9eyJ2JAj8ERIfl9T7Yoc5Ab/Hc
helKeCgaD2jJgPvldOjm4/ckKkxsxbK1oPZ5FvhKLMIz0mjKZcB5CpYxj0Zzos2g
BjLIH9sXp9x1j8coeCRyfqOx7tMhY1vCVd4JVbl5/cIWt58+dAXJPcqxRAXUPlpt
PN+uIaQ2IQa+K++dgQOsNDyUCRIJOvuCEYjhDkqagfccSF85K5v43HFIVS4c5Q1w
znF9884UhiRhR5SuZqZCQUTwFOk5rsOcX8wliIJCWP2bEaMde1vaMhTAGMXe8Qk+
6do62hAgSeUTlH2f7eMExRY8/CXQvJmu5O1ukb2mWcSJkNeL8npoCFER1W1gy9Yh
NDU9dOoKMwOtzyI3STkg
=CSRu
-----END PGP SIGNATURE-----
Merge tag 'perf-core-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core
Pull perf/core improvements and fixes from Arnaldo Carvalho de Melo:
User visible changes:
- Fixes and improvements for supporting annotating ARM binaries, support ARM
call and jump instructions, more work needed to have arch specific stuff
separated into tools/perf/arch/*/annotate/ (Russell King)
- Fix several 'perf test' entries broken by recent perf/core changes (Jiri Olsa)
Infrastructure changes:
- Consolidate perf_ev{list,sel}__{enable,disable}() calls (Jiri Olsa)
- Pass correct string to dso__adjust_kmod_long_name() (Wang Nan)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Add basic support to parse ARM assembly.
This:
* enables perf to correctly show the disassembly, rather than chopping
some constants off at the '#' (which is not a comment character on
ARM).
* allows perf to identify ARM instructions that branch to other parts
within the same function, thereby properly annotating them.
* allows perf to identify function calls, allowing called functions to
be followed in the annotated view.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Will Deacon <will.deacon@arm.com>
Link: http://lkml.kernel.org/n/tip-owp1uj0nmcgfrlppfyeetuyf@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It's more readable this way and we can save one
perf_evsel__is_group_leader condition in current code.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1449133606-14429-7-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Currently we have 2 kinds of stat counters based on when the event is
enabled:
1) tracee command events, which are enable once the
tracee executes exec syscall (enable_on_exec bit)
2) all other events which get alive within the
perf_event_open syscall
And 2) case could raise a problem in case we want additional filter to
be attached for event. In this case we want the event to be enabled
after it's configured with filter.
Changing the behaviour of 2) events, so they all are created as disabled
(disabled bit). Adding extra enable call to make them alive once they
finish setup.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1449133606-14429-6-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
No need to mimic the behaviour of perf_evlist__enable, we can use it
directly.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1449133606-14429-5-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Use perf_evsel__(enable|disable) functions in perf_evlist__(enable|disable)
functions in order to centralize ioctl enable/disable calls. This way we
eliminate 2 places calling directly ioctl.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1449133606-14429-4-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Adding perf_evsel__disable function to have complement for
perf_evsel__enable function. Both will be used in following patch to
factor perf_evlist__(enable|disable).
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1449133606-14429-3-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
All events now share proper cpu and thread maps. There's no need to pass
those maps from evlist, it's safe to use evsel maps for enabling event.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1449133606-14429-2-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It fixes segfault within machine__exit, that's caused
but not creating kernel maps for machine.. We're calling
machine__destroy_kernel_maps in machine__exit since commit:
ebe9729c8c perf machine: Fix to destroy kernel maps when machine exits
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/tip-k4snzv5t4dvdckggzwdzyljo@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The bpf-output is added under software events, but is not parse-able
within parse_events, which is what round trip test is expecting.
Checking software events only until dummy event.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1449131658-1841-6-git-send-email-jolsa@kernel.org
[ Make it a one liner by keeping __perf_evsel__name_array_test() around ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
In error path to try user space event, both cpus and threads map now
owned by evlist and freed by perf_evlist__set_maps call. Getting
reference to keep them alive.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1449131658-1841-5-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This is more straightforward than what we have now.
It also fixes a segfault within machine__exit, that's caused
by not creating kernel maps for machine.. We're calling
machine__destroy_kernel_maps in machine__exit since commit:
ebe9729c8c perf machine: Fix to destroy kernel maps when machine exits
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1449131658-1841-4-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This is more straightforward than what we have now.
It also fixes a segfault within machine__exit, that's caused by not
creating kernel maps for machine.. We're calling
machine__destroy_kernel_maps in machine__exit since commit:
ebe9729c8c perf machine: Fix to destroy kernel maps when machine exits
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1449131658-1841-3-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This is more straightforward than what we have now.
It also fixes a segfault within machine__exit, that's caused by not
creating kernel maps for machine.. We're calling
machine__destroy_kernel_maps in machine__exit since commit:
ebe9729c8c perf machine: Fix to destroy kernel maps when machine exits
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/r/1449131658-1841-2-git-send-email-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
There's a mistake in dso__adjust_kmod_long_name() that it use strdup()
to dup the new long_name of a dso, but passes the original string to
dso__set_long_name(). Which causes random crash during cleanup.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Reviewed-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Fixes: c03d5184f0 ("perf machine: Adjust dso->long_name for offline module")
Link: http://lkml.kernel.org/r/1449455785-42020-1-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
If feed perf a symbol filter in cmdline and the result is empty,
pressing 'Enter' in the hist browser causes crash:
# ./perf report perf.data <-- Common mistake for beginners
Then press 'Enter':
perf: Segmentation fault
-------- backtrace --------
/home/wangnan/perf[0x53e578]
/lib64/libc.so.6(+0x3545f)[0x7f76bafe045f]
/home/wangnan/perf[0x539dd4]
/home/wangnan/perf(perf_evlist__tui_browse_hists+0x96)[0x53d216]
/home/wangnan/perf(cmd_report+0x1b9f)[0x442c7f]
/home/wangnan/perf[0x47efa2]
/home/wangnan/perf(main+0x5f5)[0x432fa5]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x7f76bafccbd4]
/home/wangnan/perf[0x4330d4]
This is because 'perf.data' is interpreted as a symbol filter, and the
result is empty, so selection is empty. However,
hist_browser__toggle_fold() forgets to check it.
This patch simply return false when selection is NULL.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1449455746-41952-2-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
With the following steps:
Step 1: perf report
Step 2: Use UP/DOWN to select an entry, don't press 'ENTER'
Step 3: Use '/' to filter symbols, use a filter which returns
empty result
Step 4: Press 'ENTER'
We see that, even if we have filtered all the symbols (and the main
interface is empty), pressing 'ENTER' still selects one symbol. This
behavior surprises the user.
This patch resets browser->{he_,}selection in hist_browser__refresh()
and lets it choose default selection. In this case
browser->{he_,}selection keeps NULL so user won't see annotation item in
menu.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1449455746-41952-4-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Before this patch we can trigger a segfault by following steps:
Step 0: Use 'perf record' to generate a perf.data without callchain
Step 1: perf report
Step 2: Use UP/DOWN to select an entry, don't press 'ENTER'
Step 3: Use '/' to filter symbols, use a filter which returns
empty result
Step 4: Press 'ENTER' (notice here that the old selection is still
there. This is another problem)
Step 5: Press 'ENTER' to annotate that symbol
Step 6: Press 'LEFT' to go out.
Result: segfault:
perf: Segmentation fault
-------- backtrace --------
/home/wangnan/perf[0x53e568]
/lib64/libc.so.6(+0x3545f)[0x7fba75d3245f]
/home/wangnan/perf[0x537516]
/home/wangnan/perf[0x533fef]
/home/wangnan/perf[0x53b347]
/home/wangnan/perf(perf_evlist__tui_browse_hists+0x96)[0x53d206]
/home/wangnan/perf(cmd_report+0x1b9f)[0x442c7f]
/home/wangnan/perf[0x47efa2]
/home/wangnan/perf(main+0x5f5)[0x432fa5]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x7fba75d1ebd4]
/home/wangnan/perf[0x4330d4]
This is because in this case 'nd' could be NULL in
ui_browser__hists_seek(), but that function never checks it.
This patch adds checker for potential NULL pointer in that function.
After this patch the above steps won't segfault.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1449455746-41952-3-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The --kernel option of perf buildid-list tool should show the running
kernel buildid. The functionality has been lost during other changes of
the related code.
The build_id__sprintf() function should return length of the build-id
string, but it was the length of the build-id raw data instead. Due to
that, some return value checking caused that the final string was not
printed out.
With this patch the build_id__sprintf() returns the correct value, so
the --kernel option works again.
Before:
# perf buildid-list --kernel
#
After:
# perf buildid-list --kernel
972c1edab5bdc06cc224af45d510af662a3c6972
#
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
LPU-Reference: 1448632089.24573.114.camel@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Now that we have generic MSR trace points we can remove the old
hackish perf MSR read tracing code.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vince Weaver <vincent.weaver@maine.edu>
Cc: rostedt@goodmis.org
Link: http://lkml.kernel.org/r/1449018060-1742-4-git-send-email-andi@firstfloor.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
For debugging low level code interacting with the CPU it is often
useful to trace the MSR read/writes. This gives a concise summary of
PMU and other operations.
perf has an ad-hoc way to do this using trace_printk, but it's
somewhat limited (and also now spews ugly boot messages when enabled)
Instead define real trace points for all MSR accesses.
This adds three new trace points: read_msr and write_msr and rdpmc.
They also report if the access faulted (if *_safe is used)
This allows filtering and triggering on specific MSR values, which
allows various more advanced debugging techniques.
All the values are well defined in the CPU documentation.
The trace can be post processed with
Documentation/trace/postprocess/decode_msr.py to add symbolic MSR
names to the trace.
I only added it to native MSR accesses in C, not paravirtualized or in
entry*.S (which is not too interesting)
Originally the patch kit moved the MSRs out of line. This uses an
alternative approach recommended by Steven Rostedt of only moving the
trace calls out of line, but open coding the access to the jump label.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Steven Rostedt <rostedt@goodmis.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vince Weaver <vincent.weaver@maine.edu>
Link: http://lkml.kernel.org/r/1449018060-1742-3-git-send-email-andi@firstfloor.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Steven recommended open coding access to tracepoint->key to add
trace points to headers. Unfortunately this is difficult for some
headers (such as x86 asm/msr.h) because including tracepoint.h
includes so many other headers that it causes include loops.
The main problem is the include of linux/rcupdate.h, which
pulls in a lot of other headers. The rcu header is only needed
when actually defining trace points.
Move the struct tracepoint into a separate tracepoint-defs.h
header that can be included without pulling in all of RCU.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Steven Rostedt <rostedt@goodmis.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vince Weaver <vincent.weaver@maine.edu>
Link: http://lkml.kernel.org/r/1449018060-1742-2-git-send-email-andi@firstfloor.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
asm/atomic.h doesn't really need asm/processor.h anymore. Everything
it uses has moved to other header files. So remove that include.
processor.h is a nasty header that includes lots of
other headers and makes it prone to include loops. Removing the
include here makes asm/atomic.h a "leaf" header that can
be safely included in most other headers.
The only fallout is in the lib/atomic tester which relied on
this implicit include. Give it an explicit include.
(the include is in ifdef because the user is also in ifdef)
Signed-off-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vince Weaver <vincent.weaver@maine.edu>
Cc: rostedt@goodmis.org
Link: http://lkml.kernel.org/r/1449018060-1742-1-git-send-email-andi@firstfloor.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Fix a definition in the perf rapl driver. __initconst must
be applied to a const object, but to declare a const pointer
you need to use * const ..., not const ... *
This fixes a section attribute conflict with LTO builds.
Signed-off-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vince Weaver <vincent.weaver@maine.edu>
Link: http://lkml.kernel.org/r/1448905722-2767-1-git-send-email-andi@firstfloor.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Various functions implement the same pattern to send IPIs to an
event's CPU. Collapse the easy ones in a common helper function to
reduce duplication.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vince Weaver <vincent.weaver@maine.edu>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
In case we monitor events system wide, we get EXIT event
(when configured) twice for each task that exited.
Note doubled lines with same pid/tid in following example:
$ sudo ./perf record -a
^C[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.480 MB perf.data (2518 samples) ]
$ sudo ./perf report -D | grep EXIT
0 60290687567581 0x59910 [0x38]: PERF_RECORD_EXIT(1250:1250):(1250:1250)
0 60290687568354 0x59948 [0x38]: PERF_RECORD_EXIT(1250:1250):(1250:1250)
0 60290687988744 0x59ad8 [0x38]: PERF_RECORD_EXIT(1250:1250):(1250:1250)
0 60290687989198 0x59b10 [0x38]: PERF_RECORD_EXIT(1250:1250):(1250:1250)
1 60290692567895 0x62af0 [0x38]: PERF_RECORD_EXIT(1253:1253):(1253:1253)
1 60290692568322 0x62b28 [0x38]: PERF_RECORD_EXIT(1253:1253):(1253:1253)
2 60290692739276 0x69a18 [0x38]: PERF_RECORD_EXIT(1252:1252):(1252:1252)
2 60290692739910 0x69a50 [0x38]: PERF_RECORD_EXIT(1252:1252):(1252:1252)
The reason is that the cpu contexts are processes each time
we call perf_event_task. I'm changing the perf_event_aux logic
to serve task_ctx and cpu contexts separately, which ensure we
don't get EXIT event generated twice on same cpu context.
This does not affect other auxiliary events, as they don't
use task_ctx at all.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vince Weaver <vincent.weaver@maine.edu>
Link: http://lkml.kernel.org/r/1446649205-5822-1-git-send-email-jolsa@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
We need to add rest of the flags to the constraint mask
instead of another INTEL_ARCH_EVENT_MASK, fixing a typo.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Vince Weaver <vincent.weaver@maine.edu>
Link: http://lkml.kernel.org/r/1447061071-28085-1-git-send-email-jolsa@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
User visible:
- Fix 'perf list' segfault due to lack of support for PERF_CONF_SW_BPF_OUTPUT
in an array used just for printing available events, robustify the code
involved (Arnaldo Carvalho de Melo)
- 'perf test unwind' should create kernel maps, now that entry works and the
test passes (Jiri Olsa)
- Fix showing the running kernel build id in 'perf buildid-list' (Michael Petlan)
- Fix command line symbol filtering in 'perf report' (Namhyung Kim)
Infrastructure:
- Extract and collect map info from BPF object files in libbpf (Wang Nan)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJWWQE0AAoJENZQFvNTUqpAwnUP/3/4ObYvypD+QWw3FAFwHYHf
0Wp1qP9kpzT1Up+8kDEUs4idhoAQ2O0uKT6k7ThyZfyvHISg2P6C2fcGDt1txrDu
Lm/eO5IyzohSS0xn0ubQW3IjAaiOJXnMiB3dg0LYulO7rDF4aTogbaZhVtyTiw/t
dlkYq3exh/0uzdm14os72BfeTxVPiSQmNsMUiM5nKJHYK/VxopKexFJcVUTGyavd
rkzKK0StMsQOPtEBcT12TUHpLh0Fka0eD62Qp4o2LKTK19ck9GzpeyHwAY2D8t23
8zNimZix3ZieF0TufHPDQZMomgpDizpqP5TpP9k4eTV9hUHvrFBRybOyrPB7d//l
CXfyWxDPfnWN7KftsCVIDSkKoWzrtTtsw8fJDqYKHDetxD4Nuh5CprgC90WxzuKy
aoCqopjjq+YELLTotsh25E5ZcduHCiLZU/FHVEqd9LW1JSAWIcG0XtOIDm2BrXoI
C1pfkh7z9/+QgGE0M7eU8lPTm7AWO4fIhqHF/f0Yr6QB6gqeYkD0ZkwCVyMqk8Dp
wn/nwxDUELCzP6+QHbhItCk5qZVH/+lAPF68A/K4paQheRTNjO4Cs9MFrVEqHROK
DjBM61vEkJ1OBap+vx91KCggTBVIwWZcHzPkPPYJQWGnSp+MgBuz/BChc1+GKB1S
68DXMb7qaGKK5sVcWrTS
=FhYQ
-----END PGP SIGNATURE-----
Merge tag 'perf-core-for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core
Pull perf/core improvements and fixes from Arnaldo Carvalho de Melo:
User visible changes:
- Fix 'perf list' segfault due to lack of support for PERF_CONF_SW_BPF_OUTPUT
in an array used just for printing available events, robustify the code
involved (Arnaldo Carvalho de Melo)
- 'perf test unwind' should create kernel maps, now that entry works and the
test passes (Jiri Olsa)
- Fix showing the running kernel build id in 'perf buildid-list' (Michael Petlan)
- Fix command line symbol filtering in 'perf report' (Namhyung Kim)
Infrastructure changes:
- Extract and collect map info from BPF object files in libbpf (Wang Nan)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Following patches are going to introduce BPF object level configuration
to enable setting values into BPF maps. To avoid confusion, this patch
renames existing 'config' in bpf-loader.c to 'program config'. Following
patches would introduce 'object config'.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1448614067-197576-4-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This patch collects name of maps in BPF object files and saves them into
'maps' field in 'struct bpf_object'. 'bpf_object__get_map_by_name' is
introduced to retrive fd and definitions of a map through its name.
Signed-off-by: He Kuang <hekuang@huawei.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: He Kuang <hekuang@huawei.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1448614067-197576-3-git-send-email-wangnan0@huawei.com
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This patch collects more information from maps sections in BPF object
files into 'struct bpf_object', enables later patches access those
information (such as the type and size of the map).
In this patch, a new handler 'struct bpf_map' is extracted in parallel
with bpf_object and bpf_program. Its iterator and accessor is also
created.
Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1448614067-197576-2-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When perf report on TUI was called with -S symbol filter, it should
update nr entries even if min_pcnt is 0. IIRC the reason was to update
nr entries after applying minimum percent threshold. But if symbol
filter was given on command line (with -S option), it should use
hists->nr_non_filtered_entries instead of hists->nr_entries.
So this patch fixes a bug of navigating hists browser that the cursor
goes beyond the number of entries when -S (or similar) option is used.
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1448645559-31167-3-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
If user gives a filter, perf marks the corresponding column elided and
omits the output. But it should process and aggregates samples using
the field, otherwise samples will be aggregated as if the column was not
there resulted in incorrect output.
For example, I'd like to set a filter on native_write_msr_safe. The
original overhead of the function is negligible.
$ perf report | grep native_write_msr_safe
0.00% swapper [kernel.vmlinux] native_write_msr_safe
0.00% perf [kernel.vmlinux] native_write_msr_safe
However adding -S option gives different output.
$ perf report -S native_write_msr_safe --percentage absolute | \
> grep -e swapper -e perf
51.47% swapper [kernel.vmlinux]
4.14% perf [kernel.vmlinux]
Since it aggregated samples using comm and dso only. In fact, the above
values are same when it sorts with -s comm,dso.
$ perf report -s comm,dso | grep -e swapper -e perf
51.47% swapper [kernel.vmlinux]
4.14% perf [kernel.vmlinux]
This resulted in TUI failure with -ERANGE since it tries to increase
sample hit count for annotation with wrong symbols due to incorrect
aggregation.
This patch fixes it not to skip elided fields when comparing samples in
order to insert them to the hists.
Commiter note:
After the patch, with a different workloads:
# perf report --show-total-period -S native_write_msr_safe --stdio
#
# symbol: native_write_msr_safe
#
# Samples: 455 of event 'cycles:pp'
# Event count (approx.): 134787489
#
# Overhead Period Command Shared Object
# ........ ...... ............... ................
#
0.22% 293081 qemu-system-x86 [vmlinux]
0.19% 255914 swapper [vmlinux]
0.00% 2054 Timer [vmlinux]
0.00% 1021 firefox [vmlinux]
0.00% 2 perf [vmlinux]
# perf report --show-total-period | grep native_write_msr_safe
Failed to open /tmp/perf-14838.map, continuing without symbols
0.22% 293081 qemu-system-x86 [vmlinux] [k] native_write_msr_safe
0.19% 255914 swapper [vmlinux] [k] native_write_msr_safe
0.00% 2054 Timer [vmlinux] [k] native_write_msr_safe
0.00% 1021 firefox [vmlinux] [k] native_write_msr_safe
0.00% 2 perf [vmlinux] [k] native_write_msr_safe
#
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1448645559-31167-2-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Currently when perf fails to process samples for some reason, it doesn't
show any message about the failure. This is very inconvenient for users
especially on TUI as screen is reset after the failure.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1448645559-31167-1-git-send-email-namhyung@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When a43eec3042 ("bpf: introduce bpf_perf_event_output() helper") added
PERF_COUNT_SW_BPF_OUTPUT we ended up with a new entry in the event_symbols_sw
array that wasn't initialized, thus set to NULL, fix print_symbol_events()
to check for that case so that we don't crash if this happens again.
(gdb) bt
#0 __match_glob (ignore_space=false, pat=<optimized out>, str=<optimized out>) at util/string.c:198
#1 strglobmatch (str=<optimized out>, pat=pat@entry=0x7fffffffe61d "stall") at util/string.c:252
#2 0x00000000004993a5 in print_symbol_events (type=1, syms=0x872880 <event_symbols_sw+160>, max=11, name_only=false, event_glob=0x7fffffffe61d "stall")
at util/parse-events.c:1615
#3 print_events (event_glob=event_glob@entry=0x7fffffffe61d "stall", name_only=false) at util/parse-events.c:1675
#4 0x000000000042c79e in cmd_list (argc=1, argv=0x7fffffffe390, prefix=<optimized out>) at builtin-list.c:68
#5 0x00000000004788a5 in run_builtin (p=p@entry=0x871758 <commands+120>, argc=argc@entry=2, argv=argv@entry=0x7fffffffe390) at perf.c:370
#6 0x0000000000420ab0 in handle_internal_command (argv=0x7fffffffe390, argc=2) at perf.c:429
#7 run_argv (argv=0x7fffffffe110, argcp=0x7fffffffe11c) at perf.c:473
#8 main (argc=2, argv=0x7fffffffe390) at perf.c:588
(gdb) p event_symbols_sw[PERF_COUNT_SW_BPF_OUTPUT]
$4 = {symbol = 0x0, alias = 0x0}
(gdb)
A patch to robustify perf to not segfault when the next counter gets added in
the kernel will follow this one.
Reported-by: Ingo Molnar <mingo@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-57wysblcjfrseb0zg5u7ek10@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When PERF_COUNT_SW_BPF_OUTPUT was added to the kernel we should've
added it to tools/perf, where it is used just to list events.
This ended up causing a segfault in commands like "perf list stall".
Fix it by adding that new software counter.
A patch to robustify perf to not segfault when the next counter gets
added in the kernel will follow this one.
Reported-by: Ingo Molnar <mingo@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: David Ahern <dsahern@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Wang Nan <wangnan0@huawei.com>
Link: http://lkml.kernel.org/n/tip-uya354upi3eprsey6mi5962d@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The 'perf test unwind' is failing because it forgot to create the kernel
maps, fix it.
After the patch:
# perf test unwind
40: Test dwarf unwind : Ok
Reported-and-Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Jiri Olsa <jolsa@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Link: http://lkml.kernel.org/r/20151127082121.GA24503@krava.brq.redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The --kernel option of perf buildid-list tool should show the running
kernel buildid. The functionality has been lost during other changes of
the related code.
The build_id__sprintf() function should return length of the build-id
string, but it was the length of the build-id raw data instead. Due to
that, some return value checking caused that the final string was not
printed out.
With this patch the build_id__sprintf() returns the correct value, so
the --kernel option works again.
Before:
# perf buildid-list --kernel
#
After:
# perf buildid-list --kernel
972c1edab5bdc06cc224af45d510af662a3c6972
#
Signed-off-by: Michael Petlan <mpetlan@redhat.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
LPU-Reference: 1448632089.24573.114.camel@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
