mainlining shenanigans
Go to file
Steven Rostedt (VMware) aafe104aa9 tracing: Restructure trace_clock_global() to never block
It was reported that a fix to the ring buffer recursion detection would
cause a hung machine when performing suspend / resume testing. The
following backtrace was extracted from debugging that case:

Call Trace:
 trace_clock_global+0x91/0xa0
 __rb_reserve_next+0x237/0x460
 ring_buffer_lock_reserve+0x12a/0x3f0
 trace_buffer_lock_reserve+0x10/0x50
 __trace_graph_return+0x1f/0x80
 trace_graph_return+0xb7/0xf0
 ? trace_clock_global+0x91/0xa0
 ftrace_return_to_handler+0x8b/0xf0
 ? pv_hash+0xa0/0xa0
 return_to_handler+0x15/0x30
 ? ftrace_graph_caller+0xa0/0xa0
 ? trace_clock_global+0x91/0xa0
 ? __rb_reserve_next+0x237/0x460
 ? ring_buffer_lock_reserve+0x12a/0x3f0
 ? trace_event_buffer_lock_reserve+0x3c/0x120
 ? trace_event_buffer_reserve+0x6b/0xc0
 ? trace_event_raw_event_device_pm_callback_start+0x125/0x2d0
 ? dpm_run_callback+0x3b/0xc0
 ? pm_ops_is_empty+0x50/0x50
 ? platform_get_irq_byname_optional+0x90/0x90
 ? trace_device_pm_callback_start+0x82/0xd0
 ? dpm_run_callback+0x49/0xc0

With the following RIP:

RIP: 0010:native_queued_spin_lock_slowpath+0x69/0x200

Since the fix to the recursion detection would allow a single recursion to
happen while tracing, this lead to the trace_clock_global() taking a spin
lock and then trying to take it again:

ring_buffer_lock_reserve() {
  trace_clock_global() {
    arch_spin_lock() {
      queued_spin_lock_slowpath() {
        /* lock taken */
        (something else gets traced by function graph tracer)
          ring_buffer_lock_reserve() {
            trace_clock_global() {
              arch_spin_lock() {
                queued_spin_lock_slowpath() {
                /* DEAD LOCK! */

Tracing should *never* block, as it can lead to strange lockups like the
above.

Restructure the trace_clock_global() code to instead of simply taking a
lock to update the recorded "prev_time" simply use it, as two events
happening on two different CPUs that calls this at the same time, really
doesn't matter which one goes first. Use a trylock to grab the lock for
updating the prev_time, and if it fails, simply try again the next time.
If it failed to be taken, that means something else is already updating
it.

Link: https://lkml.kernel.org/r/20210430121758.650b6e8a@gandalf.local.home

Cc: stable@vger.kernel.org
Tested-by: Konstantin Kharlamov <hi-angel@yandex.ru>
Tested-by: Todd Brandt <todd.e.brandt@linux.intel.com>
Fixes: b02414c8f0 ("ring-buffer: Fix recursion protection transitions between interrupt context") # started showing the problem
Fixes: 14131f2f98 ("tracing: implement trace_clock_*() APIs") # where the bug happened
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=212761
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2021-04-30 13:48:07 -04:00
arch tracing: Fix various typos in comments 2021-03-23 14:08:18 -04:00
block block: Discard page cache of zone reset target range 2021-03-11 11:49:25 -07:00
certs certs: Replace K{U,G}IDT_INIT() with GLOBAL_ROOT_{U,G}ID 2021-01-21 16:16:10 +00:00
crypto crypto: mips/poly1305 - enable for all MIPS processors 2021-03-08 11:52:17 +01:00
Documentation A set of irqchip updates: 2021-03-14 13:33:33 -07:00
drivers A set of irqchip updates: 2021-03-14 13:33:33 -07:00
fs tracing: Fix various typos in comments 2021-03-23 14:08:18 -04:00
include tracing: Remove unused argument from "ring_buffer_time_stamp() 2021-04-01 14:18:32 -04:00
init bootconfig: Update prototype of setup_boot_config() 2021-03-18 12:58:26 -04:00
ipc fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
kernel tracing: Restructure trace_clock_global() to never block 2021-04-30 13:48:07 -04:00
lib kasan: fix KASAN_STACK dependency for HW_TAGS 2021-03-13 11:27:31 -08:00
LICENSES LICENSES: Add the CC-BY-4.0 license 2020-12-08 10:33:27 -07:00
mm Merge branch 'akpm' (patches from Andrew) 2021-03-14 12:23:34 -07:00
net NFS Client Bugfixes for Linux 5.10-rc3 2021-03-12 14:19:35 -08:00
samples Merge git://git.kernel.org:/pub/scm/linux/kernel/git/netdev/net 2021-03-09 17:15:56 -08:00
scripts scripts/recordmcount.pl: Make vim and emacs indent the same 2021-03-22 17:51:16 -04:00
security Revert 95ebabde38 ("capabilities: Don't allow writing ambiguous v3 file capabilities") 2021-03-12 15:27:14 -06:00
sound ALSA: hda/hdmi: Cancel pending works before suspend 2021-03-10 12:52:01 +01:00
tools tools/latency-collector: Remove unneeded semicolon 2021-03-18 12:58:26 -04:00
usr Kbuild updates for v5.12 2021-02-25 10:17:31 -08:00
virt KVM: x86/mmu: Consider the hva in mmu_notifier retry 2021-02-22 13:16:53 -05:00
.clang-format cxl for 5.12 2021-02-24 09:38:36 -08:00
.cocciconfig
.get_maintainer.ignore Opt out of scripts/get_maintainer.pl 2019-05-16 10:53:40 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for dts files 2019-12-04 19:44:11 -08:00
.gitignore clang-lto series for v5.12-rc1 2021-02-23 09:28:51 -08:00
.mailmap treewide: Miguel has moved 2021-02-26 09:41:03 -08:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS treewide: Miguel has moved 2021-02-26 09:41:03 -08:00
Kbuild kbuild: rename hostprogs-y/always to hostprogs/always-y 2020-02-04 01:53:07 +09:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS Merge branch 'akpm' (patches from Andrew) 2021-03-14 12:23:34 -07:00
Makefile Linux 5.12-rc3 2021-03-14 14:41:02 -07:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.