linux/arch/arm
Srivatsa S. Bhat 8146875de7 arm, kvm: Fix CPU hotplug callback registration
On 03/15/2014 12:40 AM, Christoffer Dall wrote:
> On Fri, Mar 14, 2014 at 11:13:29AM +0530, Srivatsa S. Bhat wrote:
>> On 03/13/2014 04:51 AM, Christoffer Dall wrote:
>>> On Tue, Mar 11, 2014 at 02:05:38AM +0530, Srivatsa S. Bhat wrote:
>>>> Subsystems that want to register CPU hotplug callbacks, as well as perform
>>>> initialization for the CPUs that are already online, often do it as shown
>>>> below:
>>>>
[...]
>>> Just so we're clear, the existing code was simply racy as not prone to
>>> deadlocks, right?
>>>
>>> This makes it clear that the test above for compatible CPUs can be quite
>>> easily evaded by using CPU hotplug, but we don't really have a good
>>> solution for handling that yet...  Hmmm, grumble grumble, I guess if you
>>> hotplug unsupported CPUs on a KVM/ARM system for now, stuff will break.
>>>
>>
>> In this particular case, there was no deadlock possibility, rather the
>> existing code had insufficient synchronization against CPU hotplug.
>>
>> init_hyp_mode() would invoke cpu_init_hyp_mode() on currently online CPUs
>> using on_each_cpu(). If a CPU came online after this point and before calling
>> register_cpu_notifier(), that CPU would remain uninitialized because this
>> subsystem would miss the hot-online event. This patch fixes this bug and
>> also uses the new synchronization method (instead of get/put_online_cpus())
>> to ensure that we don't deadlock with CPU hotplug.
>>
>
> Yes, that was my conclusion as well.  Thanks for clarifying.  (It could
> be noted in the commit message as well if you should feel so inclined).
>

Please find the patch with updated changelog (and your Ack) below.
(No changes in code).

From: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Subject: [PATCH] arm, kvm: Fix CPU hotplug callback registration

Subsystems that want to register CPU hotplug callbacks, as well as perform
initialization for the CPUs that are already online, often do it as shown
below:

	get_online_cpus();

	for_each_online_cpu(cpu)
		init_cpu(cpu);

	register_cpu_notifier(&foobar_cpu_notifier);

	put_online_cpus();

This is wrong, since it is prone to ABBA deadlocks involving the
cpu_add_remove_lock and the cpu_hotplug.lock (when running concurrently
with CPU hotplug operations).

Instead, the correct and race-free way of performing the callback
registration is:

	cpu_notifier_register_begin();

	for_each_online_cpu(cpu)
		init_cpu(cpu);

	/* Note the use of the double underscored version of the API */
	__register_cpu_notifier(&foobar_cpu_notifier);

	cpu_notifier_register_done();

In the existing arm kvm code, there is no synchronization with CPU hotplug
to avoid missing the hotplug events that might occur after invoking
init_hyp_mode() and before calling register_cpu_notifier(). Fix this bug
and also use the new synchronization method (instead of get/put_online_cpus())
to ensure that we don't deadlock with CPU hotplug.

Cc: Gleb Natapov <gleb@kernel.org>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Ingo Molnar <mingo@kernel.org>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2014-03-20 13:43:41 +01:00
..
boot ARM: SoC fixes for 3.14-rc 2014-03-09 19:27:31 -07:00
common ARM: SoC platform changes for 3.14 2014-01-23 18:40:49 -08:00
configs ARM: tegra: add LED options back into tegra_defconfig 2014-02-28 21:35:58 +01:00
crypto CRYPTO: Fix more AES build errors 2014-01-05 13:59:56 +00:00
firmware
include ARM: fix noMMU kallsyms symbol filtering 2014-03-07 22:04:06 +00:00
kernel arm, hw-breakpoint: Fix CPU hotplug callback registration 2014-03-20 13:43:41 +01:00
kvm arm, kvm: Fix CPU hotplug callback registration 2014-03-20 13:43:41 +01:00
lib ARM: 7877/1: use built-in byte swap function 2013-12-29 12:32:45 +00:00
mach-at91 ARM: SoC platform changes for 3.14 2014-01-23 18:40:49 -08:00
mach-bcm ARM: pinctrl: Add Broadcom Capri pinctrl driver 2014-01-16 14:25:38 +01:00
mach-bcm2835 ARM: bcm2835: Fix grammar in help message 2014-01-02 12:07:52 -08:00
mach-berlin
mach-clps711x ARM: SoC platform changes for 3.14 2014-01-23 18:40:49 -08:00
mach-cns3xxx
mach-davinci ARM: driver updates for 3.14 2014-01-23 18:49:36 -08:00
mach-dove
mach-ebsa110
mach-efm32
mach-ep93xx
mach-exynos ACPI and power management updates for 3.14-rc1 2014-01-24 15:51:02 -08:00
mach-footbridge Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input 2014-01-24 17:17:30 -08:00
mach-gemini
mach-highbank Merge branches 'amba', 'fixes', 'kees', 'misc' and 'unstable/sa11x0' into for-next 2014-01-21 21:26:33 +00:00
mach-hisi ARM: hisi: fix kconfig warning on HAVE_ARM_TWD 2014-02-02 19:52:42 -08:00
mach-imx ARM: imx6: build pm-imx6q.c independently of CONFIG_PM 2014-02-18 16:03:43 -08:00
mach-integrator ARM: integrator: restore static map on the CP 2014-01-31 14:58:52 -08:00
mach-iop13xx
mach-iop32x ARM: iop32x: fix power off handling for the EM7210 board 2014-01-31 14:58:53 -08:00
mach-iop33x
mach-ixp4xx ARM: SoC platform changes for 3.14 2014-01-23 18:40:49 -08:00
mach-keystone ARM: keystone: config: fix build warning when CONFIG_DMADEVICES is not set 2014-01-31 15:03:53 -08:00
mach-kirkwood mvebu fixes for v3.13 (incremental #2) 2014-01-31 14:59:28 -08:00
mach-ks8695
mach-lpc32xx
mach-mmp ARM: SoC platform changes for 3.14 2014-01-23 18:40:49 -08:00
mach-moxart ARM: moxart: move DMA_OF selection to driver 2014-02-02 19:52:45 -08:00
mach-msm ARM: driver updates for 3.14 2014-01-23 18:49:36 -08:00
mach-mv78xx0
mach-mvebu mvebu fixes for v3.13 (incremental #2) 2014-01-31 14:59:28 -08:00
mach-mxs
mach-netx
mach-nomadik
mach-nspire
mach-omap1 ARM: OMAP1: nokia770: enable tahvo-usb 2014-02-13 15:20:14 -08:00
mach-omap2 Fixes for omaps mostly to fix the 3430 display regression, 2014-02-28 16:01:28 +01:00
mach-orion5x
mach-picoxcell
mach-prima2
mach-pxa Merge tag 'pwm_pxa_for_v3.14' of https://git.kernel.org/pub/scm/linux/kernel/git/hzhuang1/linux into fixes 2014-02-18 15:32:21 -08:00
mach-realview
mach-rockchip
mach-rpc
mach-s3c24xx ARM: SoC cleanups for 3.14 2014-01-23 18:36:55 -08:00
mach-s3c64xx ARM: SoC cleanups for 3.14 2014-01-23 18:36:55 -08:00
mach-s5p64x0
mach-s5pc100
mach-s5pv210
mach-sa1100 ARM: 7991/1: sa1100: fix compile problem on Collie 2014-03-07 22:04:08 +00:00
mach-shmobile ARM: fix HAVE_ARM_TWD selection for OMAP and shmobile 2014-02-02 19:52:53 -08:00
mach-socfpga
mach-spear
mach-sti
mach-sunxi ARM: SoC platform changes for 3.14 2014-01-23 18:40:49 -08:00
mach-tegra ARM: SoC fixes for 3.14-rc 2014-02-23 17:38:04 -08:00
mach-u300 ARM: SoC platform changes for 3.14 2014-01-23 18:40:49 -08:00
mach-ux500 ARM: SoC DT updates for 3.14 2014-01-23 18:45:38 -08:00
mach-versatile
mach-vexpress
mach-virt
mach-vt8500
mach-w90x900
mach-zynq ARM: zynq: Reserve not DMAable space in front of the kernel 2014-02-10 10:45:42 -08:00
mm Merge branch 'fixes' of git://ftp.arm.linux.org.uk/~rmk/linux-arm 2014-03-07 17:39:32 -08:00
net bpf: do not use reciprocal divide 2014-01-15 17:02:08 -08:00
nwfpe
oprofile
plat-iop
plat-omap
plat-orion mvebu fixes for v3.13 (incremental #2) 2014-01-31 14:59:28 -08:00
plat-pxa
plat-samsung MTD updates for 3.14: 2014-01-28 18:56:37 -08:00
plat-versatile ARM: SoC platform changes for 3.14 2014-01-23 18:40:49 -08:00
tools
vfp
xen xen/gnttab: Use phys_addr_t to describe the grant frame base address 2014-01-30 12:56:34 +00:00
Kconfig Merge branch 'fixes' of git://ftp.arm.linux.org.uk/~rmk/linux-arm 2014-03-07 17:39:32 -08:00
Kconfig-nommu
Kconfig.debug ARM: SoC platform changes for 3.14 2014-01-23 18:40:49 -08:00
Makefile Merge branches 'fixes' and 'misc' into for-linus 2014-01-28 21:38:48 +00:00