linux/arch/x86/xen
Stefano Stabellini b9e0d95c04 xen: mark local pages as FOREIGN in the m2p_override
When the frontend and the backend reside on the same domain, even if we
add pages to the m2p_override, these pages will never be returned by
mfn_to_pfn because the check "get_phys_to_machine(pfn) != mfn" will
always fail, so the pfn of the frontend will be returned instead
(resulting in a deadlock because the frontend pages are already locked).

INFO: task qemu-system-i38:1085 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
qemu-system-i38 D ffff8800cfc137c0     0  1085      1 0x00000000
 ffff8800c47ed898 0000000000000282 ffff8800be4596b0 00000000000137c0
 ffff8800c47edfd8 ffff8800c47ec010 00000000000137c0 00000000000137c0
 ffff8800c47edfd8 00000000000137c0 ffffffff82213020 ffff8800be4596b0
Call Trace:
 [<ffffffff81101ee0>] ? __lock_page+0x70/0x70
 [<ffffffff81a0fdd9>] schedule+0x29/0x70
 [<ffffffff81a0fe80>] io_schedule+0x60/0x80
 [<ffffffff81101eee>] sleep_on_page+0xe/0x20
 [<ffffffff81a0e1ca>] __wait_on_bit_lock+0x5a/0xc0
 [<ffffffff81101ed7>] __lock_page+0x67/0x70
 [<ffffffff8106f750>] ? autoremove_wake_function+0x40/0x40
 [<ffffffff811867e6>] ? bio_add_page+0x36/0x40
 [<ffffffff8110b692>] set_page_dirty_lock+0x52/0x60
 [<ffffffff81186021>] bio_set_pages_dirty+0x51/0x70
 [<ffffffff8118c6b4>] do_blockdev_direct_IO+0xb24/0xeb0
 [<ffffffff811e71a0>] ? ext3_get_blocks_handle+0xe00/0xe00
 [<ffffffff8118ca95>] __blockdev_direct_IO+0x55/0x60
 [<ffffffff811e71a0>] ? ext3_get_blocks_handle+0xe00/0xe00
 [<ffffffff811e91c8>] ext3_direct_IO+0xf8/0x390
 [<ffffffff811e71a0>] ? ext3_get_blocks_handle+0xe00/0xe00
 [<ffffffff81004b60>] ? xen_mc_flush+0xb0/0x1b0
 [<ffffffff81104027>] generic_file_aio_read+0x737/0x780
 [<ffffffff813bedeb>] ? gnttab_map_refs+0x15b/0x1e0
 [<ffffffff811038f0>] ? find_get_pages+0x150/0x150
 [<ffffffff8119736c>] aio_rw_vect_retry+0x7c/0x1d0
 [<ffffffff811972f0>] ? lookup_ioctx+0x90/0x90
 [<ffffffff81198856>] aio_run_iocb+0x66/0x1a0
 [<ffffffff811998b8>] do_io_submit+0x708/0xb90
 [<ffffffff81199d50>] sys_io_submit+0x10/0x20
 [<ffffffff81a18d69>] system_call_fastpath+0x16/0x1b

The explanation is in the comment within the code:

We need to do this because the pages shared by the frontend
(xen-blkfront) can be already locked (lock_page, called by
do_read_cache_page); when the userspace backend tries to use them
with direct_IO, mfn_to_pfn returns the pfn of the frontend, so
do_blockdev_direct_IO is going to try to lock the same pages
again resulting in a deadlock.

A simplified call graph looks like this:

pygrub                          QEMU
-----------------------------------------------
do_read_cache_page              io_submit
  |                              |
lock_page                       ext3_direct_IO
                                 |
                                bio_add_page
                                 |
                                lock_page

Internally the xen-blkback uses m2p_add_override to swizzle (temporarily)
a 'struct page' to have a different MFN (so that it can point to another
guest). It also can easily find out whether another pfn corresponding
to the mfn exists in the m2p, and can set the FOREIGN bit
in the p2m, making sure that mfn_to_pfn returns the pfn of the backend.

This allows the backend to perform direct_IO on these pages, but as a
side effect prevents the frontend from using get_user_pages_fast on
them while they are being shared with the backend.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
2012-06-14 14:03:41 -04:00
..
debugfs.c debugfs: Add support to print u32 array in debugfs 2012-04-17 00:18:36 -04:00
debugfs.h debugfs: Add support to print u32 array in debugfs 2012-04-17 00:18:36 -04:00
enlighten.c xen/setup: filter APERFMPERF cpuid feature out 2012-05-31 12:16:52 -04:00
grant-table.c Merge commit 'v3.2-rc3' into stable/for-linus-3.3 2011-12-20 17:01:18 -05:00
irq.c xen: use this_cpu_xxx replace percpu_xxx funcs 2012-01-24 12:20:24 -05:00
Kconfig xen: Make XEN_MAX_DOMAIN_MEMORY have more sensible defaults 2011-11-21 17:14:46 -05:00
Makefile xen/tracing: Fix tracing config option properly 2011-08-22 11:28:33 -04:00
mmu.c Merge branch 'stable/autoballoon.v5.2' into stable/for-linus-3.5 2012-05-07 15:33:27 -04:00
mmu.h xen: make a pile of mmu pvop functions static 2011-05-20 14:25:24 -07:00
multicalls.c xen/multicall: move *idx fields to start of mc_buffer 2011-07-18 15:43:46 -07:00
multicalls.h xen: use this_cpu_xxx replace percpu_xxx funcs 2012-01-24 12:20:24 -05:00
p2m.c xen: mark local pages as FOREIGN in the m2p_override 2012-06-14 14:03:41 -04:00
pci-swiotlb-xen.c X86 & IA64: adapt for dma_map_ops changes 2012-03-28 16:36:31 +02:00
platform-pci-unplug.c xen:pvhvm: Modpost section mismatch fix 2011-07-11 13:37:04 -04:00
setup.c xen/balloon: Subtract from xen_released_pages the count that is populated. 2012-05-30 10:16:37 -04:00
smp.c xen/smp: unbind irqworkX when unplugging vCPUs. 2012-05-21 09:26:04 -04:00
smp.h xen: implement apic ipi interface 2012-05-07 15:33:15 -04:00
spinlock.c debugfs: Add support to print u32 array in debugfs 2012-04-17 00:18:36 -04:00
suspend.c xen: suspend: add "arch" to pre/post suspend hooks 2011-02-25 16:43:12 +00:00
time.c Merge branch 'upstream/xen-settime' of git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen 2011-11-06 20:15:05 -08:00
trace.c xen/trace: Fix compile error when CONFIG_XEN_PRIVILEGED_GUEST is not set 2011-08-05 09:43:02 -04:00
vdso.h
vga.c xen: allow enable use of VGA console on dom0 2011-06-06 11:46:00 -04:00
xen-asm_32.S xen: x86_32: do not enable iterrupts when returning from exception in interrupt context 2011-09-01 12:54:42 -04:00
xen-asm_64.S xen: use iret for return from 64b kernel to 32b usermode 2009-12-03 11:14:54 -08:00
xen-asm.h xen: make direct versions of irq_enable/disable/save/restore to common code 2009-02-04 16:59:04 -08:00
xen-asm.S x86: style cleanups for xen assemblies 2009-02-05 20:25:41 +01:00
xen-head.S x86, asm: Cleanup unnecssary macros in asm-offsets.c 2011-02-25 16:37:32 -08:00
xen-ops.h xen/setup: update VA mapping when releasing memory during setup 2012-05-07 15:32:24 -04:00