linux/arch/x86/xen
Konrad Rzeszutek Wilk 2e2fb75475 xen/setup: Populate freed MFNs from non-RAM E820 entries and gaps to E820 RAM
When the Xen hypervisor boots a PV kernel it hands it two pieces
of information: nr_pages and a made up E820 entry.

The nr_pages value defines the range from zero to nr_pages of PFNs
which have a valid Machine Frame Number (MFN) underneath it. The
E820 mirrors that (with the VGA hole):
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 00000000000a0000 (usable)
 Xen: 00000000000a0000 - 0000000000100000 (reserved)
 Xen: 0000000000100000 - 0000000080800000 (usable)

The fun comes when a PV guest that is run with a machine E820 - that
can either be the initial domain or a PCI PV guest, where the E820
looks like the normal thing:

BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 000000000009e000 (usable)
 Xen: 000000000009ec00 - 0000000000100000 (reserved)
 Xen: 0000000000100000 - 0000000020000000 (usable)
 Xen: 0000000020000000 - 0000000020200000 (reserved)
 Xen: 0000000020200000 - 0000000040000000 (usable)
 Xen: 0000000040000000 - 0000000040200000 (reserved)
 Xen: 0000000040200000 - 00000000bad80000 (usable)
 Xen: 00000000bad80000 - 00000000badc9000 (ACPI NVS)
..
With that overlaying the nr_pages directly on the E820 does not
work as there are gaps and non-RAM regions that won't be used
by the memory allocator. The 'xen_release_chunk' helps with that
by punching holes in the P2M (PFN to MFN lookup tree) for those
regions and tells us that:

Freeing  20000-20200 pfn range: 512 pages freed
Freeing  40000-40200 pfn range: 512 pages freed
Freeing  bad80-badf4 pfn range: 116 pages freed
Freeing  badf6-bae7f pfn range: 137 pages freed
Freeing  bb000-100000 pfn range: 282624 pages freed
Released 283999 pages of unused memory

Those 283999 pages are subtracted from the nr_pages and are returned
to the hypervisor. The end result is that the initial domain
boots with 1GB less memory as the nr_pages has been subtracted by
the amount of pages residing within the PCI hole. It can balloon up
to that if desired using 'xl mem-set 0 8092', but the balloon driver
is not always compiled in for the initial domain.

This patch, implements the populate hypercall (XENMEM_populate_physmap)
which increases the the domain with the same amount of pages that
were released.

The other solution (that did not work) was to transplant the MFN in
the P2M tree - the ones that were going to be freed were put in
the E820_RAM regions past the nr_pages. But the modifications to the
M2P array (the other side of creating PTEs) were not carried away.
As the hypervisor is the only one capable of modifying that and the
only two hypercalls that would do this are: the update_va_mapping
(which won't work, as during initial bootup only PFNs up to nr_pages
are mapped in the guest) or via the populate hypercall.

The end result is that the kernel can now boot with the
nr_pages without having to subtract the 283999 pages.

On a 8GB machine, with various dom0_mem= parameters this is what we get:

no dom0_mem
-Memory: 6485264k/9435136k available (5817k kernel code, 1136060k absent, 1813812k reserved, 2899k data, 696k init)
+Memory: 7619036k/9435136k available (5817k kernel code, 1136060k absent, 680040k reserved, 2899k data, 696k init)

dom0_mem=3G
-Memory: 2616536k/9435136k available (5817k kernel code, 1136060k absent, 5682540k reserved, 2899k data, 696k init)
+Memory: 2703776k/9435136k available (5817k kernel code, 1136060k absent, 5595300k reserved, 2899k data, 696k init)

dom0_mem=max:3G
-Memory: 2696732k/4281724k available (5817k kernel code, 1136060k absent, 448932k reserved, 2899k data, 696k init)
+Memory: 2702204k/4281724k available (5817k kernel code, 1136060k absent, 443460k reserved, 2899k data, 696k init)

And the 'xm list' or 'xl list' now reflect what the dom0_mem=
argument is.

Acked-by: David Vrabel <david.vrabel@citrix.com>
[v2: Use populate hypercall]
[v3: Remove debug printks]
[v4: Simplify code]
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
2012-05-07 15:31:46 -04:00
..
debugfs.c switch debugfs to umode_t 2012-01-03 22:54:56 -05:00
debugfs.h switch debugfs to umode_t 2012-01-03 22:54:56 -05:00
enlighten.c xen/enlighten: Expose MWAIT and MWAIT_LEAF if hypervisor OKs it. 2012-03-10 12:44:44 -05:00
grant-table.c Merge commit 'v3.2-rc3' into stable/for-linus-3.3 2011-12-20 17:01:18 -05:00
irq.c xen: use this_cpu_xxx replace percpu_xxx funcs 2012-01-24 12:20:24 -05:00
Kconfig xen: Make XEN_MAX_DOMAIN_MEMORY have more sensible defaults 2011-11-21 17:14:46 -05:00
Makefile xen/tracing: Fix tracing config option properly 2011-08-22 11:28:33 -04:00
mmu.c xen/x86: Workaround 'x86/ioapic: Add register level checks to detect bogus io-apic entries' 2012-04-06 12:13:06 -04:00
mmu.h xen: make a pile of mmu pvop functions static 2011-05-20 14:25:24 -07:00
multicalls.c xen/multicall: move *idx fields to start of mc_buffer 2011-07-18 15:43:46 -07:00
multicalls.h xen: use this_cpu_xxx replace percpu_xxx funcs 2012-01-24 12:20:24 -05:00
p2m.c xen/p2m: An early bootup variant of set_phys_to_machine 2012-04-06 17:03:06 -04:00
pci-swiotlb-xen.c treewide: fix a few typos in comments 2011-05-10 10:16:21 +02:00
platform-pci-unplug.c xen:pvhvm: Modpost section mismatch fix 2011-07-11 13:37:04 -04:00
setup.c xen/setup: Populate freed MFNs from non-RAM E820 entries and gaps to E820 RAM 2012-05-07 15:31:46 -04:00
smp.c xen/smp: Remove unnecessary call to smp_processor_id() 2012-04-06 12:13:30 -04:00
spinlock.c xen: Use this_cpu_ops 2010-12-17 15:07:19 +01:00
suspend.c xen: suspend: add "arch" to pre/post suspend hooks 2011-02-25 16:43:12 +00:00
time.c Merge branch 'upstream/xen-settime' of git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen 2011-11-06 20:15:05 -08:00
trace.c xen/trace: Fix compile error when CONFIG_XEN_PRIVILEGED_GUEST is not set 2011-08-05 09:43:02 -04:00
vdso.h
vga.c xen: allow enable use of VGA console on dom0 2011-06-06 11:46:00 -04:00
xen-asm_32.S xen: x86_32: do not enable iterrupts when returning from exception in interrupt context 2011-09-01 12:54:42 -04:00
xen-asm_64.S xen: use iret for return from 64b kernel to 32b usermode 2009-12-03 11:14:54 -08:00
xen-asm.h
xen-asm.S
xen-head.S x86, asm: Cleanup unnecssary macros in asm-offsets.c 2011-02-25 16:37:32 -08:00
xen-ops.h Merge branch 'stable/vga.support' into stable/drivers 2011-06-21 09:25:41 -04:00