linux/Documentation
Vladimir Davydov 33c3fc71c8 mm: introduce idle page tracking
Knowing the portion of memory that is not used by a certain application or
memory cgroup (idle memory) can be useful for partitioning the system
efficiently, e.g.  by setting memory cgroup limits appropriately.
Currently, the only means to estimate the amount of idle memory provided
by the kernel is /proc/PID/{clear_refs,smaps}: the user can clear the
access bit for all pages mapped to a particular process by writing 1 to
clear_refs, wait for some time, and then count smaps:Referenced.  However,
this method has two serious shortcomings:

 - it does not count unmapped file pages
 - it affects the reclaimer logic

To overcome these drawbacks, this patch introduces two new page flags,
Idle and Young, and a new sysfs file, /sys/kernel/mm/page_idle/bitmap.
A page's Idle flag can only be set from userspace by setting bit in
/sys/kernel/mm/page_idle/bitmap at the offset corresponding to the page,
and it is cleared whenever the page is accessed either through page tables
(it is cleared in page_referenced() in this case) or using the read(2)
system call (mark_page_accessed()). Thus by setting the Idle flag for
pages of a particular workload, which can be found e.g.  by reading
/proc/PID/pagemap, waiting for some time to let the workload access its
working set, and then reading the bitmap file, one can estimate the amount
of pages that are not used by the workload.

The Young page flag is used to avoid interference with the memory
reclaimer.  A page's Young flag is set whenever the Access bit of a page
table entry pointing to the page is cleared by writing to the bitmap file.
If page_referenced() is called on a Young page, it will add 1 to its
return value, therefore concealing the fact that the Access bit was
cleared.

Note, since there is no room for extra page flags on 32 bit, this feature
uses extended page flags when compiled on 32 bit.

[akpm@linux-foundation.org: fix build]
[akpm@linux-foundation.org: kpageidle requires an MMU]
[akpm@linux-foundation.org: decouple from page-flags rework]
Signed-off-by: Vladimir Davydov <vdavydov@parallels.com>
Reviewed-by: Andres Lagar-Cavilla <andreslc@google.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Greg Thelen <gthelen@google.com>
Cc: Michel Lespinasse <walken@google.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Pavel Emelyanov <xemul@parallels.com>
Cc: Cyrill Gorcunov <gorcunov@openvz.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-09-10 13:29:01 -07:00
..
ABI xen: features and fixes for 4.3-rc0 2015-09-08 11:46:48 -07:00
accounting
acpi ACPI / Documentation: Update method tracing documentation. 2015-08-07 02:42:22 +02:00
aoe
arm ARM: SoC platform updates for v4.3 2015-09-01 12:18:40 -07:00
arm64 arm64/Documentation: clarify wording regarding memory below the Image 2015-07-29 18:32:10 +01:00
auxdisplay
backlight
blackfin Docs: blackfin: Use new switch macro SAMPLE_IRQ_TIMER instead of IRQ_TIMER5 2015-05-07 09:35:14 -06:00
block Documentation: update notes in biovecs about arbitrarily sized bios 2015-08-13 12:32:07 -06:00
blockdev zsmalloc: account the number of compacted pages 2015-09-08 15:35:28 -07:00
bus-devices
cdrom
cgroups Merge branch 'for-4.3-unified-base' into for-4.3 2015-08-25 14:19:29 -04:00
cma cma: debug: document new debugfs interface 2015-04-14 16:49:00 -07:00
connector
console
cpu-freq cpufreq: remove redundant CPUFREQ_INCOMPATIBLE notifier event 2015-09-01 15:50:38 +02:00
cpuidle
cris
crypto
development-process
device-mapper dm raid: document RAID 4/5/6 discard support 2015-08-31 15:05:31 -04:00
devicetree IPMI updates for 4.3 2015-09-08 18:19:17 -07:00
dmaengine Documentation: dmaengine: Add DMA_CTRL_REUSE documentation 2015-08-17 13:46:22 +05:30
DocBook media updates for v4.3-rc1 2015-09-05 18:21:14 -07:00
driver-model Char/Misc driver patches for 4.1-rc1 2015-04-21 09:42:58 -07:00
dvb
early-userspace
EDID
extcon
fault-injection futex: Fault/error injection capabilities 2015-07-20 11:45:45 +02:00
fb Documentation/fb: add documentation for sm712fb 2015-08-07 15:05:01 -07:00
features Documentation/features/vm: add feature description and arch support status for batched TLB flush after unmap 2015-09-04 16:54:41 -07:00
filesystems Merge branch 'akpm' (patches from Andrew) 2015-09-08 17:52:23 -07:00
firmware_class
fmc
frv
gpio Documentation: gpio: consumer: describe active low property 2015-08-17 10:38:52 +02:00
hid HID: sensor: Update document for custom sensor 2015-04-10 22:22:56 +02:00
hwmon hwmon: (fam15h_power) Add documentation for new processors support 2015-08-27 07:51:15 -07:00
i2c i2c: support 10 bit and slave addresses in sysfs 'new_device' 2015-08-24 14:05:15 +02:00
ia64
ide
infiniband IB/hfi1: add driver files 2015-08-28 22:59:36 -04:00
input Merge branch 'next' into for-linus 2015-09-03 10:01:44 -07:00
ioctl platform-drivers-x86 for 4.3-1 2015-09-08 16:26:18 -07:00
isdn
ja_JP Doc: ja_JP: Fix typo in HOWTO 2015-06-08 16:43:09 -06:00
kbuild modsign: Allow password to be specified for signing key 2015-08-07 16:26:14 +01:00
kdump
ko_KR
laptops Move freefall program from Documentation/ to tools/ 2015-06-08 16:42:07 -06:00
leds Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/cooloney/linux-leds 2015-07-01 19:09:11 -07:00
locking
m68k
memory-devices
metag
mic Char/Misc driver patches for 4.2-rc1 2015-06-26 14:51:15 -07:00
mips
misc-devices mei: add async event notification ioctls 2015-08-03 17:30:00 -07:00
mmc
mn10300
mtd
namespaces
netlabel
networking Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2015-09-03 08:08:17 -07:00
nfc NFC: Fix typo in nfc-hci.txt 2015-06-08 23:15:45 +02:00
nios2
nvdimm libnvdimm: Non-Volatile Devices 2015-06-26 11:23:38 -04:00
nvmem Documentation: nvmem: add nvmem api level and how-to doc 2015-08-05 13:43:45 -07:00
parisc
PCI The documentation tree update for 4.1. Numerous fixes, the overdue removal 2015-04-18 11:10:49 -04:00
pcmcia pcmcia: Fix typo in locking documentation 2015-08-07 14:34:58 +02:00
phy
platform
power Power management and ACPI material for v4.3-rc1 2015-09-01 19:45:46 -07:00
powerpc SCSI misc on 20150901 2015-09-02 12:22:54 -07:00
pps Doc: pps: Fix file name in pps.txt 2015-07-14 12:35:42 -06:00
prctl Documentation/prctl: don't build tsc tests when cross compiling 2015-06-22 16:05:04 -06:00
pti
ptp
rapidio
RCU Merge branches 'doc.2015.07.15a' and 'torture.2015.07.15a' into HEAD 2015-08-04 08:42:02 -07:00
s390 KVM: s390: remove outdated documentation 2015-07-29 11:02:35 +02:00
scheduler sched/dl/Documentation: Split Section 3 2015-05-19 08:39:21 +02:00
scsi Merge branch 'for-4.2/sg' of git://git.kernel.dk/linux-block 2015-06-25 15:22:36 -07:00
security Merge branch 'smack-for-4.3' of https://github.com/cschaufler/smack-next into next 2015-08-11 11:18:53 +10:00
serial Doc: serial-rs485.txt: update RS485 driver interface 2015-06-12 17:39:51 -07:00
sh
sound Doc: sound:oss: Fix typo in sound/oss 2015-06-09 17:23:00 +02:00
spi Documentation/spi/spidev_test.c: fix warning 2015-04-17 09:04:12 -04:00
sysctl mm/page_alloc.c: fix a misleading comment 2015-09-08 15:35:28 -07:00
target Documentation/target: Fix tcm_mod_builder.py build breakage 2015-07-24 17:48:55 -07:00
thermal thermal: support slope and offset coefficients 2015-05-11 19:46:52 -07:00
timers
tpm
trace Mostly this is just clean ups and micro optimizations. 2015-09-08 14:04:14 -07:00
usb Revert "usb: interface authorization: Documentation part" 2015-08-18 09:57:15 -07:00
vDSO Documentation/vDSO: don't build tests when cross compiling 2015-06-22 16:04:57 -06:00
video4linux [media] Documentation: update cardlists 2015-06-09 18:43:47 -03:00
virtual kvm/x86: add sending hyper-v crash notification to user space 2015-07-23 08:27:06 +02:00
vm mm: introduce idle page tracking 2015-09-10 13:29:01 -07:00
w1 w1: Add support for DS28EA00 sequence to w1-therm 2015-05-24 12:08:59 -07:00
watchdog watchdog: omap_wdt: early_enable module parameter 2015-07-01 16:43:17 +02:00
wimax
x86 Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-09-01 10:07:40 -07:00
xtensa
zh_CN Documentation updates for 4.2 2015-06-24 20:01:36 -07:00
00-INDEX
adding-syscalls.txt Documentation: describe how to add a system call 2015-08-13 17:54:06 -06:00
applying-patches.txt
assoc_array.txt
atomic_ops.txt locking/Documentation: Clarify failed cmpxchg() memory ordering semantics 2015-08-03 10:57:09 +02:00
bad_memory.txt
basic_profiling.txt
bcache.txt
binfmt_misc.txt
braille-console.txt
bt8xxgpio.txt
btmrvl.txt
BUG-HUNTING
bus-virt-phys-mapping.txt
cachetlb.txt
Changes Documentation/Changes: Now need OpenSSL devel packages for module signing 2015-08-28 08:00:30 +10:00
circular-buffers.txt
clk.txt clk: change clk_ops' ->determine_rate() prototype 2015-07-27 18:12:01 -07:00
coccinelle.txt
CodeOfConflict
CodingStyle Documentation: CodingStyle: remove broken links in the References section 2015-07-10 13:54:34 -06:00
cpu-hotplug.txt
cpu-load.txt
cputopology.txt Documentation: Update cputopology.txt 2015-05-27 15:22:15 +02:00
crc32.txt
dcdbas.txt
debugging-modules.txt
debugging-via-ohci1394.txt Doc: Change wikipedia's URL from http to https 2015-06-22 10:14:05 -06:00
dell_rbu.txt
devices.txt
digsig.txt
DMA-API-HOWTO.txt Documentation updates for 4.2 2015-06-24 20:01:36 -07:00
DMA-API.txt mm: add dma_pool_zalloc() call to DMA API 2015-09-08 15:35:28 -07:00
DMA-attributes.txt
dma-buf-sharing.txt dma-buf: cleanup dma_buf_export() to make it easily extensible 2015-04-21 14:47:16 +05:30
DMA-ISA-LPC.txt
dontdiff
dynamic-debug-howto.txt
edac.txt EDAC: Update Documentation/edac.txt 2015-06-24 18:17:40 +02:00
efi-stub.txt
eisa.txt
email-clients.txt fix Evolution submenu name in email-clients.txt 2015-07-24 15:05:57 +02:00
flexible-arrays.txt
futex-requeue-pi.txt
gcov.txt
gdb-kernel-debugging.txt
highuid.txt
HOWTO docs: update HOWTO for 3.x -> 4.x versioning 2015-08-24 11:28:17 -06:00
hsi.txt
hw_random.txt
hwspinlock.txt hwspinlock/core: add device tree support 2015-05-02 09:54:30 +03:00
init.txt
initrd.txt
intel_txt.txt
Intel-IOMMU.txt x86/vt-d: Fix documentation of DRHD 2015-08-25 10:44:49 +02:00
io_ordering.txt
io-mapping.txt
iostats.txt
IPMI.txt ipmi:ssif: Ignore spaces when comparing I2C adapter names 2015-05-05 14:24:45 -05:00
IRQ-affinity.txt
IRQ-domain.txt IRQCHIP: Update docs regarding irq_domain_add_tree() 2015-04-01 17:21:35 +02:00
IRQ.txt
irqflags-tracing.txt
isapnp.txt
java.txt
kasan.txt Merge branch 'doc/4.2' into docs-next 2015-06-08 17:04:11 -06:00
kernel-doc-nano-HOWTO.txt Documenation: Update location of docproc.c 2015-07-14 12:36:39 -06:00
kernel-docs.txt
kernel-parameters.txt xen: features and fixes for 4.3-rc0 2015-09-08 11:46:48 -07:00
kernel-per-CPU-kthreads.txt
kmemcheck.txt
kmemleak.txt Doc: Change wikipedia's URL from http to https 2015-06-22 10:14:05 -06:00
kobject.txt
kprobes.txt
kref.txt
kselftest.txt
ldm.txt
local_ops.txt
lockup-watchdogs.txt watchdog: add watchdog_cpumask sysctl to assist nohz 2015-06-24 17:49:40 -07:00
logo.gif
logo.txt
lzo.txt
magic-number.txt Documentation/magic-number: Remove SCC_MAGIC 2015-05-13 15:39:04 -04:00
mailbox.txt Documentation: minor typo fix in mailbox.txt 2015-08-13 18:03:18 -06:00
Makefile
ManagementStyle
md-cluster.txt md-cluster: fix deadlock issue on message lock 2015-08-31 19:41:41 +02:00
md.txt doc:md: fix typo in md.txt. 2015-06-23 06:49:44 -06:00
media-framework.txt
memory-barriers.txt Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-09-03 15:46:07 -07:00
memory-hotplug.txt
men-chameleon-bus.txt Documentation: Minor changes to men-chameleon-bus.txt 2015-07-24 15:15:17 +02:00
module-signing.txt Move certificate handling to its own directory 2015-08-14 16:06:13 +01:00
mono.txt
nommu-mmap.txt
ntb.txt NTB: Rename Intel code names to platform names 2015-07-04 14:09:25 -04:00
numastat.txt
oops-tracing.txt
padata.txt
parport-lowlevel.txt
parport.txt
percpu-rw-semaphore.txt
phy.txt phy: core: Add devm_of_phy_get_by_index to phy-core 2015-05-11 21:42:23 +05:30
pi-futex.txt
pinctrl.txt pinctrl: move strict option to pinmux_ops 2015-05-06 14:45:19 +02:00
pnp.txt
preempt-locking.txt x86/fpu: Rename math_state_restore() to fpu__restore() 2015-05-19 15:47:18 +02:00
printk-formats.txt The documentation tree update for 4.1. Numerous fixes, the overdue removal 2015-04-18 11:10:49 -04:00
pwm.txt
ramoops.txt
rbtree.txt
remoteproc.txt remoteproc: introduce rproc_get_by_phandle API 2015-06-16 21:12:52 +03:00
rfkill.txt
robust-futex-ABI.txt
robust-futexes.txt
rpmsg.txt
rtc.txt
SAK.txt
SecurityBugs
serial-console.txt
sgi-ioc4.txt
SM501.txt
smsc_ece1099.txt
sparse.txt
stable_api_nonsense.txt
stable_kernel_rules.txt stable: Update documentation to clarify preferred procedure 2015-05-22 09:38:56 -06:00
static-keys.txt jump label, locking/static_keys: Update docs 2015-08-03 11:51:14 +02:00
SubmitChecklist
SubmittingDrivers
SubmittingPatches Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2015-09-01 18:46:42 -07:00
svga.txt
sysfs-rules.txt
sysrq.txt mm, oom: do not panic for oom kills triggered from sysrq 2015-09-08 15:35:28 -07:00
this_cpu_ops.txt
unaligned-memory-access.txt
unicode.txt
unshare.txt
vfio.txt vfio: powerpc/spapr: Support Dynamic DMA windows 2015-06-11 15:16:55 +10:00
VGA-softcursor.txt
vgaarbiter.txt
video-output.txt
vme_api.txt Documentation: mention vme_master_mmap() in VME API 2015-06-12 17:26:56 -07:00
volatile-considered-harmful.txt
workqueue.txt workqueue: fix trivial typo in Documentation/workqueue.txt 2015-05-05 09:50:38 -04:00
xillybus.txt
xz.txt
zorro.txt