Since, through struct pci_host_bridge.map/swizzle_irq hooks, IRQs are now
allocated in the pci_assign_irq() callback automatically, PCI host bridge
drivers can stop relying on pci_fixup_irqs() for IRQ allocation.
Drop pci_fixup_irqs() usage from PCI designware host bridge driver.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Joao Pinto <Joao.Pinto@synopsys.com>
Since, through struct pci_host_bridge.map/swizzle_irq hooks, IRQs are now
allocated in the pci_assign_irq() callback automatically, PCI host bridge
drivers can stop relying on pci_fixup_irqs() for IRQ allocation.
Drop pci_fixup_irqs() usage from PCI iproc host bridge driver.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Scott Branden <sbranden@broadcom.com>
Cc: Ray Jui <rjui@broadcom.com>
Cc: Jon Mason <jonmason@broadcom.com>
Since, through struct pci_host_bridge.map/swizzle_irq hooks, IRQs are now
allocated in the pci_assign_irq() callback automatically, PCI host bridge
drivers can stop relying on pci_fixup_irqs() for IRQ allocation.
Drop pci_fixup_irqs() usage from PCI rcar host bridge driver.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Simon Horman <horms@verge.net.au>
Since, through struct pci_host_bridge.map/swizzle_irq hooks, IRQs are now
allocated in the pci_assign_irq() callback automatically, PCI host bridge
drivers can stop relying on pci_fixup_irqs() for IRQ allocation.
Drop pci_fixup_irqs() usage from PCI xilinx host bridge driver.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Michal Simek <michal.simek@xilinx.com>
Since, through struct pci_host_bridge.map/swizzle_irq hooks, IRQs are now
allocated in the pci_assign_irq() callback automatically, PCI host bridge
drivers can stop relying on pci_fixup_irqs() for IRQ allocation
Drop pci_fixup_irqs() usage from PCI tegra host bridge driver.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Legacy PCI host controllers (ie host controllers that set-up the PCI bus
through the ARM pci_common_init() API) are currently relying on
pci_fixup_irqs() to assign legacy PCI irqs to devices. This is not ideal
in that pci_fixup_irqs() assigns IRQs for all PCI devices present in a given
system some of which may well be enabled by the time pci_fixup_irqs() is
called (ie a system with multiple host controllers). With the introduction
of struct pci_host_bridge.(*map_irq) pointer it is possible to assign IRQs
for all devices originating from a PCI host bridge at probe time; this is
implemented through pci_assign_irq() that relies on the struct
pci_host_bridge.map_irq pointer to map IRQ for a given device.
The benefits this brings are twofold:
- the IRQ for a device is assigned once at probe time
- the IRQ assignment works also for hotplugged devices
Remove pci_fixup_irqs() call from bios32 code and rely on pci_assign_irq()
to carry out the IRQ mapping at device probe time.
The map_irq() and swizzle_irq() struct pci_host_bridge callbacks are set-up
in the struct pci_host_bridge created in the bios32 pcibios_init_hw()
function and mach-* code paths (for PCI mach implementations that require a
specific struct hw_pci.(*scan) function callback).
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
[bhelgaas: folded in fixes from Lorenzo:
http://lkml.kernel.org/r/20170701140629.GC8977@red-moon]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Andrew Lunn <andrew@lunn.ch>
The pci_assign_irq() function allows assignment of an IRQ to devices during
device enable time rather than only at boot. Therefore call it in the
pci_device_probe() function during the enable device code path so this
assignment can be performed.
This patch will do nothing on arches which do not set the IRQ mapping
function pointers and is therefore currently a nop, however as support for
these function pointers is added to arch-specific code this will cause IRQ
assignment to migrate to device enable time allowing the new code paths to
be used.
Signed-off-by: Matthew Minter <matt@masarand.com>
[lorenzo.pieralisi@arm.com: moved pci_assign_irq() call site]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
With the introduction of struct pci_host_bridge.(*map_irq) function pointer
add another direct usage of of_irq_parse_and_map_pci().
Update the function comment to reflect this behaviour.
Signed-off-by: Matthew Minter <matt@masarand.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Here we delete the static pdev_fixup_irq() function which is currently what
pci_fixup_irqs() uses to actually assign the IRQs and replace it with the
pci_assign_irq() function which changes the interface and uses the new
function pointers stored in the host bridge structure.
Eventually this will allow pci_fixup_irqs() to be removed entirely and the
new deferred assignment code path will call pci_assign_irq() directly.
However to ensure current users continue to work, a new implementation of
pci_fixup_irqs() is introduced which simply wraps the functionality of
pci_assign_irq().
Signed-off-by: Matthew Minter <matt@masarand.com>
[lorenzo.pieralisi@arm.com: reworked comments/log]
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
In order to defer IRQ assignment arches must be able to register functions
to map and swizzle interrupts. These registered functions are stored in
the pci_host_bridge struct.
Signed-off-by: Matthew Minter <matt@masarand.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
The functions included in setup-irq.o currently apply only to a selection
of architectures which share common IRQ assignment code. However this code
needs to be generalised for all arches to allow deferred IRQ assignment.
So the first step is to build it on all architectures.
Signed-off-by: Matthew Minter <matt@masarand.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
The pci_scan_root_bus_bridge() function allows passing a parameterized
struct pci_host_bridge and scanning the resulting PCI bus; since the struct
msi_controller is part of the struct pci_host_bridge and the struct
pci_host_bridge can now be passed to pci_scan_root_bus_bridge() explicitly,
there is no need for a scan interface with a MSI controller parameter.
With all PCI host controller drivers and platform code relying on
pci_scan_root_bus_msi() converted over to pci_scan_root_bus_bridge() the
pci_scan_root_bus_msi() becomes obsolete and can be removed.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
The introduction of pci_scan_root_bus_bridge() provides a PCI core API to
scan a PCI root bus backed by an already initialized struct pci_host_bridge
object, which simplifies the bus scan interface and makes the PCI scan root
bus interface easier to generalize as members are added to the struct
pci_host_bridge.
Convert PCI xilinx-nwl host code to pci_scan_root_bus_bridge() to improve
the PCI root bus scanning interface.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com>
The introduction of pci_scan_root_bus_bridge() provides a PCI core API to
scan a PCI root bus backed by an already initialized struct pci_host_bridge
object, which simplifies the bus scan interface and makes the PCI scan root
bus interface easier to generalize as members are added to the struct
pci_host_bridge.
Convert PCI rockchip host code to pci_scan_root_bus_bridge() to improve the
PCI root bus scanning interface.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Wenrui Li <wenrui.li@rock-chips.com>
Cc: Shawn Lin <shawn.lin@rock-chips.com>
The introduction of pci_scan_root_bus_bridge() provides a PCI core API to
scan a PCI root bus backed by an already initialized struct pci_host_bridge
object, which simplifies the bus scan interface and makes the PCI scan root
bus interface easier to generalize as members are added to the struct
pci_host_bridge.
Convert PCI host-common code to pci_scan_root_bus_bridge() to improve the
PCI root bus scanning interface.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Will Deacon <will.deacon@arm.com>
The introduction of pci_scan_root_bus_bridge() provides a PCI core API to
scan a PCI root bus backed by an already initialized struct pci_host_bridge
object, which simplifies the bus scan interface and makes the PCI scan root
bus interface easier to generalize as members are added to the struct
pci_host_bridge.
Convert PCI xgene host code to pci_scan_root_bus_bridge() to improve the
PCI root bus scanning interface.
Tested-by: Khuong Dinh <kdinh@apm.com> # with e1000e
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Tanmay Inamdar <tinamdar@apm.com>
The introduction of pci_scan_root_bus_bridge() provides a PCI core API to
scan a PCI root bus backed by an already initialized struct pci_host_bridge
object, which simplifies the bus scan interface and makes the PCI scan root
bus interface easier to generalize as members are added to the struct
pci_host_bridge.
Convert PCI xilinx host code to pci_scan_root_bus_bridge() to improve the
PCI root bus scanning interface.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
CC: Michal Simek <michal.simek@xilinx.com>
The introduction of pci_scan_root_bus_bridge() provides a PCI core API to
scan a PCI root bus backed by an already initialized struct pci_host_bridge
object, which simplifies the bus scan interface and makes the PCI scan root
bus interface easier to generalize as members are added to the struct
pci_host_bridge.
Convert PCI altera host code to pci_scan_root_bus_bridge() to improve the
PCI root bus scanning interface.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Ley Foon Tan <lftan@altera.com>
The introduction of pci_scan_root_bus_bridge() provides a PCI core API to
scan a PCI root bus backed by an already initialized struct pci_host_bridge
object, which simplifies the bus scan interface and makes the PCI scan root
bus interface easier to generalize as members are added to the struct
pci_host_bridge.
Convert PCI versatile host code to pci_scan_root_bus_bridge() to improve
the PCI root bus scanning interface.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
[bhelgaas: folded in fix from Arnd Bergmann <arnd@arndb.de>:
http://lkml.kernel.org/r/20170621215323.3921382-3-arnd@arndb.de]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Rob Herring <robh@kernel.org>
The introduction of pci_scan_root_bus_bridge() provides a PCI core API to
scan a PCI root bus backed by an already initialized struct pci_host_bridge
object, which simplifies the bus scan interface and makes the PCI scan root
bus interface easier to generalize as members are added to the struct
pci_host_bridge.
Convert PCI iproc host code to pci_scan_root_bus_bridge() to improve the
PCI root bus scanning interface.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Scott Branden <sbranden@broadcom.com>
Cc: Ray Jui <rjui@broadcom.com>
Cc: Jon Mason <jonmason@broadcom.com>
The introduction of pci_scan_root_bus_bridge() provides a PCI core API to
scan a PCI root bus backed by an already initialized struct pci_host_bridge
object, which simplifies the bus scan interface and makes the PCI scan root
bus interface easier to generalize as members are added to the struct
pci_host_bridge.
Convert PCI rcar host code to pci_scan_root_bus_bridge() to improve the PCI
root bus scanning interface.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Simon Horman <horms@verge.net.au>
The introduction of pci_scan_root_bus_bridge() provides a PCI core API to
scan a PCI root bus backed by an already initialized struct pci_host_bridge
object, which simplifies the bus scan interface and makes the PCI scan root
bus interface easier to generalize as members are added to the struct
pci_host_bridge.
Convert PCI aardvark host code to pci_scan_root_bus_bridge() to improve the
PCI root bus scanning interface.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The introduction of pci_scan_root_bus_bridge() provides a PCI core API to
scan a PCI root bus backed by an already initialized struct pci_host_bridge
object, which simplifies the bus scan interface and makes the PCI scan root
bus interface easier to generalize as members are added to the struct
pci_host_bridge.
Convert PCI designware host code to pci_scan_root_bus_bridge() to improve
the PCI root bus scanning interface.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Joao Pinto <Joao.Pinto@synopsys.com>
The introduction of pci_scan_root_bus_bridge() provides a PCI core API to
scan a PCI root bus backed by an already initialized struct pci_host_bridge
object, which simplifies the bus scan interface and makes the PCI scan root
bus interface easier to generalize as members are added to the struct
pci_host_bridge.
Convert ARM bios32 code to pci_scan_root_bus_bridge() to improve the PCI
root bus scanning interface.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
[bhelgaas: fold in warning fix from Arnd Bergmann <arnd@arndb.de>:
http://lkml.kernel.org/r/20170621215323.3921382-1-arnd@arndb.de]
[bhelgaas: set bridge->ops for mv78xx0]
[bhelgaas: fold in fixes from Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>:
http://lkml.kernel.org/r/20170701135457.GB8977@red-moon]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Andrew Lunn <andrew@lunn.ch>
With the introduction of pci_scan_root_bus_bridge() there is no need to
export pci_register_host_bridge() to other kernel subsystems other than the
PCI compilation unit that needs it.
Make pci_register_host_bridge() static to its compilation unit and convert
the existing drivers usage over to pci_scan_root_bus_bridge().
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Arnd Bergmann <arnd@arndb.de>
The current pci_scan_root_bus() interface is made up of two main code
paths:
- pci_create_root_bus()
- pci_scan_child_bus()
pci_create_root_bus() is a wrapper function that allows to create a struct
pci_host_bridge structure, initialize it with the passed parameters and
register it with the kernel.
As the struct pci_host_bridge require additional struct members,
pci_create_root_bus() parameters list has grown in time, making it unwieldy
to add further parameters to it in case the struct pci_host_bridge gains
more members fields to augment its functionality.
Since PCI core code provides functions to allocate struct pci_host_bridge,
instead of forcing the pci_create_root_bus() interface to add new
parameters to cater for new struct pci_host_bridge functionality, it is
more suitable to add an interface in PCI core code to scan a PCI bus
straight from a struct pci_host_bridge created and customized by each
specific PCI host controller driver.
Add a pci_scan_root_bus_bridge() function to allow PCI host controller
drivers to create and initialize struct pci_host_bridge and scan the
resulting bus.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Arnd Bergmann <arnd@arndb.de>
When probing the PCI host controller driver, if an error occurs, the probe
function code does not free memory allocated for the struct pci_host_bridge
resulting in memory leakage.
Move the struct pci_host_bridge allocation over to the respective devm
interface to fix the issue.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Arnd Bergmann <arnd@arndb.de>
When probing the PCI host controller driver, if an error occurs, the probe
function code does not free memory allocated for the struct pci_host_bridge
resulting in memory leakage.
Move the struct pci_host_bridge allocation over to the respective devm
interface to fix the issue.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Struct pci_host_bridge can be allocated by PCI host bridge drivers which
usually allocate and map memory through devm managed interfaces.
Add a devm version for the pci_alloc_host_bridge() interface to simplify
PCI host controller driver porting and simplify the driver failure paths.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Commit a52d1443bb ("PCI: Export host bridge registration interface")
exported the pci_alloc_host_bridge() interface so that PCI host controllers
drivers can make use of it.
Introduce pci_alloc_host_bridge() kernel counterpart to free the host
bridge data structures, pci_free_host_bridge(), export it and update kernel
functions releasing host bridge objects allocated memory to make use of it.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Arnd Bergmann <arnd@arndb.de>
The introduction of pci_register_host_bridge() kernel interface allows PCI
host controller drivers to create the struct pci_host_bridge object,
initialize it and register it with the kernel so that its corresponding PCI
bus can be scanned and its devices probed.
The host bridge device release function pci_release_host_bridge_dev() is a
static function common for all struct pci_host_bridge allocated objects, so
in its current form cannot be used by PCI host bridge controllers drivers
to initialize the allocated struct pci_host_bridge, which leaves struct
pci_host_bridge devices release function uninitialized.
Since pci_release_host_bridge_dev() is a function common to all PCI host
bridge objects, initialize it in pci_alloc_host_bridge() (ie common host
bridge allocation interface) so that all struct pci_host_bridge objects
have their release function initialized by default at allocation time,
removing the need for exporting the common pci_release_host_bridge_dev()
function to other compilation units.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Current ftpci100 driver host bridge controller driver requires struct
pci_bus to be created in order to mask and clear IRQs using standard PCI
bus config accessors.
This struct pci_bus dependency is fictitious and burdens the driver with
unneeded constraints (eg to use separate APIs to create and scan the root
bus).
Add PCI raw config space accessors to PCIe ftpci100 driver and remove the
fictitious struct pci_bus dependency.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
[bhelgaas: folded in raw PCI read accessor from
http://lkml.kernel.org/r/20170621162651.25315-1-linus.walleij@linaro.org
The clock piece of the above posting goes with the separate "Add clock
handling" patch.]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
The current iproc driver host bridge controller driver requires struct
pci_bus to be created in order to carry out PCI link checks with standard
PCI config space accessors.
This struct pci_bus dependency is fictitious and burdens the driver with
unneeded constraints (eg to use separate APIs to create and scan the root
bus).
Add PCI raw config space accessors to the iproc driver and remove the
fictitious struct pci_bus dependency.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Scott Branden <sbranden@broadcom.com>
Cc: Ray Jui <rjui@broadcom.com>
Cc: Jon Mason <jonmason@broadcom.com>
The nwl_pcie_enable_msi() second parameter (ie "bus") is unused and creates
a fake dependency on the struct pci_bus that need not exist.
Remove it.
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Bharat Kumar Gogada <bharat.kumar.gogada@xilinx.com>
Pull some more input subsystem updates from Dmitry Torokhov:
"An updated xpad driver with a few more recognized device IDs, and a
new psxpad-spi driver, allowing connecting Playstation 1 and 2 joypads
via SPI bus"
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/dtor/input:
Input: cros_ec_keyb - remove extraneous 'const'
Input: add support for PlayStation 1/2 joypads connected via SPI
Input: xpad - add USB IDs for Mad Catz Brawlstick and Razer Sabertooth
Input: xpad - sync supported devices with xboxdrv
Input: xpad - sort supported devices by USB ID
Pull UML fixes from Richard Weinberger:
"No new stuff, just fixes"
* 'for-linus-4.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rw/uml:
um: Add missing NR_CPUS include
um: Fix to call read_initrd after init_bootmem
um: Include kbuild.h instead of duplicating its macros
um: Fix PTRACE_POKEUSER on x86_64
um: Set number of CPUs
um: Fix _print_addr()
Merge misc fixes from Andrew Morton:
"15 fixes"
* emailed patches from Andrew Morton <akpm@linux-foundation.org>:
mm, docs: update memory.stat description with workingset* entries
mm: vmscan: scan until it finds eligible pages
mm, thp: copying user pages must schedule on collapse
dax: fix PMD data corruption when fault races with write
dax: fix data corruption when fault races with write
ext4: return to starting transaction in ext4_dax_huge_fault()
mm: fix data corruption due to stale mmap reads
dax: prevent invalidation of mapped DAX entries
Tigran has moved
mm, vmalloc: fix vmalloc users tracking properly
mm/khugepaged: add missed tracepoint for collapse_huge_page_swapin
gcov: support GCC 7.1
mm, vmstat: Remove spurious WARN() during zoneinfo print
time: delete current_fs_time()
hwpoison, memcg: forcibly uncharge LRU pages
Commit 4b4cea91691d ("mm: vmscan: fix IO/refault regression in cache
workingset transition") introduced three new entries in memory stat
file:
- workingset_refault
- workingset_activate
- workingset_nodereclaim
This commit adds a corresponding description to the cgroup v2 docs.
Link: http://lkml.kernel.org/r/1494530293-31236-1-git-send-email-guro@fb.com
Signed-off-by: Roman Gushchin <guro@fb.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Vladimir Davydov <vdavydov.dev@gmail.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Li Zefan <lizefan@huawei.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
We have encountered need_resched warnings in __collapse_huge_page_copy()
while doing {clear,copy}_user_highpage() over HPAGE_PMD_NR source pages.
mm->mmap_sem is held for write, but the iteration is well bounded.
Reschedule as needed.
Link: http://lkml.kernel.org/r/alpine.DEB.2.10.1705101426380.109808@chino.kir.corp.google.com
Signed-off-by: David Rientjes <rientjes@google.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This is based on a patch from Jan Kara that fixed the equivalent race in
the DAX PTE fault path.
Currently DAX PMD read fault can race with write(2) in the following
way:
CPU1 - write(2) CPU2 - read fault
dax_iomap_pmd_fault()
->iomap_begin() - sees hole
dax_iomap_rw()
iomap_apply()
->iomap_begin - allocates blocks
dax_iomap_actor()
invalidate_inode_pages2_range()
- there's nothing to invalidate
grab_mapping_entry()
- we add huge zero page to the radix tree
and map it to page tables
The result is that hole page is mapped into page tables (and thus zeros
are seen in mmap) while file has data written in that place.
Fix the problem by locking exception entry before mapping blocks for the
fault. That way we are sure invalidate_inode_pages2_range() call for
racing write will either block on entry lock waiting for the fault to
finish (and unmap stale page tables after that) or read fault will see
already allocated blocks by write(2).
Fixes: 9f141d6ef6 ("dax: Call ->iomap_begin without entry lock during dax fault")
Link: http://lkml.kernel.org/r/20170510172700.18991-1-ross.zwisler@linux.intel.com
Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Currently DAX read fault can race with write(2) in the following way:
CPU1 - write(2) CPU2 - read fault
dax_iomap_pte_fault()
->iomap_begin() - sees hole
dax_iomap_rw()
iomap_apply()
->iomap_begin - allocates blocks
dax_iomap_actor()
invalidate_inode_pages2_range()
- there's nothing to invalidate
grab_mapping_entry()
- we add zero page in the radix tree
and map it to page tables
The result is that hole page is mapped into page tables (and thus zeros
are seen in mmap) while file has data written in that place.
Fix the problem by locking exception entry before mapping blocks for the
fault. That way we are sure invalidate_inode_pages2_range() call for
racing write will either block on entry lock waiting for the fault to
finish (and unmap stale page tables after that) or read fault will see
already allocated blocks by write(2).
Fixes: 9f141d6ef6
Link: http://lkml.kernel.org/r/20170510085419.27601-5-jack@suse.cz
Signed-off-by: Jan Kara <jack@suse.cz>
Reviewed-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
DAX will return to locking exceptional entry before mapping blocks for a
page fault to fix possible races with concurrent writes. To avoid lock
inversion between exceptional entry lock and transaction start, start
the transaction already in ext4_dax_huge_fault().
Fixes: 9f141d6ef6
Link: http://lkml.kernel.org/r/20170510085419.27601-4-jack@suse.cz
Signed-off-by: Jan Kara <jack@suse.cz>
Cc: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Currently, we didn't invalidate page tables during invalidate_inode_pages2()
for DAX. That could result in e.g. 2MiB zero page being mapped into
page tables while there were already underlying blocks allocated and
thus data seen through mmap were different from data seen by read(2).
The following sequence reproduces the problem:
- open an mmap over a 2MiB hole
- read from a 2MiB hole, faulting in a 2MiB zero page
- write to the hole with write(3p). The write succeeds but we
incorrectly leave the 2MiB zero page mapping intact.
- via the mmap, read the data that was just written. Since the zero
page mapping is still intact we read back zeroes instead of the new
data.
Fix the problem by unconditionally calling invalidate_inode_pages2_range()
in dax_iomap_actor() for new block allocations and by properly
invalidating page tables in invalidate_inode_pages2_range() for DAX
mappings.
Fixes: c6dcf52c23
Link: http://lkml.kernel.org/r/20170510085419.27601-3-jack@suse.cz
Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Patch series "mm,dax: Fix data corruption due to mmap inconsistency",
v4.
This series fixes data corruption that can happen for DAX mounts when
page faults race with write(2) and as a result page tables get out of
sync with block mappings in the filesystem and thus data seen through
mmap is different from data seen through read(2).
The series passes testing with t_mmap_stale test program from Ross and
also other mmap related tests on DAX filesystem.
This patch (of 4):
dax_invalidate_mapping_entry() currently removes DAX exceptional entries
only if they are clean and unlocked. This is done via:
invalidate_mapping_pages()
invalidate_exceptional_entry()
dax_invalidate_mapping_entry()
However, for page cache pages removed in invalidate_mapping_pages()
there is an additional criteria which is that the page must not be
mapped. This is noted in the comments above invalidate_mapping_pages()
and is checked in invalidate_inode_page().
For DAX entries this means that we can can end up in a situation where a
DAX exceptional entry, either a huge zero page or a regular DAX entry,
could end up mapped but without an associated radix tree entry. This is
inconsistent with the rest of the DAX code and with what happens in the
page cache case.
We aren't able to unmap the DAX exceptional entry because according to
its comments invalidate_mapping_pages() isn't allowed to block, and
unmap_mapping_range() takes a write lock on the mapping->i_mmap_rwsem.
Since we essentially never have unmapped DAX entries to evict from the
radix tree, just remove dax_invalidate_mapping_entry().
Fixes: c6dcf52c23 ("mm: Invalidate DAX radix tree entries only if appropriate")
Link: http://lkml.kernel.org/r/20170510085419.27601-2-jack@suse.cz
Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com>
Signed-off-by: Jan Kara <jack@suse.cz>
Reported-by: Jan Kara <jack@suse.cz>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: <stable@vger.kernel.org> [4.10+]
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Commit 1f5307b1e0 ("mm, vmalloc: properly track vmalloc users") has
pulled asm/pgtable.h include dependency to linux/vmalloc.h and that
turned out to be a bad idea for some architectures. E.g. m68k fails
with
In file included from arch/m68k/include/asm/pgtable_mm.h:145:0,
from arch/m68k/include/asm/pgtable.h:4,
from include/linux/vmalloc.h:9,
from arch/m68k/kernel/module.c:9:
arch/m68k/include/asm/mcf_pgtable.h: In function 'nocache_page':
>> arch/m68k/include/asm/mcf_pgtable.h:339:43: error: 'init_mm' undeclared (first use in this function)
#define pgd_offset_k(address) pgd_offset(&init_mm, address)
as spotted by kernel build bot. nios2 fails for other reason
In file included from include/asm-generic/io.h:767:0,
from arch/nios2/include/asm/io.h:61,
from include/linux/io.h:25,
from arch/nios2/include/asm/pgtable.h:18,
from include/linux/mm.h:70,
from include/linux/pid_namespace.h:6,
from include/linux/ptrace.h:9,
from arch/nios2/include/uapi/asm/elf.h:23,
from arch/nios2/include/asm/elf.h:22,
from include/linux/elf.h:4,
from include/linux/module.h:15,
from init/main.c:16:
include/linux/vmalloc.h: In function '__vmalloc_node_flags':
include/linux/vmalloc.h:99:40: error: 'PAGE_KERNEL' undeclared (first use in this function); did you mean 'GFP_KERNEL'?
which is due to the newly added #include <asm/pgtable.h>, which on nios2
includes <linux/io.h> and thus <asm/io.h> and <asm-generic/io.h> which
again includes <linux/vmalloc.h>.
Tweaking that around just turns out a bigger headache than necessary.
This patch reverts 1f5307b1e0 and reimplements the original fix in a
different way. __vmalloc_node_flags can stay static inline which will
cover vmalloc* functions. We only have one external user
(kvmalloc_node) and we can export __vmalloc_node_flags_caller and
provide the caller directly. This is much simpler and it doesn't really
need any games with header files.
[akpm@linux-foundation.org: coding-style fixes]
[mhocko@kernel.org: revert old comment]
Link: http://lkml.kernel.org/r/20170509211054.GB16325@dhcp22.suse.cz
Fixes: 1f5307b1e0 ("mm, vmalloc: properly track vmalloc users")
Link: http://lkml.kernel.org/r/20170509153702.GR6481@dhcp22.suse.cz
Signed-off-by: Michal Hocko <mhocko@suse.com>
Cc: Tobias Klauser <tklauser@distanz.ch>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
One return case of `__collapse_huge_page_swapin()` does not invoke
tracepoint while every other return case does. This commit adds a
tracepoint invocation for the case.
Link: http://lkml.kernel.org/r/20170507101813.30187-1-sj38.park@gmail.com
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
