Commit Graph

35 Commits

Author SHA1 Message Date
Armen Baloyan
bc123a515c PCI/P2PDMA: Add Intel SkyLake-E to the whitelist
Intel SkyLake-E was successfully tested for p2pdma transactions spanning
over a host bridge and PCI bridge with IOMMU on.  Add it to the whitelist.

Link: https://lore.kernel.org/r/1575669165-31697-1-git-send-email-abaloyan@gigaio.com
Signed-off-by: Armen Baloyan <abaloyan@gigaio.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
2019-12-10 15:15:51 -06:00
Logan Gunthorpe
27235b15e4 PCI/P2PDMA: Update pci_p2pdma_distance_many() documentation
The comment describing pci_p2pdma_distance_many() still referred to
the devices being behind the same root port. This no longer applies
so reword the documentation.

Link: https://lore.kernel.org/r/20190730163545.4915-15-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-15-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:41:59 -05:00
Logan Gunthorpe
5daf40b039 PCI/P2PDMA: Allow IOMMU for host bridge whitelist
Now that we map the requests correctly we can remove the iommu_present()
restriction.

Link: https://lore.kernel.org/r/20190730163545.4915-14-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-14-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:41:54 -05:00
Logan Gunthorpe
5d52e1abcd PCI/P2PDMA: dma_map() requests that traverse the host bridge
Any requests that traverse the host bridge will need to be mapped into the
IOMMU, so call dma_map_sg() inside pci_p2pdma_map_sg() when appropriate.

Similarly, call dma_unmap_sg() inside pci_p2pdma_unmap_sg().

Link: https://lore.kernel.org/r/20190730163545.4915-13-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-13-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:41:48 -05:00
Logan Gunthorpe
110203bee0 PCI/P2PDMA: Store mapping method in an xarray
When upstream_bridge_distance() is called, store the method required to map
the DMA transfers in an xarray so it can be looked up efficiently on the
hot path in pci_p2pdma_map_sg().

Link: https://lore.kernel.org/r/20190730163545.4915-12-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-12-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:41:43 -05:00
Logan Gunthorpe
142c242e6f PCI/P2PDMA: Factor out __pci_p2pdma_map_sg()
Factor out the bus-only mapping into its own static function.  No
functional changes. The original pci_p2pdma_map_sg_attrs() will be used to
decide whether this is an appropriate way to map.

Link: https://lore.kernel.org/r/20190730163545.4915-11-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-11-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:41:35 -05:00
Logan Gunthorpe
7f73eac3a7 PCI/P2PDMA: Introduce pci_p2pdma_unmap_sg()
Add pci_p2pdma_unmap_sg() to the two places that call pci_p2pdma_map_sg().

This is a prep patch to introduce correct mappings for p2pdma transactions
that go through the root complex.

Link: https://lore.kernel.org/r/20190730163545.4915-10-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-10-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:41:26 -05:00
Logan Gunthorpe
2b9f4bb2a4 PCI/P2PDMA: Add attrs argument to pci_p2pdma_map_sg()
This is to match the dma_map_sg() API which this function will have to call
in an future patch.

Add a pci_p2pdma_map_sg_attrs() function and helper to call it with no
attributes just like the dma_map_sg() function.

Link: https://lore.kernel.org/r/20190730163545.4915-9-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-9-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:41:15 -05:00
Logan Gunthorpe
494d63b0d5 PCI/P2PDMA: Whitelist some Intel host bridges
Intel devices do not have good support for P2P requests that span different
host bridges as the transactions will cross the QPI/UPI bus and this does
not perform well.

Therefore, enable support for these devices only if the host bridges match.

Add Intel devices that have been tested and are known to work. There are
likely many others out there that will need to be tested and added.

Link: https://lore.kernel.org/r/20190730163545.4915-8-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-8-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:41:07 -05:00
Logan Gunthorpe
2c84d818ae PCI/P2PDMA: Factor out host_bridge_whitelist()
Push both PCI devices into the whitelist checking function seeing some
hardware will require us ensuring they are on the same host bridge.

At the same time we rename root_complex_whitelist() to
host_bridge_whitelist() to match the terminology used in the code.

Link: https://lore.kernel.org/r/20190730163545.4915-7-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-7-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:40:59 -05:00
Logan Gunthorpe
e2cbfbf789 PCI/P2PDMA: Apply host bridge whitelist for ACS
When a P2PDMA transfer is rejected due to ACS being set, we can also check
the whitelist and allow the transactions.

Do this by pushing the whitelist check into the upstream_bridge_distance()
function.

Link: https://lore.kernel.org/r/20190730163545.4915-6-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-6-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:40:50 -05:00
Logan Gunthorpe
c6bfaeb573 PCI/P2PDMA: Factor out __upstream_bridge_distance()
This is a prep patch to create a second level helper. There are no
functional changes.

The root complex whitelist code will be moved into this function in a
subsequent patch.

Link: https://lore.kernel.org/r/20190730163545.4915-5-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-5-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:40:38 -05:00
Logan Gunthorpe
72583084e3 PCI/P2PDMA: Add constants for map type results to upstream_bridge_distance()
Add constant flags to indicate how two devices will be mapped or if they
are unsupported. upstream_bridge_distance() will now return the
mapping type and the distance in a passed-by-reference argument.

This helps annotate the code better, but the main reason is so we can use
the information to store the required mapping method in an xarray.

Link: https://lore.kernel.org/r/20190730163545.4915-4-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-4-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:40:23 -05:00
Logan Gunthorpe
0afea38143 PCI/P2PDMA: Add provider's pci_dev to pci_p2pdma_pagemap struct
The provider will be needed to figure out how to map a device.

Link: https://lore.kernel.org/r/20190730163545.4915-3-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-3-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:40:05 -05:00
Logan Gunthorpe
a6e6fe6549 PCI/P2PDMA: Introduce private pagemap structure
Move the PCI bus offset from the generic dev_pagemap structure to a
specific pci_p2pdma_pagemap structure.

This structure will grow in subsequent patches.

Link: https://lore.kernel.org/r/20190730163545.4915-2-logang@deltatee.com
Link: https://lore.kernel.org/r/20190812173048.9186-2-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-08-16 08:38:52 -05:00
Linus Torvalds
fb4da215ed pci-v5.3-changes
-----BEGIN PGP SIGNATURE-----
 
 iQJIBAABCgAyFiEEgMe7l+5h9hnxdsnuWYigwDrT+vwFAl0siFoUHGJoZWxnYWFz
 QGdvb2dsZS5jb20ACgkQWYigwDrT+vzi9A//S4jRyyZrgUr88Az0GbgMhE4b3yqc
 uL7om/Sf+443gG6C+aKkZSM/IE9hrbyIKuYq7GGxDkzZ/HkucZo2yIuAHkPgG4ik
 QQYJ8fJsmMq1bUht87c1ZZwGP0++Deq/Ns2+VNy/WBYqKLulnV0DvEEaJgPs9C5D
 ppwccGdo6UghiujBTpE4ddUBjFjjURWqT6wSnMRDQ4EGwfUhG0MWwwHKI4hbBuaL
 N6refuggdYyUUX5FeUOHa6VF6uTnSSAQ75k+40n4nljdayqoumHLskst77o9q5ZI
 oXjdpwgmuEqYhfp03HEA4Xo/bBxiRj76NuTiEMKvPokxjpanwbLrdV0GhF0OIlM0
 rp1NOI1w+vppFrU+rc2gtq+7hYXFmvdhjS29hFLeD91PP36N5d29jW5NVFpm7GCm
 n4TMGAOsu8RB+bNua6ZbZVcDk2EnPgQeIcM0ZPoBtPK19Fg/rScdEU4u/aFE1Y0Q
 C+Ks7D1qCvFpHzl/xAg0oo9v/jFsWef3qnQWOzot964Zz4W4NSVvB9Ox6Vbfj6C4
 v331LJmlPxG8fxBNA3q28FrTxcG1NW6sgo3WY9VoSp/vc0aqaPKhm7sbraTt5IrI
 TwqA/WhnAHv90MQCGFcofANyYTkjPkKk2QBFK6b0suoAmVdwVWWELi1WaZ+HdvgQ
 JP7YpmC2cXcQBPk=
 =ZGxL
 -----END PGP SIGNATURE-----

Merge tag 'pci-v5.3-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci

Pull PCI updates from Bjorn Helgaas:
 "Enumeration changes:

   - Evaluate PCI Boot Configuration _DSM to learn if firmware wants us
     to preserve its resource assignments (Benjamin Herrenschmidt)

   - Simplify resource distribution (Nicholas Johnson)

   - Decode 32 GT/s link speed (Gustavo Pimentel)

  Virtualization:

   - Fix incorrect caching of VF config space size (Alex Williamson)

   - Fix VF driver probing sysfs knobs (Alex Williamson)

  Peer-to-peer DMA:

   - Fix dma_virt_ops check (Logan Gunthorpe)

  Altera host bridge driver:

   - Allow building as module (Ley Foon Tan)

  Armada 8K host bridge driver:

   - add PHYs support (Miquel Raynal)

  DesignWare host bridge driver:

   - Export APIs to support removable loadable module (Vidya Sagar)

   - Enable Relaxed Ordering erratum workaround only on Tegra20 &
     Tegra30 (Vidya Sagar)

  Hyper-V host bridge driver:

   - Fix use-after-free in eject (Dexuan Cui)

  Mobiveil host bridge driver:

   - Clean up and fix many issues, including non-identify mapped
     windows, 64-bit windows, multi-MSI, class code, INTx clearing (Hou
     Zhiqiang)

  Qualcomm host bridge driver:

   - Use clk bulk API for 2.4.0 controllers (Bjorn Andersson)

   - Add QCS404 support (Bjorn Andersson)

   - Assert PERST for at least 100ms (Niklas Cassel)

  R-Car host bridge driver:

   - Add r8a774a1 DT support (Biju Das)

  Tegra host bridge driver:

   - Add support for Gen2, opportunistic UpdateFC and ACK (PCIe protocol
     details) AER, GPIO-based PERST# (Manikanta Maddireddy)

   - Fix many issues, including power-on failure cases, interrupt
     masking in suspend, UPHY settings, AFI dynamic clock gating,
     pending DLL transactions (Manikanta Maddireddy)

  Xilinx host bridge driver:

   - Fix NWL Multi-MSI programming (Bharat Kumar Gogada)

  Endpoint support:

   - Fix 64bit BAR support (Alan Mikhak)

   - Fix pcitest build issues (Alan Mikhak, Andy Shevchenko)

  Bug fixes:

   - Fix NVIDIA GPU multi-function power dependencies (Abhishek Sahu)

   - Fix NVIDIA GPU HDA enablement issue (Lukas Wunner)

   - Ignore lockdep for sysfs "remove" (Marek Vasut)

  Misc:

   - Convert docs to reST (Changbin Du, Mauro Carvalho Chehab)"

* tag 'pci-v5.3-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (107 commits)
  PCI: Enable NVIDIA HDA controllers
  tools: PCI: Fix installation when `make tools/pci_install`
  PCI: dwc: pci-dra7xx: Fix compilation when !CONFIG_GPIOLIB
  PCI: Fix typos and whitespace errors
  PCI: mobiveil: Fix INTx interrupt clearing in mobiveil_pcie_isr()
  PCI: mobiveil: Fix infinite-loop in the INTx handling function
  PCI: mobiveil: Move PCIe PIO enablement out of inbound window routine
  PCI: mobiveil: Add upper 32-bit PCI base address setup in inbound window
  PCI: mobiveil: Add upper 32-bit CPU base address setup in outbound window
  PCI: mobiveil: Mask out hardcoded bits in inbound/outbound windows setup
  PCI: mobiveil: Clear the control fields before updating it
  PCI: mobiveil: Add configured inbound windows counter
  PCI: mobiveil: Fix the valid check for inbound and outbound windows
  PCI: mobiveil: Clean-up program_{ib/ob}_windows()
  PCI: mobiveil: Remove an unnecessary return value check
  PCI: mobiveil: Fix error return values
  PCI: mobiveil: Refactor the MEM/IO outbound window initialization
  PCI: mobiveil: Make some register updates more readable
  PCI: mobiveil: Reformat the code for readability
  dt-bindings: PCI: mobiveil: Change gpio_slave and apb_csr to optional
  ...
2019-07-15 20:44:49 -07:00
Bjorn Helgaas
7b4b0f6b34 Merge branch 'pci/trivial'
- Fix typos and whitespace errors (Bjorn Helgaas)

* pci/trivial:
  PCI: Fix typos and whitespace errors
2019-07-12 17:08:41 -05:00
Bjorn Helgaas
f6b6aefee7 PCI: Fix typos and whitespace errors
Fix typos in drivers/pci.  Comment and whitespace changes only.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
2019-07-09 07:24:53 -05:00
Logan Gunthorpe
9c002bb66f PCI/P2PDMA: Fix missing check for dma_virt_ops
Drivers that use dma_virt_ops were meant to be rejected when testing
compatibility for P2PDMA.

This check got inadvertently dropped in one of the later versions of the
original patchset, so add it back.

Fixes: 52916982af ("PCI/P2PDMA: Support peer-to-peer memory")
Link: https://lore.kernel.org/r/20190702173544.21950-1-logang@deltatee.com
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-07-02 18:35:15 -05:00
Christoph Hellwig
d0b3517dbc PCI/P2PDMA: use the dev_pagemap internal refcount
The functionality is identical to the one currently open coded in
p2pdma.c.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Tested-by: Logan Gunthorpe <logang@deltatee.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-07-02 14:32:44 -03:00
Christoph Hellwig
d8668bb045 memremap: pass a struct dev_pagemap to ->kill and ->cleanup
Passing the actual typed structure leads to more understandable code
vs just passing the ref member.

Reported-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Reviewed-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Tested-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-07-02 14:32:44 -03:00
Christoph Hellwig
1e240e8d4a memremap: move dev_pagemap callbacks into a separate structure
The dev_pagemap is a growing too many callbacks.  Move them into a
separate ops structure so that they are not duplicated for multiple
instances, and an attacker can't easily overwrite them.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Reviewed-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: Dan Williams <dan.j.williams@intel.com>
Tested-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
2019-07-02 14:32:44 -03:00
Linus Torvalds
b253d5f3ec pci-v5.2-fixes-1
-----BEGIN PGP SIGNATURE-----
 
 iQJIBAABCgAyFiEEgMe7l+5h9hnxdsnuWYigwDrT+vwFAl0OTC0UHGJoZWxnYWFz
 QGdvb2dsZS5jb20ACgkQWYigwDrT+vxBkQ//Zye+wY3Md9M3TWHYjj73jeJTmfb/
 BQgWU2OsOvGWDgQe7gX2c5sjkJe1S450Mf9CvRYu77z1SHgT/2E25yY3OxV1wNMS
 UX1xjm91N1/KBnPj2L6ks5exequobVdAkkhUF0WsRB8L09KHe+E/eg8hKNhRA5jK
 mqNCmRox20LjbkKpKJF2p20ynU8+psFoM3Enm1JRo5UprgXsfFwBJaB75qQBGhN8
 SaRBQVMP2vJoghRVeofj2y/cZWSIKEcZ/dOY+q6MMzd2hsRxjiuqRLrm4f6MStdV
 W+0qr6qd0V6BzxMO+NrCrrhWrkjEb8cqB0F8V/hw3xU8G/17CBEk34HLrqWi2+3D
 //puT7TjXA8t/awkuz+wH2saDldZU4BfeDgpEriop//jQa30EhXM0RLiUBZofKfS
 U88qkd4N/CqPbScTe71ve5pUW5WH5kdcBYWHTN5venEW3sxCR13vFtlfDheki/Mc
 C865E7+ZEep8FhakhGrwiS6MjQrlF1Mzq3BxGviED0Cw92Rz3SEShdp+C0Qk/Av6
 5OYUaDfOw6tx92hBz6DtlbTHUNYbXCMv9aXCR2ju1DjDrIkFeIIr8cJwKI64f4LZ
 EJXIQrEKVWD7QOLe1ebRBlZKV+mEN0q9ZTII2waMcUfZ7GXLLueqYkCKRC8hd805
 +5sZtKspodBpl04=
 =0Qnd
 -----END PGP SIGNATURE-----

Merge tag 'pci-v5.2-fixes-1' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci

Pull PCI fix from Bjorn Helgaas:
 "If an IOMMU is present, ignore the P2PDMA whitelist we added for v5.2
  because we don't yet know how to support P2PDMA in that case (Logan
  Gunthorpe)"

* tag 'pci-v5.2-fixes-1' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci:
  PCI/P2PDMA: Ignore root complex whitelist when an IOMMU is present
2019-06-22 09:42:29 -07:00
Logan Gunthorpe
6dbbd053e6 PCI/P2PDMA: Ignore root complex whitelist when an IOMMU is present
Presently, there is no path to DMA map P2PDMA memory, so if a TLP targeting
this memory hits the root complex and an IOMMU is present, the IOMMU will
reject the transaction, even if the RC would support P2PDMA.

So until the kernel knows to map these DMA addresses in the IOMMU, we
should not enable the whitelist when an IOMMU is present.

Link: https://lore.kernel.org/linux-pci/20190522201252.2997-1-logang@deltatee.com/
Fixes: 0f97da8310 ("PCI/P2PDMA: Allow P2P DMA between any devices under AMD ZEN Root Complex")
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Christoph Hellwig <hch@lst.de>
2019-06-19 16:43:42 -05:00
Dan Williams
50f44ee724 mm/devm_memremap_pages: fix final page put race
Logan noticed that devm_memremap_pages_release() kills the percpu_ref
drops all the page references that were acquired at init and then
immediately proceeds to unplug, arch_remove_memory(), the backing pages
for the pagemap.  If for some reason device shutdown actually collides
with a busy / elevated-ref-count page then arch_remove_memory() should
be deferred until after that reference is dropped.

As it stands the "wait for last page ref drop" happens *after*
devm_memremap_pages_release() returns, which is obviously too late and
can lead to crashes.

Fix this situation by assigning the responsibility to wait for the
percpu_ref to go idle to devm_memremap_pages() with a new ->cleanup()
callback.  Implement the new cleanup callback for all
devm_memremap_pages() users: pmem, devdax, hmm, and p2pdma.

Link: http://lkml.kernel.org/r/155727339156.292046.5432007428235387859.stgit@dwillia2-desk3.amr.corp.intel.com
Fixes: 41e94a8513 ("add devm_memremap_pages")
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Reported-by: Logan Gunthorpe <logang@deltatee.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: "Jérôme Glisse" <jglisse@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-06-13 17:34:56 -10:00
Dan Williams
1570175abd PCI/P2PDMA: track pgmap references per resource, not globally
In preparation for fixing a race between devm_memremap_pages_release()
and the final put of a page from the device-page-map, allocate a
percpu-ref per p2pdma resource mapping.

Link: http://lkml.kernel.org/r/155727338646.292046.9922678317501435597.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: "Jérôme Glisse" <jglisse@redhat.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-06-13 17:34:56 -10:00
Dan Williams
e615a19121 PCI/P2PDMA: fix the gen_pool_add_virt() failure path
The pci_p2pdma_add_resource() implementation immediately frees the pgmap
if gen_pool_add_virt() fails.  However, that means that when @dev
triggers a devres release devm_memremap_pages_release() will crash
trying to access the freed @pgmap.

Use the new devm_memunmap_pages() to manually free the mapping in the
error path.

Link: http://lkml.kernel.org/r/155727337603.292046.13101332703665246702.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Fixes: 52916982af ("PCI/P2PDMA: Support peer-to-peer memory")
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: "Jérôme Glisse" <jglisse@redhat.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2019-06-13 17:34:56 -10:00
Christian König
0f97da8310 PCI/P2PDMA: Allow P2P DMA between any devices under AMD ZEN Root Complex
The PCI specs say that peer-to-peer DMA should be supported between any two
devices that have a common upstream PCI-to-PCI bridge.  But devices under
different Root Ports don't share a common upstream bridge, and PCIe r4.0,
sec 1.3.1, says routing peer-to-peer transactions between Root Ports in the
same Root Complex is optional.

Many Root Complexes, including AMD ZEN, *do* support peer-to-peer DMA even
between Root Ports.  Add a whitelist and allow peer-to-peer DMA if both
participants are attached to a Root Complex known to support it.

Link: https://lore.kernel.org/linux-pci/20190418115859.2394-1-christian.koenig@amd.com
Signed-off-by: Christian König <christian.koenig@amd.com>
[bhelgaas: changelog]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
2019-05-01 16:53:39 -05:00
Linus Torvalds
926b02d3eb pci-v4.21-changes
-----BEGIN PGP SIGNATURE-----
 
 iQJIBAABCgAyFiEEgMe7l+5h9hnxdsnuWYigwDrT+vwFAlwtMCIUHGJoZWxnYWFz
 QGdvb2dsZS5jb20ACgkQWYigwDrT+vwQUQ/+P5/VDpo4abjudGO2c7FU1bJOwvfN
 cfV5dvDCw0kpx0Em5SmnpAD7Punllxxvb/04K75lqarGyx/Txqaw+lbIF+qSj6my
 GsQ16Iy8T48x5hr+Pf6vTh1eE+NaAVZfOPDOt7CyTNAgwfzHeVNyfNvz7pfKTIIJ
 Mk/jRE4kkeWo60jsY5p3sFo3OVOxBOsRdN+2sruaQuWFXrKHLyNDR+7Z9ZPxubFk
 cCO/TYPhNXmmKhCAR4V/rGiqz9OL2wyFixGhGhmD3tnC9nAb/wTMzjARsyBopBPi
 b/KkR2eLFEyXN0HJrwqxiURo4J3nveAYEuNXH5KjRBQZnoBCGSCIlqFhlrp9vdBk
 B4KIdT8h/M6LsVGeVSEIxIEXCp67YE31kxraFrk4Vsggdh2TFQ0llh1sajj8IFJB
 XekutedAOlTSOaM1/jvVPUJYg04X90bp3uXn3IU45XlQ8nBOG3immFVITRLkvd3w
 ywH+SEdeZAhWl3RGy8SHhqdeCJ7nNQbcRaRJ5CoWJBDNJTBGF1X+zJD2Swi6H9vA
 nWGNRlb3CPPIMPF127nADnOE7Cj2FlpAEIEu52HpcpIrhEdrGvLkGeQfgdWBjbyU
 aHwC04bLWnvsA9SEFVnuMIBaFQmJ1RuaWAHdtscyyO2uoeCtN8Aa+BX6jXFbVZQN
 9eFzpiv0kUiXlAQ=
 =g1ia
 -----END PGP SIGNATURE-----

Merge tag 'pci-v4.21-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci

Pull PCI updates from Bjorn Helgaas:

 - Remove unused lists from ASPM pcie_link_state (Frederick Lawler)

 - Fix Broadcom CNB20LE host bridge unintended sign extension (Colin Ian
   King)

 - Expand Kconfig "PF" acronyms (Randy Dunlap)

 - Update MAINTAINERS for arch/x86/kernel/early-quirks.c (Bjorn Helgaas)

 - Add missing include to drivers/pci.h (Alexandru Gagniuc)

 - Override Synopsys USB 3.x HAPS device class so dwc3-haps can claim it
   instead of xhci (Thinh Nguyen)

 - Clean up P2PDMA documentation (Randy Dunlap)

 - Allow runtime PM even if driver doesn't supply callbacks (Jarkko
   Nikula)

 - Remove status check after submitting Switchtec MRPC Firmware Download
   commands to avoid Completion Timeouts (Kelvin Cao)

 - Set Switchtec coherent DMA mask to allow 64-bit DMA (Boris Glimcher)

 - Fix Switchtec SWITCHTEC_IOCTL_EVENT_IDX_ALL flag overwrite issue
   (Joey Zhang)

 - Enable write combining for Switchtec MRPC Input buffers (Kelvin Cao)

 - Add Switchtec MRPC DMA mode support (Wesley Sheng)

 - Skip VF scanning on powerpc, which does this in firmware (Sebastian
   Ott)

 - Add Amlogic Meson PCIe controller driver and DT bindings (Yue Wang)

 - Constify histb dw_pcie_host_ops structure (Julia Lawall)

 - Support multiple power domains for imx6 (Leonard Crestez)

 - Constify layerscape driver data (Stefan Agner)

 - Update imx6 Kconfig to allow imx6 PCIe in imx7 kernel (Trent Piepho)

 - Support armada8k GPIO reset (Baruch Siach)

 - Support suspend/resume support on imx6 (Leonard Crestez)

 - Don't hard-code DesignWare DBI/ATU offst (Stephen Warren)

 - Skip i.MX6 PHY setup on i.MX7D (Andrey Smirnov)

 - Remove Jianguo Sun from HiSilicon STB maintainers (Lorenzo Pieralisi)

 - Mask DesignWare interrupts instead of disabling them to avoid lost
   interrupts (Marc Zyngier)

 - Add locking when acking DesignWare interrupts (Marc Zyngier)

 - Ack DesignWare interrupts in the proper callbacks (Marc Zyngier)

 - Use devm resource parser in mediatek (Honghui Zhang)

 - Remove unused mediatek "num-lanes" DT property (Honghui Zhang)

 - Add UniPhier PCIe controller driver and DT bindings (Kunihiko
   Hayashi)

 - Enable MSI for imx6 downstream components (Richard Zhu)

* tag 'pci-v4.21-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (40 commits)
  PCI: imx: Enable MSI from downstream components
  s390/pci: skip VF scanning
  PCI/IOV: Add flag so platforms can skip VF scanning
  PCI/IOV: Factor out sriov_add_vfs()
  PCI: uniphier: Add UniPhier PCIe host controller support
  dt-bindings: PCI: Add UniPhier PCIe host controller description
  PCI: amlogic: Add the Amlogic Meson PCIe controller driver
  dt-bindings: PCI: meson: add DT bindings for Amlogic Meson PCIe controller
  arm64: dts: mt7622: Remove un-used property for PCIe
  arm: dts: mt7623: Remove un-used property for PCIe
  dt-bindings: PCI: MediaTek: Remove un-used property
  PCI: mediatek: Remove un-used variant in struct mtk_pcie_port
  MAINTAINERS: Remove Jianguo Sun from HiSilicon STB DWC entry
  PCI: dwc: Don't hard-code DBI/ATU offset
  PCI: imx: Add imx6sx suspend/resume support
  PCI: armada8k: Add support for gpio controlled reset signal
  PCI: dwc: Adjust Kconfig to allow IMX6 PCIe host on IMX7
  PCI: dwc: layerscape: Constify driver data
  PCI: imx: Add multi-pd support
  PCI: Override Synopsys USB 3.x HAPS device class
  ...
2019-01-05 17:57:34 -08:00
Dan Williams
02917e9f86 mm, hmm: mark hmm_devmem_{add, add_resource} EXPORT_SYMBOL_GPL
At Maintainer Summit, Greg brought up a topic I proposed around
EXPORT_SYMBOL_GPL usage.  The motivation was considerations for when
EXPORT_SYMBOL_GPL is warranted and the criteria for taking the exceptional
step of reclassifying an existing export.  Specifically, I wanted to make
the case that although the line is fuzzy and hard to specify in abstract
terms, it is nonetheless clear that devm_memremap_pages() and HMM
(Heterogeneous Memory Management) have crossed it.  The
devm_memremap_pages() facility should have been EXPORT_SYMBOL_GPL from the
beginning, and HMM as a derivative of that functionality should have
naturally picked up that designation as well.

Contrary to typical rules, the HMM infrastructure was merged upstream with
zero in-tree consumers.  There was a promise at the time that those users
would be merged "soon", but it has been over a year with no drivers
arriving.  While the Nouveau driver is about to belatedly make good on
that promise it is clear that HMM was targeted first and foremost at an
out-of-tree consumer.

HMM is derived from devm_memremap_pages(), a facility Christoph and I
spearheaded to support persistent memory.  It combines a device lifetime
model with a dynamically created 'struct page' / memmap array for any
physical address range.  It enables coordination and control of the many
code paths in the kernel built to interact with memory via 'struct page'
objects.  With HMM the integration goes even deeper by allowing device
drivers to hook and manipulate page fault and page free events.

One interpretation of when EXPORT_SYMBOL is suitable is when it is
exporting stable and generic leaf functionality.  The
devm_memremap_pages() facility continues to see expanding use cases,
peer-to-peer DMA being the most recent, with no clear end date when it
will stop attracting reworks and semantic changes.  It is not suitable to
export devm_memremap_pages() as a stable 3rd party driver API due to the
fact that it is still changing and manipulates core behavior.  Moreover,
it is not in the best interest of the long term development of the core
memory management subsystem to permit any external driver to effectively
define its own system-wide memory management policies with no
encouragement to engage with upstream.

I am also concerned that HMM was designed in a way to minimize further
engagement with the core-MM.  That, with these hooks in place,
device-drivers are free to implement their own policies without much
consideration for whether and how the core-MM could grow to meet that
need.  Going forward not only should HMM be EXPORT_SYMBOL_GPL, but the
core-MM should be allowed the opportunity and stimulus to change and
address these new use cases as first class functionality.

Original changelog:

hmm_devmem_add(), and hmm_devmem_add_resource() duplicated
devm_memremap_pages() and are now simple now wrappers around the core
facility to inject a dev_pagemap instance into the global pgmap_radix and
hook page-idle events.  The devm_memremap_pages() interface is base
infrastructure for HMM.  HMM has more and deeper ties into the kernel
memory management implementation than base ZONE_DEVICE which is itself a
EXPORT_SYMBOL_GPL facility.

Originally, the HMM page structure creation routines copied the
devm_memremap_pages() code and reused ZONE_DEVICE.  A cleanup to unify the
implementations was discussed during the initial review:
http://lkml.iu.edu/hypermail/linux/kernel/1701.2/00812.html Recent work to
extend devm_memremap_pages() for the peer-to-peer-DMA facility enabled
this cleanup to move forward.

In addition to the integration with devm_memremap_pages() HMM depends on
other GPL-only symbols:

    mmu_notifier_unregister_no_release
    percpu_ref
    region_intersects
    __class_create

It goes further to consume / indirectly expose functionality that is not
exported to any other driver:

    alloc_pages_vma
    walk_page_range

HMM is derived from devm_memremap_pages(), and extends deep core-kernel
fundamentals. Similar to devm_memremap_pages(), mark its entry points
EXPORT_SYMBOL_GPL().

[logang@deltatee.com: PCI/P2PDMA: match interface changes to devm_memremap_pages()]
  Link: http://lkml.kernel.org/r/20181130225911.2900-1-logang@deltatee.com
Link: http://lkml.kernel.org/r/154275560565.76910.15919297436557795278.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Cc: Logan Gunthorpe <logang@deltatee.com>
Cc: "Jérôme Glisse" <jglisse@redhat.com>
Cc: Balbir Singh <bsingharora@gmail.com>,
Cc: Michal Hocko <mhocko@suse.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-12-28 12:11:48 -08:00
Randy Dunlap
fcf9ab35ba PCI/P2PDMA: Clean up documentation and kernel-doc
Fix typos, spellos, and grammar in p2pdma.rst and p2pdma.c.

Fix return value(s) in function pci_p2pmem_alloc_sgl().

Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Logan Gunthorpe <logang@deltatee.com>
Cc: Jonathan Corbet <corbet@lwn.net>
2018-12-11 07:53:07 -06:00
Logan Gunthorpe
2d7bc010f4 PCI/P2PDMA: Introduce configfs/sysfs enable attribute helpers
Users of the P2PDMA infrastructure will typically need a way for the user
to tell the kernel to use P2P resources.  Typically this will be a simple
on/off boolean operation but sometimes it may be desirable for the user to
specify the exact device to use for the P2P operation.

Add new helpers for attributes which take a boolean or a PCI device.  Any
boolean as accepted by strtobool() turn P2P on or off (such as 'y', 'n',
'1', '0', etc).  Specifying a full PCI device name/BDF will select the
specific device.

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2018-10-17 12:18:17 -05:00
Logan Gunthorpe
977196b8c5 PCI/P2PDMA: Add PCI p2pmem DMA mappings to adjust the bus offset
The DMA address used when mapping PCI P2P memory must be the PCI bus
address.  Thus, introduce pci_p2pmem_map_sg() to map the correct addresses
when using P2P memory.

Memory mapped in this way does not need to be unmapped and thus if we
provided pci_p2pmem_unmap_sg() it would be empty.  This breaks the expected
balance between map/unmap but was left out as an empty function doesn't
really provide any benefit.  In the future, if this call becomes necessary
it can be added without much difficulty.

For this, we assume that an SGL passed to these functions contain all P2P
memory or no P2P memory.

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2018-10-17 12:18:16 -05:00
Logan Gunthorpe
cbb8ca69fc PCI/P2PDMA: Add sysfs group to display p2pmem stats
Add a sysfs group to display statistics about P2P memory that is registered
in each PCI device.

Attributes in the group display the total amount of P2P memory, the amount
available and whether it is published or not.

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2018-10-17 12:18:15 -05:00
Logan Gunthorpe
52916982af PCI/P2PDMA: Support peer-to-peer memory
Some PCI devices may have memory mapped in a BAR space that's intended for
use in peer-to-peer transactions.  To enable such transactions the memory
must be registered with ZONE_DEVICE pages so it can be used by DMA
interfaces in existing drivers.

Add an interface for other subsystems to find and allocate chunks of P2P
memory as necessary to facilitate transfers between two PCI peers:

  struct pci_dev *pci_p2pmem_find[_many]();
  int pci_p2pdma_distance[_many]();
  void *pci_alloc_p2pmem();

The new interface requires a driver to collect a list of client devices
involved in the transaction then call pci_p2pmem_find() to obtain any
suitable P2P memory.  Alternatively, if the caller knows a device which
provides P2P memory, they can use pci_p2pdma_distance() to determine if it
is usable.  With a suitable p2pmem device, memory can then be allocated
with pci_alloc_p2pmem() for use in DMA transactions.

Depending on hardware, using peer-to-peer memory may reduce the bandwidth
of the transfer but can significantly reduce pressure on system memory.
This may be desirable in many cases: for example a system could be designed
with a small CPU connected to a PCIe switch by a small number of lanes
which would maximize the number of lanes available to connect to NVMe
devices.

The code is designed to only utilize the p2pmem device if all the devices
involved in a transfer are behind the same PCI bridge.  This is because we
have no way of knowing whether peer-to-peer routing between PCIe Root Ports
is supported (PCIe r4.0, sec 1.3.1).  Additionally, the benefits of P2P
transfers that go through the RC is limited to only reducing DRAM usage
and, in some cases, coding convenience.  The PCI-SIG may be exploring
adding a new capability bit to advertise whether this is possible for
future hardware.

This commit includes significant rework and feedback from Christoph
Hellwig.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
[bhelgaas: fold in fix from Keith Busch <keith.busch@intel.com>:
https://lore.kernel.org/linux-pci/20181012155920.15418-1-keith.busch@intel.com,
to address comment from Dan Carpenter <dan.carpenter@oracle.com>, fold in
https://lore.kernel.org/linux-pci/20181017160510.17926-1-logang@deltatee.com]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2018-10-10 14:00:54 -05:00