Commit Graph

214 Commits

Author SHA1 Message Date
Linus Torvalds
e0fb1b3639 IOMMU Updates for Linux v4.7
The updates include:
 
 	* Rate limiting for the VT-d fault handler
 
 	* Remove statistics code from the AMD IOMMU driver. It is unused
 	  and should be replaced by something more generic if needed
 
 	* Per-domain pagesize-bitmaps in IOMMU core code to support
 	  systems with different types of IOMMUs
 
 	* Support for ACPI devices in the AMD IOMMU driver
 
 	* 4GB mode support for Mediatek IOMMU driver
 
 	* ARM-SMMU updates from Will Deacon:
 
 		- Support for 64k pages with SMMUv1 implementations
 		  (e.g MMU-401)
 
 		- Remove open-coded 64-bit MMIO accessors
 
 		- Initial support for 16-bit VMIDs, as supported by some
 		  ThunderX SMMU implementations
 
 		- A couple of errata workarounds for silicon in the
 		  field
 
 	* Various fixes here and there
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABAgAGBQJXPeM1AAoJECvwRC2XARrjA2QP/2Cz+pVkpQCuvhAse57eN4rB
 wWXKTjqSFZ4PcA3Vu5yvX6XMv15g46xXFJAhf2spE5//8+xgFfYBgkBRpnqu1brw
 SL6f8A912MnfMRgWqcdKkJNeHbiN0kOvcIQv1J8GNfciqMiyYFhiLP6fFiRmWR/F
 XDBjUeFZ5+Uwf1BAGqw0cVPexeakEbsLHUGqxFsh5g2T4i43aHzO2HJT3IdwWHDt
 F2ivs8gNFGBeJEyzhW8TD0rOEEyHAnM3N18qPEU9+dD0UmjnTQPymEZSbsGW5d4j
 Cn40QYlA+Zmbwgx6LaDVChzQyRJu6O3uvFThyRviiYKCri/Nc9cUT4vHsFGU4MXb
 1d3bqrgzaw7vw31BN7S1Py3MV+WpVnEYjFm2O+hW28OjtSpm6ZvbI8wc0rF4UT/I
 KgL0gSeA8tp25uVISM+ktpIrObYsAcoCz8nvurpDv2AGkKRzhyoSze0Jg43rusD8
 BH7iFWu1LRPlulTGlrHMtNmbZeEApUPbObcQAOcrBOj9vjuFaZ8qduZmB+hwS2iV
 p9atn+54LmGO0LuzqsGrhApIeXTeTZSrGyjlbUADWBJlTw8Xyk/CR39Wf3m/Xmpr
 DiJ/5oa8SKQtNbwvbScn1+sInNWP/pH/JgnRO3Yvqth8HWF/DlpzNj5XxAB8czwr
 qjk9WjpEXun50ocPFQeS
 =jpPD
 -----END PGP SIGNATURE-----

Merge tag 'iommu-updates-v4.7' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Pull IOMMU updates from Joerg Roedel:
 "The updates include:

   - rate limiting for the VT-d fault handler

   - remove statistics code from the AMD IOMMU driver.  It is unused and
     should be replaced by something more generic if needed

   - per-domain pagesize-bitmaps in IOMMU core code to support systems
     with different types of IOMMUs

   - support for ACPI devices in the AMD IOMMU driver

   - 4GB mode support for Mediatek IOMMU driver

   - ARM-SMMU updates from Will Deacon:
      - support for 64k pages with SMMUv1 implementations (e.g MMU-401)
      - remove open-coded 64-bit MMIO accessors
      - initial support for 16-bit VMIDs, as supported by some ThunderX
        SMMU implementations
      - a couple of errata workarounds for silicon in the field

   - various fixes here and there"

* tag 'iommu-updates-v4.7' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (44 commits)
  iommu/arm-smmu: Use per-domain page sizes.
  iommu/amd: Remove statistics code
  iommu/dma: Finish optimising higher-order allocations
  iommu: Allow selecting page sizes per domain
  iommu: of: enforce const-ness of struct iommu_ops
  iommu: remove unused priv field from struct iommu_ops
  iommu/dma: Implement scatterlist segment merging
  iommu/arm-smmu: Clear cache lock bit of ACR
  iommu/arm-smmu: Support SMMUv1 64KB supplement
  iommu/arm-smmu: Decouple context format from kernel config
  iommu/arm-smmu: Tidy up 64-bit/atomic I/O accesses
  io-64-nonatomic: Add relaxed accessor variants
  iommu/arm-smmu: Work around MMU-500 prefetch errata
  iommu/arm-smmu: Convert ThunderX workaround to new method
  iommu/arm-smmu: Differentiate specific implementations
  iommu/arm-smmu: Workaround for ThunderX erratum #27704
  iommu/arm-smmu: Add support for 16 bit VMID
  iommu/amd: Move get_device_id() and friends to beginning of file
  iommu/amd: Don't use IS_ERR_VALUE to check integer values
  iommu/amd: Signedness bug in acpihid_device_group()
  ...
2016-05-19 17:07:04 -07:00
Linus Torvalds
7afd16f882 PCI changes for the v4.7 merge window:
Enumeration
     Refine PCI support check in pcibios_init() (Adrian-Ken Rueegsegger)
     Provide common functions for ECAM mapping (Jayachandran C)
     Allow all PCIe services on non-ACPI host bridges (Jon Derrick)
     Remove return values from pcie_port_platform_notify() and relatives (Jon Derrick)
     Widen portdrv service type from 4 bits to 8 bits (Keith Busch)
     Add Downstream Port Containment portdrv service type (Keith Busch)
     Add Downstream Port Containment driver (Keith Busch)
 
   Resource management
     Identify Enhanced Allocation (EA) BAR Equivalent resources in sysfs (Alex Williamson)
     Supply CPU physical address (not bus address) to iomem_is_exclusive() (Bjorn Helgaas)
     alpha: Call iomem_is_exclusive() for IORESOURCE_MEM, but not IORESOURCE_IO (Bjorn Helgaas)
     Mark Broadwell-EP Home Agent 1 as having non-compliant BARs (Prarit Bhargava)
     Disable all BAR sizing for devices with non-compliant BARs (Prarit Bhargava)
     Move PCI I/O space management from OF to PCI core code (Tomasz Nowicki)
 
   PCI device hotplug
     acpiphp_ibm: Avoid uninitialized variable reference (Dan Carpenter)
     Use cached copy of PCI_EXP_SLTCAP_HPC bit (Lukas Wunner)
 
   Virtualization
     Mark Intel i40e NIC INTx masking as broken (Alex Williamson)
     Reverse standard ACS vs device-specific ACS enabling (Alex Williamson)
     Work around Intel Sunrise Point PCH incorrect ACS capability (Alex Williamson)
 
   IOMMU
     Add pci_add_dma_alias() to abstract implementation (Bjorn Helgaas)
     Move informational printk to pci_add_dma_alias() (Bjorn Helgaas)
     Add support for multiple DMA aliases (Jacek Lawrynowicz)
     Add DMA alias quirk for mic_x200_dma (Jacek Lawrynowicz)
 
   Thunderbolt
     Fix double free of drom buffer (Andreas Noever)
     Add Intel Thunderbolt device IDs (Lukas Wunner)
     Fix typos and magic number (Lukas Wunner)
     Support 1st gen Light Ridge controller (Lukas Wunner)
 
   Generic host bridge driver
     Use generic ECAM API (Jayachandran C)
 
   Cavium ThunderX host bridge driver
     Don't clobber read-only bits in bridge config registers (David Daney)
     Use generic ECAM API (Jayachandran C)
 
   Freescale i.MX6 host bridge driver
     Use enum instead of bool for variant indicator (Andrey Smirnov)
     Implement reset sequence for i.MX6+ (Andrey Smirnov)
     Factor out ref clock enable (Bjorn Helgaas)
     Add initial imx6sx support (Christoph Fritz)
     Add reset-gpio-active-high boolean property to DT (Petr Štetiar)
     Add DT property for link gen, default to Gen1 (Tim Harvey)
     dts: Specify imx6qp version of PCIe core (Andrey Smirnov)
     dts: Fix PCIe reset GPIO polarity on Toradex Apalis Ixora (Petr Štetiar)
 
   Marvell Armada host bridge driver
     add DT binding for Marvell Armada 7K/8K PCIe controller (Thomas Petazzoni)
     Add driver for Marvell Armada 7K/8K PCIe controller (Thomas Petazzoni)
 
   Marvell MVEBU host bridge driver
     Constify mvebu_pcie_pm_ops structure (Jisheng Zhang)
     Use SET_NOIRQ_SYSTEM_SLEEP_PM_OPS for mvebu_pcie_pm_ops (Jisheng Zhang)
 
   Microsoft Hyper-V host bridge driver
     Report resources release after stopping the bus (Vitaly Kuznetsov)
     Add explicit barriers to config space access (Vitaly Kuznetsov)
 
   Renesas R-Car host bridge driver
     Select PCI_MSI_IRQ_DOMAIN (Arnd Bergmann)
 
   Synopsys DesignWare host bridge driver
     Remove incorrect RC memory base/limit configuration (Gabriele Paoloni)
     Move Root Complex setup code to dw_pcie_setup_rc() (Jisheng Zhang)
 
   TI Keystone host bridge driver
     Add error IRQ handler (Murali Karicheri)
     Remove unnecessary goto statement (Murali Karicheri)
 
   Miscellaneous
     Fix spelling errors (Colin Ian King)
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJXPdMKAAoJEFmIoMA60/r8ofUP/j0zyzn24f0xY1wLeGJ8geB9
 6nHk1QdkPqwCiXZahEcnA5HMlFCl/ciWjjsoCqeMlvS6NXkX13KGcc1UGZszelTs
 68bFhyBKqcoMn0it53vBjBXnkfA64PmlxwY/T1ADulxL8amFOCpjjBruZ8pxJ/U7
 r6uHvhxUxHCRF7hMmpNN+V5XWXWCFFkPJZvxOTkglaxkbdnhZ0h0Xz9p9liUvjPH
 mBE72E3WUjiGogXGoLAPDclz1NI6rhRVUyTRcQ8EWaOwitV3OqMuDpAwoWH62ZZJ
 iorCkQk2/eKfN6OA6UgZh4loauAty0FeoZDX7ZVftQr52IpAzRUVx1oAq0J7u4ga
 KRX37mlK/53UcMZyv9Lz2kw4KjaLLELiInzcF+w3Bbov4UhY4/sL5uh9eNMFvSUU
 iZuY+GFlceL0P6wZuVKU5U8td/CyBr3f5vY/3htxuYHE1xJq4FkL92JpWRCvwpVr
 YdCzocscw73Yn8ZMplt8DX2fyabN7HyGezbQISrDDGY6T0ZDsRRKc6FFAt4xF+ta
 JJ+bcY8OcXtxGw6SXtrscL7vNXdR7Zg1HBSa8Sl/CopCdW9zs0VdwgFoxgORcWDT
 mphIgt57DMzaiUUaV8FRQz0mSLixnAcCEfGjVbAEEw3SP5ZChGfS3EknKb/CPRyk
 TD6I3pXTBhTWXd8aS113
 =68Iz
 -----END PGP SIGNATURE-----

Merge tag 'pci-v4.7-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci

Pull PCI updates from Bjorn Helgaas:
 "Enumeration:
   - Refine PCI support check in pcibios_init() (Adrian-Ken Rueegsegger)
   - Provide common functions for ECAM mapping (Jayachandran C)
   - Allow all PCIe services on non-ACPI host bridges (Jon Derrick)
   - Remove return values from pcie_port_platform_notify() and relatives (Jon Derrick)
   - Widen portdrv service type from 4 bits to 8 bits (Keith Busch)
   - Add Downstream Port Containment portdrv service type (Keith Busch)
   - Add Downstream Port Containment driver (Keith Busch)

  Resource management:
   - Identify Enhanced Allocation (EA) BAR Equivalent resources in sysfs (Alex Williamson)
   - Supply CPU physical address (not bus address) to iomem_is_exclusive() (Bjorn Helgaas)
   - alpha: Call iomem_is_exclusive() for IORESOURCE_MEM, but not IORESOURCE_IO (Bjorn Helgaas)
   - Mark Broadwell-EP Home Agent 1 as having non-compliant BARs (Prarit Bhargava)
   - Disable all BAR sizing for devices with non-compliant BARs (Prarit Bhargava)
   - Move PCI I/O space management from OF to PCI core code (Tomasz Nowicki)

  PCI device hotplug:
   - acpiphp_ibm: Avoid uninitialized variable reference (Dan Carpenter)
   - Use cached copy of PCI_EXP_SLTCAP_HPC bit (Lukas Wunner)

  Virtualization:
   - Mark Intel i40e NIC INTx masking as broken (Alex Williamson)
   - Reverse standard ACS vs device-specific ACS enabling (Alex Williamson)
   - Work around Intel Sunrise Point PCH incorrect ACS capability (Alex Williamson)

  IOMMU:
   - Add pci_add_dma_alias() to abstract implementation (Bjorn Helgaas)
   - Move informational printk to pci_add_dma_alias() (Bjorn Helgaas)
   - Add support for multiple DMA aliases (Jacek Lawrynowicz)
   - Add DMA alias quirk for mic_x200_dma (Jacek Lawrynowicz)

  Thunderbolt:
   - Fix double free of drom buffer (Andreas Noever)
   - Add Intel Thunderbolt device IDs (Lukas Wunner)
   - Fix typos and magic number (Lukas Wunner)
   - Support 1st gen Light Ridge controller (Lukas Wunner)

  Generic host bridge driver:
   - Use generic ECAM API (Jayachandran C)

  Cavium ThunderX host bridge driver:
   - Don't clobber read-only bits in bridge config registers (David Daney)
   - Use generic ECAM API (Jayachandran C)

  Freescale i.MX6 host bridge driver:
   - Use enum instead of bool for variant indicator (Andrey Smirnov)
   - Implement reset sequence for i.MX6+ (Andrey Smirnov)
   - Factor out ref clock enable (Bjorn Helgaas)
   - Add initial imx6sx support (Christoph Fritz)
   - Add reset-gpio-active-high boolean property to DT (Petr Štetiar)
   - Add DT property for link gen, default to Gen1 (Tim Harvey)
   - dts: Specify imx6qp version of PCIe core (Andrey Smirnov)
   - dts: Fix PCIe reset GPIO polarity on Toradex Apalis Ixora (Petr Štetiar)

  Marvell Armada host bridge driver:
   - add DT binding for Marvell Armada 7K/8K PCIe controller (Thomas Petazzoni)
   - Add driver for Marvell Armada 7K/8K PCIe controller (Thomas Petazzoni)

  Marvell MVEBU host bridge driver:
   - Constify mvebu_pcie_pm_ops structure (Jisheng Zhang)
   - Use SET_NOIRQ_SYSTEM_SLEEP_PM_OPS for mvebu_pcie_pm_ops (Jisheng Zhang)

  Microsoft Hyper-V host bridge driver:
   - Report resources release after stopping the bus (Vitaly Kuznetsov)
   - Add explicit barriers to config space access (Vitaly Kuznetsov)

  Renesas R-Car host bridge driver:
   - Select PCI_MSI_IRQ_DOMAIN (Arnd Bergmann)

  Synopsys DesignWare host bridge driver:
   - Remove incorrect RC memory base/limit configuration (Gabriele Paoloni)
   - Move Root Complex setup code to dw_pcie_setup_rc() (Jisheng Zhang)

  TI Keystone host bridge driver:
   - Add error IRQ handler (Murali Karicheri)
   - Remove unnecessary goto statement (Murali Karicheri)

  Miscellaneous:
   - Fix spelling errors (Colin Ian King)"

* tag 'pci-v4.7-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (48 commits)
  PCI: Disable all BAR sizing for devices with non-compliant BARs
  x86/PCI: Mark Broadwell-EP Home Agent 1 as having non-compliant BARs
  PCI: Identify Enhanced Allocation (EA) BAR Equivalent resources in sysfs
  PCI, of: Move PCI I/O space management to PCI core code
  PCI: generic, thunder: Use generic ECAM API
  PCI: Provide common functions for ECAM mapping
  PCI: hv: Add explicit barriers to config space access
  PCI: Use cached copy of PCI_EXP_SLTCAP_HPC bit
  PCI: Add Downstream Port Containment driver
  PCI: Add Downstream Port Containment portdrv service type
  PCI: Widen portdrv service type from 4 bits to 8 bits
  PCI: designware: Remove incorrect RC memory base/limit configuration
  PCI: hv: Report resources release after stopping the bus
  ARM: dts: imx6qp: Specify imx6qp version of PCIe core
  PCI: imx6: Implement reset sequence for i.MX6+
  PCI: imx6: Use enum instead of bool for variant indicator
  PCI: thunder: Don't clobber read-only bits in bridge config registers
  thunderbolt: Fix double free of drom buffer
  PCI: rcar: Select PCI_MSI_IRQ_DOMAIN
  PCI: armada: Add driver for Marvell Armada 7K/8K PCIe controller
  ...
2016-05-19 13:10:54 -07:00
Joerg Roedel
6c0b43df74 Merge branches 'arm/io-pgtable', 'arm/rockchip', 'arm/omap', 'x86/vt-d', 'ppc/pamu', 'core' and 'x86/amd' into next 2016-05-09 19:39:17 +02:00
Joerg Roedel
e85e8f69ce iommu/amd: Remove statistics code
The statistics are not really used for anything and should
be replaced by generic and per-device statistic counters.
Remove the code for now.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-05-09 16:58:37 +02:00
Joerg Roedel
fd6c50ee3a iommu/amd: Move get_device_id() and friends to beginning of file
They will be needed there later.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-21 18:24:02 +02:00
Joerg Roedel
9ee35e4c6f iommu/amd: Don't use IS_ERR_VALUE to check integer values
Use the better 'var < 0' check.

Fixes: 7aba6cb9ee ('iommu/amd: Make call-sites of get_device_id aware of its return value')
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-21 18:21:31 +02:00
Dan Carpenter
2d8e1f039d iommu/amd: Signedness bug in acpihid_device_group()
"devid" needs to be signed for the error handling to work.

Fixes: b097d11a0f ('iommu/amd: Manage iommu_group for ACPI HID devices')
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-15 12:09:10 +02:00
Joerg Roedel
e315604834 iommu/amd: Fix checking of pci dma aliases
Commit 61289cb ('iommu/amd: Remove old alias handling code')
removed the old alias handling code from the AMD IOMMU
driver because this is now handled by the IOMMU core code.

But this also removed the handling of PCI aliases, which is
not handled by the core code. This caused issues with PCI
devices that have hidden PCIe-to-PCI bridges that rewrite
the request-id.

Fix this bug by re-introducing some of the removed functions
from commit 61289cbaf6 and add a alias field
'struct iommu_dev_data'. This field carrys the return value
of the get_alias() function and uses that instead of the
amd_iommu_alias_table[] array in the code.

Fixes: 61289cbaf6 ('iommu/amd: Remove old alias handling code')
Cc: stable@vger.kernel.org # v4.4+
Tested-by: Tomasz Golinski <tomaszg@math.uwb.edu.pl>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-11 16:07:51 +02:00
Wan Zongshun
9a4d3bf56c iommu/amd: Set AMD iommu callbacks for amba bus
AMD Uart DMA belongs to ACPI HID type device, and its driver
is basing on AMBA Bus, need also IOMMU support.

This patch is just to set the AMD iommu callbacks for amba bus.

Signed-off-by: Wan Zongshun <Vincent.Wan@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 13:29:42 +02:00
Wan Zongshun
b097d11a0f iommu/amd: Manage iommu_group for ACPI HID devices
This patch creates a new function for finding or creating an IOMMU
group for acpihid(ACPI Hardware ID) device.

The acpihid devices with the same devid will be put into same group and
there will have the same domain id and share the same page table.

Signed-off-by: Wan Zongshun <Vincent.Wan@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 13:29:42 +02:00
Wan Zongshun
2bf9a0a127 iommu/amd: Add iommu support for ACPI HID devices
Current IOMMU driver make assumption that the downstream devices are PCI.
With the newly added ACPI-HID IVHD device entry support, this is no
longer true. This patch is to add dev type check and to distinguish the
pci and acpihid device code path.

Signed-off-by: Wan Zongshun <Vincent.Wan@amd.com>
Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 13:29:42 +02:00
Wan Zongshun
7aba6cb9ee iommu/amd: Make call-sites of get_device_id aware of its return value
This patch is to make the call-sites of get_device_id aware of its
return value.

Signed-off-by: Wan Zongshun <Vincent.Wan@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 13:29:41 +02:00
Wan Zongshun
2a0cb4e2d4 iommu/amd: Add new map for storing IVHD dev entry type HID
This patch introduces acpihid_map, which is used to store
the new IVHD device entry extracted from BIOS IVRS table.

It also provides a utility function add_acpi_hid_device(),
to add this types of devices to the map.

Signed-off-by: Wan Zongshun <Vincent.Wan@amd.com>
Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-04-07 13:29:41 +02:00
Joerg Roedel
b6809ee573 iommu/amd: Detach device from domain before removal
Detach the device that is about to be removed from its
domain (if it has one) to clear any related state like DTE
entry and device's ATS state.

Reported-by: Kelly Zytaruk <Kelly.Zytaruk@amd.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-02-29 17:25:25 +01:00
Baoquan He
9b1a12d291 iommu/amd: Correct the wrong setting of alias DTE in do_attach
In below commit alias DTE is set when its peripheral is
setting DTE. However there's a code bug here to wrongly
set the alias DTE, correct it in this patch.

commit e25bfb56ea
Author: Joerg Roedel <jroedel@suse.de>
Date:   Tue Oct 20 17:33:38 2015 +0200

    iommu/amd: Set alias DTE in do_attach/do_detach

Signed-off-by: Baoquan He <bhe@redhat.com>
Tested-by: Mark Hounschell <markh@compro.net>
Cc: stable@vger.kernel.org # v4.4
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-01-29 12:30:47 +01:00
Dan Carpenter
1fb260bc00 iommu/amd: Remove an unneeded condition
get_device_id() returns an unsigned short device id.  It never fails and
it never returns a negative so we can remove this condition.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-01-07 13:08:07 +01:00
Joerg Roedel
a639a8eecf iommu/amd: Preallocate dma_ops apertures based on dma_mask
Preallocate between 4 and 8 apertures when a device gets it
dma_mask. With more apertures we reduce the lock contention
of the domain lock significantly.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:54 +01:00
Joerg Roedel
7b5e25b84e iommu/amd: Use trylock to aquire bitmap_lock
First search for a non-contended aperture with trylock
before spinning.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:54 +01:00
Joerg Roedel
5f6bed5005 iommu/amd: Make dma_ops_domain->next_index percpu
Make this pointer percpu so that we start searching for new
addresses in the range we last stopped and which is has a
higher probability of being still in the cache.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:54 +01:00
Joerg Roedel
92d420ec02 iommu/amd: Relax locking in dma_ops path
Remove the long holding times of the domain->lock and rely
on the bitmap_lock instead.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:54 +01:00
Joerg Roedel
a73c156665 iommu/amd: Initialize new aperture range before making it visible
Make sure the aperture range is fully initialized before it
is visible to the address allocator.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:53 +01:00
Joerg Roedel
7bfa5bd270 iommu/amd: Build io page-tables with cmpxchg64
This allows to build up the page-tables without holding any
locks. As a consequence it removes the need to pre-populate
dma_ops page-tables.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:53 +01:00
Joerg Roedel
266a3bd28f iommu/amd: Allocate new aperture ranges in dma_ops_alloc_addresses
It really belongs there and not in __map_single.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:53 +01:00
Joerg Roedel
4eeca8c5e7 iommu/amd: Optimize dma_ops_free_addresses
Don't flush the iommu tlb when we free something behind the
current next_bit pointer. Update the next_bit pointer
instead and let the flush happen on the next wraparound in
the allocation path.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:53 +01:00
Joerg Roedel
ab7032bb9c iommu/amd: Remove need_flush from struct dma_ops_domain
The flushing of iommu tlbs is now done on a per-range basis.
So there is no need anymore for domain-wide flush tracking.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:53 +01:00
Joerg Roedel
2a87442c5b iommu/amd: Iterate over all aperture ranges in dma_ops_area_alloc
This way we don't need to care about the next_index wrapping
around in dma_ops_alloc_addresses.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:52 +01:00
Joerg Roedel
d41ab09896 iommu/amd: Flush iommu tlb in dma_ops_free_addresses
Instead of setting need_flush, do the flush directly in
dma_ops_free_addresses.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:52 +01:00
Joerg Roedel
ebaecb423b iommu/amd: Rename dma_ops_domain->next_address to next_index
It points to the next aperture index to allocate from. We
don't need the full address anymore because this is now
tracked in struct aperture_range.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:52 +01:00
Joerg Roedel
05ab49e005 iommu/amd: Remove 'start' parameter from dma_ops_area_alloc
Parameter is not needed because the value is part of the
already passed in struct dma_ops_domain.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:52 +01:00
Joerg Roedel
ccb50e03da iommu/amd: Flush iommu tlb in dma_ops_aperture_alloc()
Since the allocator wraparound happens in this function now,
flush the iommu tlb there too.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:51 +01:00
Joerg Roedel
60e6a7cb44 iommu/amd: Retry address allocation within one aperture
Instead of skipping to the next aperture, first try again in
the current one.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:51 +01:00
Joerg Roedel
ae62d49c7a iommu/amd: Move aperture_range.offset to another cache-line
Moving it before the pte_pages array puts in into the same
cache-line as the spin-lock and the bitmap array pointer.
This should safe a cache-miss.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:51 +01:00
Joerg Roedel
a0f51447f4 iommu/amd: Add dma_ops_aperture_alloc() function
Make this a wrapper around iommu_ops_area_alloc() for now
and add more logic to this function later on.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:51 +01:00
Joerg Roedel
b57c3c802e iommu/amd: Pass correct shift to iommu_area_alloc()
The page-offset of the aperture must be passed instead of 0.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:50 +01:00
Joerg Roedel
84b3a0bc88 iommu/amd: Flush the IOMMU TLB before the addresses are freed
This allows to keep the bitmap_lock only for a very short
period of time.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:50 +01:00
Joerg Roedel
53b3b65aa5 iommu/amd: Flush IOMMU TLB on __map_single error path
There have been present PTEs which in theory could have made
it to the IOMMU TLB. Flush the addresses out on the error
path to make sure no stale entries remain.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:50 +01:00
Joerg Roedel
08c5fb938e iommu/amd: Introduce bitmap_lock in struct aperture_range
This lock only protects the address allocation bitmap in one
aperture.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:50 +01:00
Joerg Roedel
007b74bab2 iommu/amd: Move 'struct dma_ops_domain' definition to amd_iommu.c
It is only used in this file anyway, so keep it there. Same
with 'struct aperture_range'.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:50 +01:00
Joerg Roedel
a7fb668fd8 iommu/amd: Warn only once on unexpected pte value
This prevents possible flooding of the kernel log.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-28 17:18:50 +01:00
Mel Gorman
d0164adc89 mm, page_alloc: distinguish between being unable to sleep, unwilling to sleep and avoiding waking kswapd
__GFP_WAIT has been used to identify atomic context in callers that hold
spinlocks or are in interrupts.  They are expected to be high priority and
have access one of two watermarks lower than "min" which can be referred
to as the "atomic reserve".  __GFP_HIGH users get access to the first
lower watermark and can be called the "high priority reserve".

Over time, callers had a requirement to not block when fallback options
were available.  Some have abused __GFP_WAIT leading to a situation where
an optimisitic allocation with a fallback option can access atomic
reserves.

This patch uses __GFP_ATOMIC to identify callers that are truely atomic,
cannot sleep and have no alternative.  High priority users continue to use
__GFP_HIGH.  __GFP_DIRECT_RECLAIM identifies callers that can sleep and
are willing to enter direct reclaim.  __GFP_KSWAPD_RECLAIM to identify
callers that want to wake kswapd for background reclaim.  __GFP_WAIT is
redefined as a caller that is willing to enter direct reclaim and wake
kswapd for background reclaim.

This patch then converts a number of sites

o __GFP_ATOMIC is used by callers that are high priority and have memory
  pools for those requests. GFP_ATOMIC uses this flag.

o Callers that have a limited mempool to guarantee forward progress clear
  __GFP_DIRECT_RECLAIM but keep __GFP_KSWAPD_RECLAIM. bio allocations fall
  into this category where kswapd will still be woken but atomic reserves
  are not used as there is a one-entry mempool to guarantee progress.

o Callers that are checking if they are non-blocking should use the
  helper gfpflags_allow_blocking() where possible. This is because
  checking for __GFP_WAIT as was done historically now can trigger false
  positives. Some exceptions like dm-crypt.c exist where the code intent
  is clearer if __GFP_DIRECT_RECLAIM is used instead of the helper due to
  flag manipulations.

o Callers that built their own GFP flags instead of starting with GFP_KERNEL
  and friends now also need to specify __GFP_KSWAPD_RECLAIM.

The first key hazard to watch out for is callers that removed __GFP_WAIT
and was depending on access to atomic reserves for inconspicuous reasons.
In some cases it may be appropriate for them to use __GFP_HIGH.

The second key hazard is callers that assembled their own combination of
GFP flags instead of starting with something like GFP_KERNEL.  They may
now wish to specify __GFP_KSWAPD_RECLAIM.  It's almost certainly harmless
if it's missed in most cases as other activity will wake kswapd.

Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Vitaly Wool <vitalywool@gmail.com>
Cc: Rik van Riel <riel@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-11-06 17:50:42 -08:00
Joerg Roedel
b67ad2f7c7 Merge branches 'x86/vt-d', 'arm/omap', 'arm/smmu', 's390', 'core' and 'x86/amd' into next
Conflicts:
	drivers/iommu/amd_iommu_types.h
2015-11-02 20:03:34 +09:00
Joerg Roedel
a960fadbe6 iommu: Add device_group call-back to x86 iommu drivers
Set the device_group call-back to pci_device_group() for the
Intel VT-d and the AMD IOMMU driver.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-22 00:00:49 +02:00
Joerg Roedel
deba4bce16 iommu/amd: Remove cmd_buf_size and evt_buf_size from struct amd_iommu
The driver always uses a constant size for these buffers
anyway, so there is no need to waste memory to store the
sizes.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-21 11:30:33 +02:00
Joerg Roedel
61289cbaf6 iommu/amd: Remove old alias handling code
This mostly removes the code to create dev_data structures
for alias device ids. They are not necessary anymore, as
they were only created for device ids which have no struct
pci_dev associated with it.  But these device ids are
handled in a simpler way now, so there is no need for this
code anymore.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-21 11:30:32 +02:00
Joerg Roedel
e25bfb56ea iommu/amd: Set alias DTE in do_attach/do_detach
With this we don't have to create dev_data entries for
non-existent devices (which only exist as request-ids).

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-21 11:30:32 +02:00
Joerg Roedel
272e4f99e9 iommu/amd: WARN when __[attach|detach]_device are called with irqs enabled
These functions rely on being called with IRQs disabled. Add
a WARN_ON to detect early when its not.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-21 11:29:27 +02:00
Joerg Roedel
f1dd0a8bcd iommu/amd: Don't disable IRQs in __detach_device
This function is already called with IRQs disabled already.
So no need to disable them again.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-21 11:29:27 +02:00
Joerg Roedel
150952f969 iommu/amd: Do not iterate over alias-list in __[attach|detach]_device
The alias list is handled aleady by iommu core code. No need
anymore to handle it in this part of the AMD IOMMU code

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-21 11:29:27 +02:00
Joerg Roedel
f34c73f55a iommu/amd: Do not BUG_ON in __detach_device()
The condition in the BUG_ON is an indicator of a BUG, but no
reason to kill the code path. Turn it into a WARN_ON and
bail out if it is hit.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-21 11:29:26 +02:00
Joerg Roedel
cbf3ccd09d iommu/amd: Don't clear DTE flags when modifying it
During device assignment/deassignment the flags in the DTE
get lost, which might cause spurious faults, for example
when the device tries to access the system management range.
Fix this by not clearing the flags with the rest of the DTE.

Reported-by: G. Richard Bellamy <rbellamy@pteradigm.com>
Tested-by: G. Richard Bellamy <rbellamy@pteradigm.com>
Cc: stable@vger.kernel.org
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-21 11:29:06 +02:00