AuxXxilium/linux_dsm_epyc7002
1
0
Fork 0
mirror of https://github.com/AuxXxilium/linux_dsm_epyc7002.git synced 2024-12-27 07:25:14 +07:00
Code Issues Actions Packages Projects Releases Wiki Activity
2a6d0d63d9
linux_dsm_epyc7002/drivers/pci/controller/vmd.c

745 lines
19 KiB
C
Raw Normal View History

PCI: Add SPDX GPL-2.0 to replace GPL v2 boilerplate Add SPDX GPL-2.0 to all PCI files that specified the GPL version 2 license. Remove the boilerplate GPL version 2 language, relying on the assertion in b24413180f56 ("License cleanup: add SPDX GPL-2.0 license identifier to files with no license") that the SPDX identifier may be used instead of the full boilerplate text. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-01-27 01:50:27 +07:00
// SPDX-License-Identifier: GPL-2.0
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
/*
* Volume Management Device driver
* Copyright (c) 2015, Intel Corporation.
*/
#include <linux/device.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/msi.h>
#include <linux/pci.h>
PCI: vmd: Use SRCU as a local RCU to prevent delaying global RCU SRCU lets synchronize_srcu() depend on VMD-local RCU primitives, preventing long delays from locking up RCU in other systems. VMD performs a synchronize when removing a device, but will hit all IRQ lists if the device uses all VMD vectors. This patch will not help VMD's RCU synchronization, but will isolate the read side delays to the VMD subsystem. Additionally, the use of SRCU in VMD's ISR will keep it isolated from any other RCU waiters in the rest of the system. Tested using concurrent FIO and NVMe resets: [global] rw=read bs=4k direct=1 ioengine=libaio iodepth=32 norandommap timeout=300 runtime=1000000000 [nvme0] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme0n1 [nvme1] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme1n1 while (true) do for i in /sys/class/nvme/nvme*; do echo "Resetting ${i##*/}" echo 1 > $i/reset_controller; sleep 5 done; done Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-11-12 06:08:45 +07:00
#include <linux/srcu.h>
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
#include <linux/rculist.h>
#include <linux/rcupdate.h>
#include <asm/irqdomain.h>
#include <asm/device.h>
#include <asm/msi.h>
#include <asm/msidef.h>
#define VMD_CFGBAR 0
#define VMD_MEMBAR1 2
#define VMD_MEMBAR2 4
PCI: vmd: Add offset to bus numbers if necessary Depending on platform configuration, certain VMD devices may have an additional configuration option which specifies the range of bus numbers allowed in a VMD PCIe domain. We determine this requirement by checking the value of two vendor specific config registers in the VMD endpoint: VMCAP[0] | VMCONFIG[9:8] | Bus Numbers ---------------------------------------- 0 | * | 0-255 1 | 00 | 0-127 1 | 01 | 128-255 1 | 10 | 0-255 This feature is also added as a bit in driver_data, to allow future conforming device ids which support these features to be enabled through sysfs new_id. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:28:00 +07:00
#define PCI_REG_VMCAP 0x40
#define BUS_RESTRICT_CAP(vmcap) (vmcap & 0x1)
#define PCI_REG_VMCONFIG 0x44
#define BUS_RESTRICT_CFG(vmcfg) ((vmcfg >> 8) & 0x3)
PCI: vmd: Assign membar addresses from shadow registers Certain VMD devices have registers within membar 2 which may shadow the membar 1 and membar 2 addresses. These are intended to be used in virtualization, where assigning a guest address wouldn't be translated in the assignment to root port and child devices because the addresses exist within the assignment message. These values will only reflect the membars when enabled in the BIOS, as determined by a register in the VMD device. This patch declares this option as a bit in the pci id driver_data, so that future conforming device ids can be enabled through sysfs new_id if necessary. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:59 +07:00
#define PCI_REG_VMLOCK 0x70
#define MB2_SHADOW_EN(vmlock) (vmlock & 0x2)
PCI: vmd: Fix shadow offsets to reflect spec changes The shadow offset scratchpad was moved to 0x2000-0x2010. Update the location to get the correct shadow offset. Fixes: 6788958e4f3c ("PCI: vmd: Assign membar addresses from shadow registers") Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: stable@vger.kernel.org # v5.2+
2019-09-16 20:54:35 +07:00
#define MB2_SHADOW_OFFSET 0x2000
#define MB2_SHADOW_SIZE 16
PCI: vmd: Assign membar addresses from shadow registers Certain VMD devices have registers within membar 2 which may shadow the membar 1 and membar 2 addresses. These are intended to be used in virtualization, where assigning a guest address wouldn't be translated in the assignment to root port and child devices because the addresses exist within the assignment message. These values will only reflect the membars when enabled in the BIOS, as determined by a register in the VMD device. This patch declares this option as a bit in the pci id driver_data, so that future conforming device ids can be enabled through sysfs new_id if necessary. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:59 +07:00
enum vmd_features {
/*
* Device may contain registers which hint the physical location of the
* membars, in order to allow proper address translation during
* resource assignment to enable guest virtualization
*/
VMD_FEAT_HAS_MEMBAR_SHADOW = (1 << 0),
PCI: vmd: Add offset to bus numbers if necessary Depending on platform configuration, certain VMD devices may have an additional configuration option which specifies the range of bus numbers allowed in a VMD PCIe domain. We determine this requirement by checking the value of two vendor specific config registers in the VMD endpoint: VMCAP[0] | VMCONFIG[9:8] | Bus Numbers ---------------------------------------- 0 | * | 0-255 1 | 00 | 0-127 1 | 01 | 128-255 1 | 10 | 0-255 This feature is also added as a bit in driver_data, to allow future conforming device ids which support these features to be enabled through sysfs new_id. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:28:00 +07:00
/*
* Device may provide root port configuration information which limits
* bus numbering
*/
VMD_FEAT_HAS_BUS_RESTRICTIONS = (1 << 1),
PCI: vmd: Assign membar addresses from shadow registers Certain VMD devices have registers within membar 2 which may shadow the membar 1 and membar 2 addresses. These are intended to be used in virtualization, where assigning a guest address wouldn't be translated in the assignment to root port and child devices because the addresses exist within the assignment message. These values will only reflect the membars when enabled in the BIOS, as determined by a register in the VMD device. This patch declares this option as a bit in the pci id driver_data, so that future conforming device ids can be enabled through sysfs new_id if necessary. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:59 +07:00
};
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
/*
* Lock for manipulating VMD IRQ lists.
*/
static DEFINE_RAW_SPINLOCK(list_lock);
/**
* struct vmd_irq - private data to map driver IRQ to the VMD shared vector
* @node: list item for parent traversal.
* @irq: back pointer to parent.
x86/PCI: VMD: Fix infinite loop executing irq's We can't initialize the list head on deletion as this causes the node to point to itself, which causes an infinite loop if vmd_irq() happens to be servicing that node. The list initialization was trying to fix a bug from multiple calls to disable the same IRQ. Fix this instead by having the VMD driver track if the interrupt is enabled. [bhelgaas: changelog, add "Fixes"] Fixes: 97e923063575 ("x86/PCI: VMD: Initialize list item in IRQ disable") Reported-by: Grzegorz Koczot <grzegorz.koczot@intel.com> Tested-by: Miroslaw Drost <miroslaw.drost@intel.com> Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by Jon Derrick: <jonathan.derrick@intel.com>
2016-08-24 04:36:42 +07:00
* @enabled: true if driver enabled IRQ
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
* @virq: the virtual IRQ value provided to the requesting driver.
*
* Every MSI/MSI-X IRQ requested for a device in a VMD domain will be mapped to
* a VMD IRQ using this structure.
*/
struct vmd_irq {
struct list_head node;
struct vmd_irq_list *irq;
x86/PCI: VMD: Fix infinite loop executing irq's We can't initialize the list head on deletion as this causes the node to point to itself, which causes an infinite loop if vmd_irq() happens to be servicing that node. The list initialization was trying to fix a bug from multiple calls to disable the same IRQ. Fix this instead by having the VMD driver track if the interrupt is enabled. [bhelgaas: changelog, add "Fixes"] Fixes: 97e923063575 ("x86/PCI: VMD: Initialize list item in IRQ disable") Reported-by: Grzegorz Koczot <grzegorz.koczot@intel.com> Tested-by: Miroslaw Drost <miroslaw.drost@intel.com> Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by Jon Derrick: <jonathan.derrick@intel.com>
2016-08-24 04:36:42 +07:00
bool enabled;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
unsigned int virq;
};
/**
* struct vmd_irq_list - list of driver requested IRQs mapping to a VMD vector
* @irq_list: the list of irq's the VMD one demuxes to.
PCI: vmd: Use SRCU as a local RCU to prevent delaying global RCU SRCU lets synchronize_srcu() depend on VMD-local RCU primitives, preventing long delays from locking up RCU in other systems. VMD performs a synchronize when removing a device, but will hit all IRQ lists if the device uses all VMD vectors. This patch will not help VMD's RCU synchronization, but will isolate the read side delays to the VMD subsystem. Additionally, the use of SRCU in VMD's ISR will keep it isolated from any other RCU waiters in the rest of the system. Tested using concurrent FIO and NVMe resets: [global] rw=read bs=4k direct=1 ioengine=libaio iodepth=32 norandommap timeout=300 runtime=1000000000 [nvme0] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme0n1 [nvme1] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme1n1 while (true) do for i in /sys/class/nvme/nvme*; do echo "Resetting ${i##*/}" echo 1 > $i/reset_controller; sleep 5 done; done Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-11-12 06:08:45 +07:00
* @srcu: SRCU struct for local synchronization.
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
* @count: number of child IRQs assigned to this vector; used to track
* sharing.
*/
struct vmd_irq_list {
struct list_head irq_list;
PCI: vmd: Use SRCU as a local RCU to prevent delaying global RCU SRCU lets synchronize_srcu() depend on VMD-local RCU primitives, preventing long delays from locking up RCU in other systems. VMD performs a synchronize when removing a device, but will hit all IRQ lists if the device uses all VMD vectors. This patch will not help VMD's RCU synchronization, but will isolate the read side delays to the VMD subsystem. Additionally, the use of SRCU in VMD's ISR will keep it isolated from any other RCU waiters in the rest of the system. Tested using concurrent FIO and NVMe resets: [global] rw=read bs=4k direct=1 ioengine=libaio iodepth=32 norandommap timeout=300 runtime=1000000000 [nvme0] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme0n1 [nvme1] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme1n1 while (true) do for i in /sys/class/nvme/nvme*; do echo "Resetting ${i##*/}" echo 1 > $i/reset_controller; sleep 5 done; done Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-11-12 06:08:45 +07:00
struct srcu_struct srcu;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
unsigned int count;
};
struct vmd_dev {
struct pci_dev *dev;
spinlock_t cfg_lock;
char __iomem *cfgbar;
int msix_count;
struct vmd_irq_list *irqs;
struct pci_sysdata sysdata;
struct resource resources[3];
struct irq_domain *irq_domain;
struct pci_bus *bus;
PCI: vmd: Fix config addressing when using bus offsets VMD maps child device config spaces to the VMD Config BAR linearly regardless of the starting bus offset. Because of this, the config address decode must ignore starting bus offsets when mapping the BDF to the config space address. Fixes: 2a5a9c9a20f9 ("PCI: vmd: Add offset to bus numbers if necessary") Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: stable@vger.kernel.org # v5.2+
2019-09-16 20:54:34 +07:00
u8 busn_start;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
};
static inline struct vmd_dev *vmd_from_bus(struct pci_bus *bus)
{
return container_of(bus->sysdata, struct vmd_dev, sysdata);
}
x86/PCI: VMD: Eliminate index member from IRQ list Use math to discover the IRQ list index number relative to the IRQ list head. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-09-03 00:53:05 +07:00
static inline unsigned int index_from_irqs(struct vmd_dev *vmd,
struct vmd_irq_list *irqs)
{
return irqs - vmd->irqs;
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
/*
* Drivers managing a device in a VMD domain allocate their own IRQs as before,
* but the MSI entry for the hardware it's driving will be programmed with a
* destination ID for the VMD MSI-X table. The VMD muxes interrupts in its
* domain into one of its own, and the VMD driver de-muxes these for the
* handlers sharing that VMD IRQ. The vmd irq_domain provides the operations
* and irq_chip to set this up.
*/
static void vmd_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
{
struct vmd_irq *vmdirq = data->chip_data;
struct vmd_irq_list *irq = vmdirq->irq;
x86/PCI: VMD: Eliminate index member from IRQ list Use math to discover the IRQ list index number relative to the IRQ list head. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-09-03 00:53:05 +07:00
struct vmd_dev *vmd = irq_data_get_irq_handler_data(data);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
msg->address_hi = MSI_ADDR_BASE_HI;
x86/PCI: VMD: Eliminate index member from IRQ list Use math to discover the IRQ list index number relative to the IRQ list head. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-09-03 00:53:05 +07:00
msg->address_lo = MSI_ADDR_BASE_LO |
MSI_ADDR_DEST_ID(index_from_irqs(vmd, irq));
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
msg->data = 0;
}
/*
* We rely on MSI_FLAG_USE_DEF_CHIP_OPS to set the IRQ mask/unmask ops.
*/
static void vmd_irq_enable(struct irq_data *data)
{
struct vmd_irq *vmdirq = data->chip_data;
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 22:39:51 +07:00
unsigned long flags;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 22:39:51 +07:00
raw_spin_lock_irqsave(&list_lock, flags);
x86/PCI: VMD: Fix infinite loop executing irq's We can't initialize the list head on deletion as this causes the node to point to itself, which causes an infinite loop if vmd_irq() happens to be servicing that node. The list initialization was trying to fix a bug from multiple calls to disable the same IRQ. Fix this instead by having the VMD driver track if the interrupt is enabled. [bhelgaas: changelog, add "Fixes"] Fixes: 97e923063575 ("x86/PCI: VMD: Initialize list item in IRQ disable") Reported-by: Grzegorz Koczot <grzegorz.koczot@intel.com> Tested-by: Miroslaw Drost <miroslaw.drost@intel.com> Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by Jon Derrick: <jonathan.derrick@intel.com>
2016-08-24 04:36:42 +07:00
WARN_ON(vmdirq->enabled);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
list_add_tail_rcu(&vmdirq->node, &vmdirq->irq->irq_list);
x86/PCI: VMD: Fix infinite loop executing irq's We can't initialize the list head on deletion as this causes the node to point to itself, which causes an infinite loop if vmd_irq() happens to be servicing that node. The list initialization was trying to fix a bug from multiple calls to disable the same IRQ. Fix this instead by having the VMD driver track if the interrupt is enabled. [bhelgaas: changelog, add "Fixes"] Fixes: 97e923063575 ("x86/PCI: VMD: Initialize list item in IRQ disable") Reported-by: Grzegorz Koczot <grzegorz.koczot@intel.com> Tested-by: Miroslaw Drost <miroslaw.drost@intel.com> Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by Jon Derrick: <jonathan.derrick@intel.com>
2016-08-24 04:36:42 +07:00
vmdirq->enabled = true;
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 22:39:51 +07:00
raw_spin_unlock_irqrestore(&list_lock, flags);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
data->chip->irq_unmask(data);
}
static void vmd_irq_disable(struct irq_data *data)
{
struct vmd_irq *vmdirq = data->chip_data;
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 22:39:51 +07:00
unsigned long flags;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
data->chip->irq_mask(data);
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 22:39:51 +07:00
raw_spin_lock_irqsave(&list_lock, flags);
x86/PCI: VMD: Fix infinite loop executing irq's We can't initialize the list head on deletion as this causes the node to point to itself, which causes an infinite loop if vmd_irq() happens to be servicing that node. The list initialization was trying to fix a bug from multiple calls to disable the same IRQ. Fix this instead by having the VMD driver track if the interrupt is enabled. [bhelgaas: changelog, add "Fixes"] Fixes: 97e923063575 ("x86/PCI: VMD: Initialize list item in IRQ disable") Reported-by: Grzegorz Koczot <grzegorz.koczot@intel.com> Tested-by: Miroslaw Drost <miroslaw.drost@intel.com> Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by Jon Derrick: <jonathan.derrick@intel.com>
2016-08-24 04:36:42 +07:00
if (vmdirq->enabled) {
list_del_rcu(&vmdirq->node);
vmdirq->enabled = false;
}
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 22:39:51 +07:00
raw_spin_unlock_irqrestore(&list_lock, flags);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
}
/*
* XXX: Stubbed until we develop acceptable way to not create conflicts with
* other devices sharing the same vector.
*/
static int vmd_irq_set_affinity(struct irq_data *data,
const struct cpumask *dest, bool force)
{
return -EINVAL;
}
static struct irq_chip vmd_msi_controller = {
.name = "VMD-MSI",
.irq_enable = vmd_irq_enable,
.irq_disable = vmd_irq_disable,
.irq_compose_msi_msg = vmd_compose_msi_msg,
.irq_set_affinity = vmd_irq_set_affinity,
};
static irq_hw_number_t vmd_get_hwirq(struct msi_domain_info *info,
msi_alloc_info_t *arg)
{
return 0;
}
/*
* XXX: We can be even smarter selecting the best IRQ once we solve the
* affinity problem.
*/
x86/PCI: VMD: Separate MSI and MSI-X vector sharing Child devices in a VMD domain that want to use MSI are slowing down MSI-X using devices sharing the same vectors. Move all MSI usage to a single VMD vector, and MSI-X devices can share the rest. Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Jon Derrick <jonathan.derrick@intel.com>
2016-06-20 22:39:53 +07:00
static struct vmd_irq_list *vmd_next_irq(struct vmd_dev *vmd, struct msi_desc *desc)
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
{
x86/PCI: VMD: Separate MSI and MSI-X vector sharing Child devices in a VMD domain that want to use MSI are slowing down MSI-X using devices sharing the same vectors. Move all MSI usage to a single VMD vector, and MSI-X devices can share the rest. Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Jon Derrick <jonathan.derrick@intel.com>
2016-06-20 22:39:53 +07:00
int i, best = 1;
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 22:39:51 +07:00
unsigned long flags;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
PCI: vmd: White list for fast interrupt handlers Devices with slow interrupt handlers are significantly harming performance when their interrupt vector is shared with a fast device. Create a class code white list for devices with known fast interrupt handlers and let all other devices share a single vector so that they don't interfere with performance. At the moment, only the NVM Express class code is on the list, but more may be added if VMD users desire to use other low-latency devices in these domains. Signed-off-by: Keith Busch <keith.busch@intel.com> [lorenzo.pieralisi@arm.com: changelog] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Acked-by: Jon Derrick: <jonathan.derrick@intel.com>
2018-05-08 23:00:22 +07:00
if (vmd->msix_count == 1)
x86/PCI: VMD: Separate MSI and MSI-X vector sharing Child devices in a VMD domain that want to use MSI are slowing down MSI-X using devices sharing the same vectors. Move all MSI usage to a single VMD vector, and MSI-X devices can share the rest. Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Jon Derrick <jonathan.derrick@intel.com>
2016-06-20 22:39:53 +07:00
return &vmd->irqs[0];
PCI: vmd: White list for fast interrupt handlers Devices with slow interrupt handlers are significantly harming performance when their interrupt vector is shared with a fast device. Create a class code white list for devices with known fast interrupt handlers and let all other devices share a single vector so that they don't interfere with performance. At the moment, only the NVM Express class code is on the list, but more may be added if VMD users desire to use other low-latency devices in these domains. Signed-off-by: Keith Busch <keith.busch@intel.com> [lorenzo.pieralisi@arm.com: changelog] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Acked-by: Jon Derrick: <jonathan.derrick@intel.com>
2018-05-08 23:00:22 +07:00
/*
* White list for fast-interrupt handlers. All others will share the
* "slow" interrupt vector.
*/
switch (msi_desc_to_pci_dev(desc)->class) {
case PCI_CLASS_STORAGE_EXPRESS:
break;
default:
return &vmd->irqs[0];
}
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 22:39:51 +07:00
raw_spin_lock_irqsave(&list_lock, flags);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
for (i = 1; i < vmd->msix_count; i++)
if (vmd->irqs[i].count < vmd->irqs[best].count)
best = i;
vmd->irqs[best].count++;
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 22:39:51 +07:00
raw_spin_unlock_irqrestore(&list_lock, flags);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
return &vmd->irqs[best];
}
static int vmd_msi_init(struct irq_domain *domain, struct msi_domain_info *info,
unsigned int virq, irq_hw_number_t hwirq,
msi_alloc_info_t *arg)
{
x86/PCI: VMD: Separate MSI and MSI-X vector sharing Child devices in a VMD domain that want to use MSI are slowing down MSI-X using devices sharing the same vectors. Move all MSI usage to a single VMD vector, and MSI-X devices can share the rest. Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Jon Derrick <jonathan.derrick@intel.com>
2016-06-20 22:39:53 +07:00
struct msi_desc *desc = arg->desc;
struct vmd_dev *vmd = vmd_from_bus(msi_desc_to_pci_dev(desc)->bus);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
struct vmd_irq *vmdirq = kzalloc(sizeof(*vmdirq), GFP_KERNEL);
x86/PCI: VMD: Eliminate index member from IRQ list Use math to discover the IRQ list index number relative to the IRQ list head. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-09-03 00:53:05 +07:00
unsigned int index, vector;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
if (!vmdirq)
return -ENOMEM;
INIT_LIST_HEAD(&vmdirq->node);
x86/PCI: VMD: Separate MSI and MSI-X vector sharing Child devices in a VMD domain that want to use MSI are slowing down MSI-X using devices sharing the same vectors. Move all MSI usage to a single VMD vector, and MSI-X devices can share the rest. Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Jon Derrick <jonathan.derrick@intel.com>
2016-06-20 22:39:53 +07:00
vmdirq->irq = vmd_next_irq(vmd, desc);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
vmdirq->virq = virq;
x86/PCI: VMD: Eliminate index member from IRQ list Use math to discover the IRQ list index number relative to the IRQ list head. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-09-03 00:53:05 +07:00
index = index_from_irqs(vmd, vmdirq->irq);
vector = pci_irq_vector(vmd->dev, index);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
x86/PCI: VMD: Eliminate index member from IRQ list Use math to discover the IRQ list index number relative to the IRQ list head. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-09-03 00:53:05 +07:00
irq_domain_set_info(domain, virq, vector, info->chip, vmdirq,
x86/PCI: VMD: Eliminate vmd_vector member from list type Eliminate unused vmd and vector members from vmd_irq_list and discover the vector using pci_irq_vector(). Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-09-03 00:53:04 +07:00
handle_untracked_irq, vmd, NULL);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
return 0;
}
static void vmd_msi_free(struct irq_domain *domain,
struct msi_domain_info *info, unsigned int virq)
{
struct vmd_irq *vmdirq = irq_get_chip_data(virq);
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 22:39:51 +07:00
unsigned long flags;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
PCI: vmd: Use SRCU as a local RCU to prevent delaying global RCU SRCU lets synchronize_srcu() depend on VMD-local RCU primitives, preventing long delays from locking up RCU in other systems. VMD performs a synchronize when removing a device, but will hit all IRQ lists if the device uses all VMD vectors. This patch will not help VMD's RCU synchronization, but will isolate the read side delays to the VMD subsystem. Additionally, the use of SRCU in VMD's ISR will keep it isolated from any other RCU waiters in the rest of the system. Tested using concurrent FIO and NVMe resets: [global] rw=read bs=4k direct=1 ioengine=libaio iodepth=32 norandommap timeout=300 runtime=1000000000 [nvme0] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme0n1 [nvme1] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme1n1 while (true) do for i in /sys/class/nvme/nvme*; do echo "Resetting ${i##*/}" echo 1 > $i/reset_controller; sleep 5 done; done Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-11-12 06:08:45 +07:00
synchronize_srcu(&vmdirq->irq->srcu);
x86/PCI: VMD: Synchronize with RCU freeing MSI IRQ descs Fix a potential race when disabling MSI/MSI-X on a VMD domain device. If the VMD interrupt service is running, it may see a disabled IRQ. We can synchronize RCU just before freeing the MSI descriptor. This is safe since the irq_desc lock isn't held, and the descriptor is valid even though it is disabled. After vmd_msi_free(), though, the handler is reinitialized to handle_bad_irq(), so we can't let the VMD ISR's list iteration see the disabled IRQ after this. Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by Jon Derrick: <jonathan.derrick@intel.com>
2016-08-05 05:09:09 +07:00
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
/* XXX: Potential optimization to rebalance */
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 22:39:51 +07:00
raw_spin_lock_irqsave(&list_lock, flags);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
vmdirq->irq->count--;
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 22:39:51 +07:00
raw_spin_unlock_irqrestore(&list_lock, flags);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
PCI: vmd: Use SRCU as a local RCU to prevent delaying global RCU SRCU lets synchronize_srcu() depend on VMD-local RCU primitives, preventing long delays from locking up RCU in other systems. VMD performs a synchronize when removing a device, but will hit all IRQ lists if the device uses all VMD vectors. This patch will not help VMD's RCU synchronization, but will isolate the read side delays to the VMD subsystem. Additionally, the use of SRCU in VMD's ISR will keep it isolated from any other RCU waiters in the rest of the system. Tested using concurrent FIO and NVMe resets: [global] rw=read bs=4k direct=1 ioengine=libaio iodepth=32 norandommap timeout=300 runtime=1000000000 [nvme0] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme0n1 [nvme1] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme1n1 while (true) do for i in /sys/class/nvme/nvme*; do echo "Resetting ${i##*/}" echo 1 > $i/reset_controller; sleep 5 done; done Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-11-12 06:08:45 +07:00
kfree(vmdirq);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
}
static int vmd_msi_prepare(struct irq_domain *domain, struct device *dev,
int nvec, msi_alloc_info_t *arg)
{
struct pci_dev *pdev = to_pci_dev(dev);
struct vmd_dev *vmd = vmd_from_bus(pdev->bus);
if (nvec > vmd->msix_count)
return vmd->msix_count;
memset(arg, 0, sizeof(*arg));
return 0;
}
static void vmd_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
{
arg->desc = desc;
}
static struct msi_domain_ops vmd_msi_domain_ops = {
.get_hwirq = vmd_get_hwirq,
.msi_init = vmd_msi_init,
.msi_free = vmd_msi_free,
.msi_prepare = vmd_msi_prepare,
.set_desc = vmd_set_desc,
};
static struct msi_domain_info vmd_msi_domain_info = {
.flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
MSI_FLAG_PCI_MSIX,
.ops = &vmd_msi_domain_ops,
.chip = &vmd_msi_controller,
};
static char __iomem *vmd_cfg_addr(struct vmd_dev *vmd, struct pci_bus *bus,
unsigned int devfn, int reg, int len)
{
char __iomem *addr = vmd->cfgbar +
PCI: vmd: Fix config addressing when using bus offsets VMD maps child device config spaces to the VMD Config BAR linearly regardless of the starting bus offset. Because of this, the config address decode must ignore starting bus offsets when mapping the BDF to the config space address. Fixes: 2a5a9c9a20f9 ("PCI: vmd: Add offset to bus numbers if necessary") Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: stable@vger.kernel.org # v5.2+
2019-09-16 20:54:34 +07:00
((bus->number - vmd->busn_start) << 20) +
(devfn << 12) + reg;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
if ((addr - vmd->cfgbar) + len >=
resource_size(&vmd->dev->resource[VMD_CFGBAR]))
return NULL;
return addr;
}
/*
* CPU may deadlock if config space is not serialized on some versions of this
* hardware, so all config space access is done under a spinlock.
*/
static int vmd_pci_read(struct pci_bus *bus, unsigned int devfn, int reg,
int len, u32 *value)
{
struct vmd_dev *vmd = vmd_from_bus(bus);
char __iomem *addr = vmd_cfg_addr(vmd, bus, devfn, reg, len);
unsigned long flags;
int ret = 0;
if (!addr)
return -EFAULT;
spin_lock_irqsave(&vmd->cfg_lock, flags);
switch (len) {
case 1:
*value = readb(addr);
break;
case 2:
*value = readw(addr);
break;
case 4:
*value = readl(addr);
break;
default:
ret = -EINVAL;
break;
}
spin_unlock_irqrestore(&vmd->cfg_lock, flags);
return ret;
}
/*
* VMD h/w converts non-posted config writes to posted memory writes. The
* read-back in this function forces the completion so it returns only after
* the config space was written, as expected.
*/
static int vmd_pci_write(struct pci_bus *bus, unsigned int devfn, int reg,
int len, u32 value)
{
struct vmd_dev *vmd = vmd_from_bus(bus);
char __iomem *addr = vmd_cfg_addr(vmd, bus, devfn, reg, len);
unsigned long flags;
int ret = 0;
if (!addr)
return -EFAULT;
spin_lock_irqsave(&vmd->cfg_lock, flags);
switch (len) {
case 1:
writeb(value, addr);
readb(addr);
break;
case 2:
writew(value, addr);
readw(addr);
break;
case 4:
writel(value, addr);
readl(addr);
break;
default:
ret = -EINVAL;
break;
}
spin_unlock_irqrestore(&vmd->cfg_lock, flags);
return ret;
}
static struct pci_ops vmd_ops = {
.read = vmd_pci_read,
.write = vmd_pci_write,
};
x86/PCI: VMD: Attach VMD resources to parent domain's resource tree Attach the new VMD domain's resources to the VMD device's resources. This allows /proc/iomem to display a more complete picture. Before: c0000000-c1ffffff : 0000:5d:05.5 c2000000-c3ffffff : 0000:5d:05.5 c2010000-c2013fff : nvme c4000000-c40fffff : 0000:5d:05.5 After: c0000000-c1ffffff : 0000:5d:05.5 c2000000-c3ffffff : 0000:5d:05.5 c2000000-c3ffffff : VMD MEMBAR1 c2000000-c22fffff : PCI Bus 10000:01 c2000000-c200ffff : 10000:01:00.0 c2010000-c2013fff : 10000:01:00.0 c2010000-c2013fff : nvme c2300000-c24fffff : PCI Bus 10000:01 c4000000-c40fffff : 0000:5d:05.5 c4002000-c40fffff : VMD MEMBAR2 Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-02-25 00:06:37 +07:00
static void vmd_attach_resources(struct vmd_dev *vmd)
{
vmd->dev->resource[VMD_MEMBAR1].child = &vmd->resources[1];
vmd->dev->resource[VMD_MEMBAR2].child = &vmd->resources[2];
}
static void vmd_detach_resources(struct vmd_dev *vmd)
{
vmd->dev->resource[VMD_MEMBAR1].child = NULL;
vmd->dev->resource[VMD_MEMBAR2].child = NULL;
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
/*
PCI: vmd: Correct comment: VMD domains start at 0x10000, not 0x1000 VMD domains are allocated starting at 0x10000, not 0x1000 as the comment said. Correct the comment and add a reference to the ACPI spec for _SEG. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2017-06-20 03:26:57 +07:00
* VMD domains start at 0x10000 to not clash with ACPI _SEG domains.
* Per ACPI r6.0, sec 6.5.6, _SEG returns an integer, of which the lower
* 16 bits are the PCI Segment Group (domain) number. Other bits are
* currently reserved.
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
*/
static int vmd_find_free_domain(void)
{
int domain = 0xffff;
struct pci_bus *bus = NULL;
while ((bus = pci_find_next_bus(bus)) != NULL)
domain = max_t(int, domain, pci_domain_nr(bus));
return domain + 1;
}
PCI: vmd: Assign membar addresses from shadow registers Certain VMD devices have registers within membar 2 which may shadow the membar 1 and membar 2 addresses. These are intended to be used in virtualization, where assigning a guest address wouldn't be translated in the assignment to root port and child devices because the addresses exist within the assignment message. These values will only reflect the membars when enabled in the BIOS, as determined by a register in the VMD device. This patch declares this option as a bit in the pci id driver_data, so that future conforming device ids can be enabled through sysfs new_id if necessary. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:59 +07:00
static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
{
struct pci_sysdata *sd = &vmd->sysdata;
PCI/vmd: Create named irq domain Use the fwnode to create a named domain so diagnosis works. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Jens Axboe <axboe@kernel.dk> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Keith Busch <keith.busch@intel.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: linux-pci@vger.kernel.org Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Christoph Hellwig <hch@lst.de> Link: http://lkml.kernel.org/r/20170619235444.379861978@linutronix.de
2017-06-20 06:37:15 +07:00
struct fwnode_handle *fn;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
struct resource *res;
u32 upper_bits;
unsigned long flags;
LIST_HEAD(resources);
PCI: vmd: Assign membar addresses from shadow registers Certain VMD devices have registers within membar 2 which may shadow the membar 1 and membar 2 addresses. These are intended to be used in virtualization, where assigning a guest address wouldn't be translated in the assignment to root port and child devices because the addresses exist within the assignment message. These values will only reflect the membars when enabled in the BIOS, as determined by a register in the VMD device. This patch declares this option as a bit in the pci id driver_data, so that future conforming device ids can be enabled through sysfs new_id if necessary. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:59 +07:00
resource_size_t offset[2] = {0};
PCI: vmd: Fix config addressing when using bus offsets VMD maps child device config spaces to the VMD Config BAR linearly regardless of the starting bus offset. Because of this, the config address decode must ignore starting bus offsets when mapping the BDF to the config space address. Fixes: 2a5a9c9a20f9 ("PCI: vmd: Add offset to bus numbers if necessary") Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: stable@vger.kernel.org # v5.2+
2019-09-16 20:54:34 +07:00
resource_size_t membar2_offset = 0x2000;
PCI/VMD: Configure MPS settings before adding devices In order to provide the most performance and/or compatible settings, ensure VMD root buses observe the pcie bus tuning settings by configuring those settings prior to adding the devices to the pcie tree. This patch open-codes pci_rescan_bus() and configures the buses prior to adding devices and attaching drivers. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2018-12-13 23:23:42 +07:00
struct pci_bus *child;
PCI: vmd: Assign membar addresses from shadow registers Certain VMD devices have registers within membar 2 which may shadow the membar 1 and membar 2 addresses. These are intended to be used in virtualization, where assigning a guest address wouldn't be translated in the assignment to root port and child devices because the addresses exist within the assignment message. These values will only reflect the membars when enabled in the BIOS, as determined by a register in the VMD device. This patch declares this option as a bit in the pci id driver_data, so that future conforming device ids can be enabled through sysfs new_id if necessary. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:59 +07:00
/*
* Shadow registers may exist in certain VMD device ids which allow
* guests to correctly assign host physical addresses to the root ports
* and child devices. These registers will either return the host value
* or 0, depending on an enable bit in the VMD device.
*/
if (features & VMD_FEAT_HAS_MEMBAR_SHADOW) {
u32 vmlock;
int ret;
PCI: vmd: Fix shadow offsets to reflect spec changes The shadow offset scratchpad was moved to 0x2000-0x2010. Update the location to get the correct shadow offset. Fixes: 6788958e4f3c ("PCI: vmd: Assign membar addresses from shadow registers") Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: stable@vger.kernel.org # v5.2+
2019-09-16 20:54:35 +07:00
membar2_offset = MB2_SHADOW_OFFSET + MB2_SHADOW_SIZE;
PCI: vmd: Assign membar addresses from shadow registers Certain VMD devices have registers within membar 2 which may shadow the membar 1 and membar 2 addresses. These are intended to be used in virtualization, where assigning a guest address wouldn't be translated in the assignment to root port and child devices because the addresses exist within the assignment message. These values will only reflect the membars when enabled in the BIOS, as determined by a register in the VMD device. This patch declares this option as a bit in the pci id driver_data, so that future conforming device ids can be enabled through sysfs new_id if necessary. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:59 +07:00
ret = pci_read_config_dword(vmd->dev, PCI_REG_VMLOCK, &vmlock);
if (ret || vmlock == ~0)
return -ENODEV;
if (MB2_SHADOW_EN(vmlock)) {
void __iomem *membar2;
membar2 = pci_iomap(vmd->dev, VMD_MEMBAR2, 0);
if (!membar2)
return -ENOMEM;
offset[0] = vmd->dev->resource[VMD_MEMBAR1].start -
PCI: vmd: Fix shadow offsets to reflect spec changes The shadow offset scratchpad was moved to 0x2000-0x2010. Update the location to get the correct shadow offset. Fixes: 6788958e4f3c ("PCI: vmd: Assign membar addresses from shadow registers") Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: stable@vger.kernel.org # v5.2+
2019-09-16 20:54:35 +07:00
readq(membar2 + MB2_SHADOW_OFFSET);
PCI: vmd: Assign membar addresses from shadow registers Certain VMD devices have registers within membar 2 which may shadow the membar 1 and membar 2 addresses. These are intended to be used in virtualization, where assigning a guest address wouldn't be translated in the assignment to root port and child devices because the addresses exist within the assignment message. These values will only reflect the membars when enabled in the BIOS, as determined by a register in the VMD device. This patch declares this option as a bit in the pci id driver_data, so that future conforming device ids can be enabled through sysfs new_id if necessary. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:59 +07:00
offset[1] = vmd->dev->resource[VMD_MEMBAR2].start -
PCI: vmd: Fix shadow offsets to reflect spec changes The shadow offset scratchpad was moved to 0x2000-0x2010. Update the location to get the correct shadow offset. Fixes: 6788958e4f3c ("PCI: vmd: Assign membar addresses from shadow registers") Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: stable@vger.kernel.org # v5.2+
2019-09-16 20:54:35 +07:00
readq(membar2 + MB2_SHADOW_OFFSET + 8);
PCI: vmd: Assign membar addresses from shadow registers Certain VMD devices have registers within membar 2 which may shadow the membar 1 and membar 2 addresses. These are intended to be used in virtualization, where assigning a guest address wouldn't be translated in the assignment to root port and child devices because the addresses exist within the assignment message. These values will only reflect the membars when enabled in the BIOS, as determined by a register in the VMD device. This patch declares this option as a bit in the pci id driver_data, so that future conforming device ids can be enabled through sysfs new_id if necessary. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:59 +07:00
pci_iounmap(vmd->dev, membar2);
}
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
PCI: vmd: Add offset to bus numbers if necessary Depending on platform configuration, certain VMD devices may have an additional configuration option which specifies the range of bus numbers allowed in a VMD PCIe domain. We determine this requirement by checking the value of two vendor specific config registers in the VMD endpoint: VMCAP[0] | VMCONFIG[9:8] | Bus Numbers ---------------------------------------- 0 | * | 0-255 1 | 00 | 0-127 1 | 01 | 128-255 1 | 10 | 0-255 This feature is also added as a bit in driver_data, to allow future conforming device ids which support these features to be enabled through sysfs new_id. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:28:00 +07:00
/*
* Certain VMD devices may have a root port configuration option which
PCI: vmd: Add bus 224-255 restriction decode VMD bus restrictions are required when IO fabric is multiplexed such that VMD cannot use the entire bus range. This patch adds another bus restriction decode bit that can be set by firmware to restrict the VMD bus range to between 224-255. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2019-11-12 19:47:52 +07:00
* limits the bus range to between 0-127, 128-255, or 224-255
PCI: vmd: Add offset to bus numbers if necessary Depending on platform configuration, certain VMD devices may have an additional configuration option which specifies the range of bus numbers allowed in a VMD PCIe domain. We determine this requirement by checking the value of two vendor specific config registers in the VMD endpoint: VMCAP[0] | VMCONFIG[9:8] | Bus Numbers ---------------------------------------- 0 | * | 0-255 1 | 00 | 0-127 1 | 01 | 128-255 1 | 10 | 0-255 This feature is also added as a bit in driver_data, to allow future conforming device ids which support these features to be enabled through sysfs new_id. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:28:00 +07:00
*/
if (features & VMD_FEAT_HAS_BUS_RESTRICTIONS) {
PCI: vmd: Add bus 224-255 restriction decode VMD bus restrictions are required when IO fabric is multiplexed such that VMD cannot use the entire bus range. This patch adds another bus restriction decode bit that can be set by firmware to restrict the VMD bus range to between 224-255. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2019-11-12 19:47:52 +07:00
u16 reg16;
pci_read_config_word(vmd->dev, PCI_REG_VMCAP, &reg16);
if (BUS_RESTRICT_CAP(reg16)) {
pci_read_config_word(vmd->dev, PCI_REG_VMCONFIG,
&reg16);
switch (BUS_RESTRICT_CFG(reg16)) {
case 1:
vmd->busn_start = 128;
break;
case 2:
vmd->busn_start = 224;
break;
case 3:
pci_err(vmd->dev, "Unknown Bus Offset Setting\n");
return -ENODEV;
default:
break;
}
}
PCI: vmd: Add offset to bus numbers if necessary Depending on platform configuration, certain VMD devices may have an additional configuration option which specifies the range of bus numbers allowed in a VMD PCIe domain. We determine this requirement by checking the value of two vendor specific config registers in the VMD endpoint: VMCAP[0] | VMCONFIG[9:8] | Bus Numbers ---------------------------------------- 0 | * | 0-255 1 | 00 | 0-127 1 | 01 | 128-255 1 | 10 | 0-255 This feature is also added as a bit in driver_data, to allow future conforming device ids which support these features to be enabled through sysfs new_id. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:28:00 +07:00
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
res = &vmd->dev->resource[VMD_CFGBAR];
vmd->resources[0] = (struct resource) {
.name = "VMD CFGBAR",
PCI: vmd: Fix config addressing when using bus offsets VMD maps child device config spaces to the VMD Config BAR linearly regardless of the starting bus offset. Because of this, the config address decode must ignore starting bus offsets when mapping the BDF to the config space address. Fixes: 2a5a9c9a20f9 ("PCI: vmd: Add offset to bus numbers if necessary") Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: stable@vger.kernel.org # v5.2+
2019-09-16 20:54:34 +07:00
.start = vmd->busn_start,
.end = vmd->busn_start + (resource_size(res) >> 20) - 1,
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
.flags = IORESOURCE_BUS | IORESOURCE_PCI_FIXED,
};
x86/PCI: VMD: Document code for maintainability Comment the less obvious portion of the code for setting up memory windows, and the platform dependency for initializing the h/w with appropriate resources. Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2016-03-03 05:31:03 +07:00
/*
* If the window is below 4GB, clear IORESOURCE_MEM_64 so we can
* put 32-bit resources in the window.
*
* There's no hardware reason why a 64-bit window *couldn't*
* contain a 32-bit resource, but pbus_size_mem() computes the
* bridge window size assuming a 64-bit window will contain no
* 32-bit resources. __pci_assign_resource() enforces that
* artificial restriction to make sure everything will fit.
*
PCI: Fix typos and whitespace errors Fix typos in drivers/pci. Comment and whitespace changes only. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
2019-05-30 20:05:58 +07:00
* The only way we could use a 64-bit non-prefetchable MEMBAR is
x86/PCI: VMD: Document code for maintainability Comment the less obvious portion of the code for setting up memory windows, and the platform dependency for initializing the h/w with appropriate resources. Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2016-03-03 05:31:03 +07:00
* if its address is <4GB so that we can convert it to a 32-bit
* resource. To be visible to the host OS, all VMD endpoints must
* be initially configured by platform BIOS, which includes setting
* up these resources. We can assume the device is configured
* according to the platform needs.
*/
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
res = &vmd->dev->resource[VMD_MEMBAR1];
upper_bits = upper_32_bits(res->end);
flags = res->flags & ~IORESOURCE_SIZEALIGN;
if (!upper_bits)
flags &= ~IORESOURCE_MEM_64;
vmd->resources[1] = (struct resource) {
.name = "VMD MEMBAR1",
.start = res->start,
.end = res->end,
.flags = flags,
x86/PCI: VMD: Attach VMD resources to parent domain's resource tree Attach the new VMD domain's resources to the VMD device's resources. This allows /proc/iomem to display a more complete picture. Before: c0000000-c1ffffff : 0000:5d:05.5 c2000000-c3ffffff : 0000:5d:05.5 c2010000-c2013fff : nvme c4000000-c40fffff : 0000:5d:05.5 After: c0000000-c1ffffff : 0000:5d:05.5 c2000000-c3ffffff : 0000:5d:05.5 c2000000-c3ffffff : VMD MEMBAR1 c2000000-c22fffff : PCI Bus 10000:01 c2000000-c200ffff : 10000:01:00.0 c2010000-c2013fff : 10000:01:00.0 c2010000-c2013fff : nvme c2300000-c24fffff : PCI Bus 10000:01 c4000000-c40fffff : 0000:5d:05.5 c4002000-c40fffff : VMD MEMBAR2 Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-02-25 00:06:37 +07:00
.parent = res,
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
};
res = &vmd->dev->resource[VMD_MEMBAR2];
upper_bits = upper_32_bits(res->end);
flags = res->flags & ~IORESOURCE_SIZEALIGN;
if (!upper_bits)
flags &= ~IORESOURCE_MEM_64;
vmd->resources[2] = (struct resource) {
.name = "VMD MEMBAR2",
PCI: vmd: Assign membar addresses from shadow registers Certain VMD devices have registers within membar 2 which may shadow the membar 1 and membar 2 addresses. These are intended to be used in virtualization, where assigning a guest address wouldn't be translated in the assignment to root port and child devices because the addresses exist within the assignment message. These values will only reflect the membars when enabled in the BIOS, as determined by a register in the VMD device. This patch declares this option as a bit in the pci id driver_data, so that future conforming device ids can be enabled through sysfs new_id if necessary. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:59 +07:00
.start = res->start + membar2_offset,
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
.end = res->end,
.flags = flags,
x86/PCI: VMD: Attach VMD resources to parent domain's resource tree Attach the new VMD domain's resources to the VMD device's resources. This allows /proc/iomem to display a more complete picture. Before: c0000000-c1ffffff : 0000:5d:05.5 c2000000-c3ffffff : 0000:5d:05.5 c2010000-c2013fff : nvme c4000000-c40fffff : 0000:5d:05.5 After: c0000000-c1ffffff : 0000:5d:05.5 c2000000-c3ffffff : 0000:5d:05.5 c2000000-c3ffffff : VMD MEMBAR1 c2000000-c22fffff : PCI Bus 10000:01 c2000000-c200ffff : 10000:01:00.0 c2010000-c2013fff : 10000:01:00.0 c2010000-c2013fff : nvme c2300000-c24fffff : PCI Bus 10000:01 c4000000-c40fffff : 0000:5d:05.5 c4002000-c40fffff : VMD MEMBAR2 Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-02-25 00:06:37 +07:00
.parent = res,
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
};
x86/PCI: Expose VMD's pci_dev in struct pci_sysdata Expose VMD's pci_dev pointer in struct pci_sysdata. This will be used indirectly by intel-iommu.c to find the correct domain. Link: https://lore.kernel.org/r/1579613871-301529-3-git-send-email-jonathan.derrick@intel.com [bhelgaas: commit log] Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Cc: Christoph Hellwig <hch@lst.de>
2020-01-21 20:37:46 +07:00
sd->vmd_dev = vmd->dev;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
sd->domain = vmd_find_free_domain();
if (sd->domain < 0)
return sd->domain;
sd->node = pcibus_to_node(vmd->dev->bus);
PCI/vmd: Create named irq domain Use the fwnode to create a named domain so diagnosis works. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Jens Axboe <axboe@kernel.dk> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Keith Busch <keith.busch@intel.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: linux-pci@vger.kernel.org Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Christoph Hellwig <hch@lst.de> Link: http://lkml.kernel.org/r/20170619235444.379861978@linutronix.de
2017-06-20 06:37:15 +07:00
fn = irq_domain_alloc_named_id_fwnode("VMD-MSI", vmd->sysdata.domain);
if (!fn)
return -ENODEV;
vmd->irq_domain = pci_msi_create_irq_domain(fn, &vmd_msi_domain_info,
x86/PCI: VMD: Use x86_vector_domain as parent domain Otherwise APIC code assumes VMD's IRQ domain can be managed by the APIC, resulting in an invalid cast of irq_data during irq_force_complete_move(). Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2016-06-20 22:39:52 +07:00
x86_vector_domain);
PCI/vmd: Create named irq domain Use the fwnode to create a named domain so diagnosis works. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Cc: Jens Axboe <axboe@kernel.dk> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Keith Busch <keith.busch@intel.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: linux-pci@vger.kernel.org Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Christoph Hellwig <hch@lst.de> Link: http://lkml.kernel.org/r/20170619235444.379861978@linutronix.de
2017-06-20 06:37:15 +07:00
irq_domain_free_fwnode(fn);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
if (!vmd->irq_domain)
return -ENODEV;
pci_add_resource(&resources, &vmd->resources[0]);
PCI: vmd: Assign membar addresses from shadow registers Certain VMD devices have registers within membar 2 which may shadow the membar 1 and membar 2 addresses. These are intended to be used in virtualization, where assigning a guest address wouldn't be translated in the assignment to root port and child devices because the addresses exist within the assignment message. These values will only reflect the membars when enabled in the BIOS, as determined by a register in the VMD device. This patch declares this option as a bit in the pci id driver_data, so that future conforming device ids can be enabled through sysfs new_id if necessary. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:59 +07:00
pci_add_resource_offset(&resources, &vmd->resources[1], offset[0]);
pci_add_resource_offset(&resources, &vmd->resources[2], offset[1]);
PCI: vmd: Fix config addressing when using bus offsets VMD maps child device config spaces to the VMD Config BAR linearly regardless of the starting bus offset. Because of this, the config address decode must ignore starting bus offsets when mapping the BDF to the config space address. Fixes: 2a5a9c9a20f9 ("PCI: vmd: Add offset to bus numbers if necessary") Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: stable@vger.kernel.org # v5.2+
2019-09-16 20:54:34 +07:00
vmd->bus = pci_create_root_bus(&vmd->dev->dev, vmd->busn_start,
&vmd_ops, sd, &resources);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
if (!vmd->bus) {
pci_free_resource_list(&resources);
irq_domain_remove(vmd->irq_domain);
return -ENODEV;
}
x86/PCI: VMD: Attach VMD resources to parent domain's resource tree Attach the new VMD domain's resources to the VMD device's resources. This allows /proc/iomem to display a more complete picture. Before: c0000000-c1ffffff : 0000:5d:05.5 c2000000-c3ffffff : 0000:5d:05.5 c2010000-c2013fff : nvme c4000000-c40fffff : 0000:5d:05.5 After: c0000000-c1ffffff : 0000:5d:05.5 c2000000-c3ffffff : 0000:5d:05.5 c2000000-c3ffffff : VMD MEMBAR1 c2000000-c22fffff : PCI Bus 10000:01 c2000000-c200ffff : 10000:01:00.0 c2010000-c2013fff : 10000:01:00.0 c2010000-c2013fff : nvme c2300000-c24fffff : PCI Bus 10000:01 c4000000-c40fffff : 0000:5d:05.5 c4002000-c40fffff : VMD MEMBAR2 Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-02-25 00:06:37 +07:00
vmd_attach_resources(vmd);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
dev_set_msi_domain(&vmd->bus->dev, vmd->irq_domain);
PCI/VMD: Configure MPS settings before adding devices In order to provide the most performance and/or compatible settings, ensure VMD root buses observe the pcie bus tuning settings by configuring those settings prior to adding the devices to the pcie tree. This patch open-codes pci_rescan_bus() and configures the buses prior to adding devices and attaching drivers. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2018-12-13 23:23:42 +07:00
pci_scan_child_bus(vmd->bus);
pci_assign_unassigned_bus_resources(vmd->bus);
/*
* VMD root buses are virtual and don't return true on pci_is_pcie()
* and will fail pcie_bus_configure_settings() early. It can instead be
* run on each of the real root ports.
*/
list_for_each_entry(child, &vmd->bus->children, node)
pcie_bus_configure_settings(child);
pci_bus_add_devices(vmd->bus);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
WARN(sysfs_create_link(&vmd->dev->dev.kobj, &vmd->bus->dev.kobj,
"domain"), "Can't create symlink to domain\n");
return 0;
}
static irqreturn_t vmd_irq(int irq, void *data)
{
struct vmd_irq_list *irqs = data;
struct vmd_irq *vmdirq;
PCI: vmd: Use SRCU as a local RCU to prevent delaying global RCU SRCU lets synchronize_srcu() depend on VMD-local RCU primitives, preventing long delays from locking up RCU in other systems. VMD performs a synchronize when removing a device, but will hit all IRQ lists if the device uses all VMD vectors. This patch will not help VMD's RCU synchronization, but will isolate the read side delays to the VMD subsystem. Additionally, the use of SRCU in VMD's ISR will keep it isolated from any other RCU waiters in the rest of the system. Tested using concurrent FIO and NVMe resets: [global] rw=read bs=4k direct=1 ioengine=libaio iodepth=32 norandommap timeout=300 runtime=1000000000 [nvme0] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme0n1 [nvme1] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme1n1 while (true) do for i in /sys/class/nvme/nvme*; do echo "Resetting ${i##*/}" echo 1 > $i/reset_controller; sleep 5 done; done Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-11-12 06:08:45 +07:00
int idx;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
PCI: vmd: Use SRCU as a local RCU to prevent delaying global RCU SRCU lets synchronize_srcu() depend on VMD-local RCU primitives, preventing long delays from locking up RCU in other systems. VMD performs a synchronize when removing a device, but will hit all IRQ lists if the device uses all VMD vectors. This patch will not help VMD's RCU synchronization, but will isolate the read side delays to the VMD subsystem. Additionally, the use of SRCU in VMD's ISR will keep it isolated from any other RCU waiters in the rest of the system. Tested using concurrent FIO and NVMe resets: [global] rw=read bs=4k direct=1 ioengine=libaio iodepth=32 norandommap timeout=300 runtime=1000000000 [nvme0] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme0n1 [nvme1] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme1n1 while (true) do for i in /sys/class/nvme/nvme*; do echo "Resetting ${i##*/}" echo 1 > $i/reset_controller; sleep 5 done; done Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-11-12 06:08:45 +07:00
idx = srcu_read_lock(&irqs->srcu);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
list_for_each_entry_rcu(vmdirq, &irqs->irq_list, node)
generic_handle_irq(vmdirq->virq);
PCI: vmd: Use SRCU as a local RCU to prevent delaying global RCU SRCU lets synchronize_srcu() depend on VMD-local RCU primitives, preventing long delays from locking up RCU in other systems. VMD performs a synchronize when removing a device, but will hit all IRQ lists if the device uses all VMD vectors. This patch will not help VMD's RCU synchronization, but will isolate the read side delays to the VMD subsystem. Additionally, the use of SRCU in VMD's ISR will keep it isolated from any other RCU waiters in the rest of the system. Tested using concurrent FIO and NVMe resets: [global] rw=read bs=4k direct=1 ioengine=libaio iodepth=32 norandommap timeout=300 runtime=1000000000 [nvme0] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme0n1 [nvme1] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme1n1 while (true) do for i in /sys/class/nvme/nvme*; do echo "Resetting ${i##*/}" echo 1 > $i/reset_controller; sleep 5 done; done Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-11-12 06:08:45 +07:00
srcu_read_unlock(&irqs->srcu, idx);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
return IRQ_HANDLED;
}
static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
{
struct vmd_dev *vmd;
int i, err;
if (resource_size(&dev->resource[VMD_CFGBAR]) < (1 << 20))
return -ENOMEM;
vmd = devm_kzalloc(&dev->dev, sizeof(*vmd), GFP_KERNEL);
if (!vmd)
return -ENOMEM;
vmd->dev = dev;
err = pcim_enable_device(dev);
if (err < 0)
return err;
vmd->cfgbar = pcim_iomap(dev, VMD_CFGBAR, 0);
if (!vmd->cfgbar)
return -ENOMEM;
pci_set_master(dev);
if (dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(64)) &&
dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(32)))
return -ENODEV;
vmd->msix_count = pci_msix_vec_count(dev);
if (vmd->msix_count < 0)
return -ENODEV;
PCI: vmd: Remove IRQ affinity so we can allocate more IRQs VMD hardware has to share its vectors among child devices in its PCI domain so we should allocate as many as possible rather than just ones that can be affinitized. pci_alloc_irq_vectors_affinity() limits the number of affinitized IRQs to the number of present CPUs (see irq_calc_affinity_vectors()). But we'd prefer to have more vectors, even if they aren't distributed across the CPUs, so use pci_alloc_irq_vectors() instead. Reported-by: Brad Goodman <Bradley.Goodman@dell.com> Signed-off-by: Keith Busch <keith.busch@intel.com> [bhelgaas: add irq_calc_affinity_vectors() reference to changelog] Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-08-30 23:15:04 +07:00
vmd->msix_count = pci_alloc_irq_vectors(dev, 1, vmd->msix_count,
PCI_IRQ_MSIX);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
if (vmd->msix_count < 0)
return vmd->msix_count;
vmd->irqs = devm_kcalloc(&dev->dev, vmd->msix_count, sizeof(*vmd->irqs),
GFP_KERNEL);
if (!vmd->irqs)
return -ENOMEM;
for (i = 0; i < vmd->msix_count; i++) {
PCI: vmd: Use SRCU as a local RCU to prevent delaying global RCU SRCU lets synchronize_srcu() depend on VMD-local RCU primitives, preventing long delays from locking up RCU in other systems. VMD performs a synchronize when removing a device, but will hit all IRQ lists if the device uses all VMD vectors. This patch will not help VMD's RCU synchronization, but will isolate the read side delays to the VMD subsystem. Additionally, the use of SRCU in VMD's ISR will keep it isolated from any other RCU waiters in the rest of the system. Tested using concurrent FIO and NVMe resets: [global] rw=read bs=4k direct=1 ioengine=libaio iodepth=32 norandommap timeout=300 runtime=1000000000 [nvme0] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme0n1 [nvme1] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme1n1 while (true) do for i in /sys/class/nvme/nvme*; do echo "Resetting ${i##*/}" echo 1 > $i/reset_controller; sleep 5 done; done Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-11-12 06:08:45 +07:00
err = init_srcu_struct(&vmd->irqs[i].srcu);
if (err)
return err;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
INIT_LIST_HEAD(&vmd->irqs[i].irq_list);
x86/PCI: VMD: Eliminate vmd_vector member from list type Eliminate unused vmd and vector members from vmd_irq_list and discover the vector using pci_irq_vector(). Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-09-03 00:53:04 +07:00
err = devm_request_irq(&dev->dev, pci_irq_vector(dev, i),
PCI: host: Mark PCIe/PCI (MSI) cascade ISR as IRQF_NO_THREAD Similar as commit 8ff0ef996ca0 ("PCI: host: Mark PCIe/PCI (MSI) IRQ cascade handlers as IRQF_NO_THREAD"), we should mark PCIe/PCI (MSI) IRQ cascade handlers in designware, qcom, and vmd as IRQF_NO_THREAD explicitly. Signed-off-by: Jisheng Zhang <jszhang@marvell.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com> # vmd Acked-by: Jingoo Han <jingoohan1@gmail.com> # pcie-designware-plat.c
2017-04-20 17:27:18 +07:00
vmd_irq, IRQF_NO_THREAD,
"vmd", &vmd->irqs[i]);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
if (err)
return err;
}
spin_lock_init(&vmd->cfg_lock);
pci_set_drvdata(dev, vmd);
PCI: vmd: Assign membar addresses from shadow registers Certain VMD devices have registers within membar 2 which may shadow the membar 1 and membar 2 addresses. These are intended to be used in virtualization, where assigning a guest address wouldn't be translated in the assignment to root port and child devices because the addresses exist within the assignment message. These values will only reflect the membars when enabled in the BIOS, as determined by a register in the VMD device. This patch declares this option as a bit in the pci id driver_data, so that future conforming device ids can be enabled through sysfs new_id if necessary. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:59 +07:00
err = vmd_enable_domain(vmd, (unsigned long) id->driver_data);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
if (err)
return err;
dev_info(&vmd->dev->dev, "Bound to PCI domain %04x\n",
vmd->sysdata.domain);
return 0;
}
PCI: vmd: Use SRCU as a local RCU to prevent delaying global RCU SRCU lets synchronize_srcu() depend on VMD-local RCU primitives, preventing long delays from locking up RCU in other systems. VMD performs a synchronize when removing a device, but will hit all IRQ lists if the device uses all VMD vectors. This patch will not help VMD's RCU synchronization, but will isolate the read side delays to the VMD subsystem. Additionally, the use of SRCU in VMD's ISR will keep it isolated from any other RCU waiters in the rest of the system. Tested using concurrent FIO and NVMe resets: [global] rw=read bs=4k direct=1 ioengine=libaio iodepth=32 norandommap timeout=300 runtime=1000000000 [nvme0] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme0n1 [nvme1] cpus_allowed=0-63 numjobs=8 filename=/dev/nvme1n1 while (true) do for i in /sys/class/nvme/nvme*; do echo "Resetting ${i##*/}" echo 1 > $i/reset_controller; sleep 5 done; done Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-11-12 06:08:45 +07:00
static void vmd_cleanup_srcu(struct vmd_dev *vmd)
{
int i;
for (i = 0; i < vmd->msix_count; i++)
cleanup_srcu_struct(&vmd->irqs[i].srcu);
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
static void vmd_remove(struct pci_dev *dev)
{
struct vmd_dev *vmd = pci_get_drvdata(dev);
sysfs_remove_link(&vmd->dev->dev.kobj, "domain");
pci_stop_root_bus(vmd->bus);
pci_remove_root_bus(vmd->bus);
PCI: vmd: Move SRCU cleanup after bus, child device removal Recent __call_srcu() changes have exposed that we need to cleanup SRCU structures after pci_stop_root_bus() calls into vmd_msi_free(). Fixes: 3906b91844d6 ("PCI: vmd: Use SRCU as a local RCU to prevent delaying global RCU") Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com> Cc: <stable@vger.kernel.org> # 4.11
2017-06-22 22:15:42 +07:00
vmd_cleanup_srcu(vmd);
PCI: vmd: Detach resources after stopping root bus The VMD removal path calls pci_stop_root_busi(), which tears down the pcie tree, including detaching all of the attached drivers. During driver detachment, devices may use pci_release_region() to release resources. This path relies on the resource being accessible in resource tree. By detaching the child domain from the parent resource domain prior to stopping the bus, we are preventing the list traversal from finding the resource to be freed. If we instead detach the resource after stopping the bus, we will have properly freed the resource and detaching is simply accounting at that point. Without this order, the resource is never freed and is orphaned on VMD removal, leading to a warning: [ 181.940162] Trying to free nonexistent resource <e5a10000-e5a13fff> Fixes: 2c2c5c5cd213 ("x86/PCI: VMD: Attach VMD resources to parent domain's resource tree") Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit log] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Reviewed-by: Keith Busch <keith.busch@intel.com>
2018-10-16 07:48:07 +07:00
vmd_detach_resources(vmd);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
irq_domain_remove(vmd->irq_domain);
}
PCI: vmd: Fix suspend handlers defined-but-not-used warning Fix the following warnings: drivers/pci/host/vmd.c:731:12: warning: ‘vmd_suspend’ defined but not used [-Wunused-function] static int vmd_suspend(struct device *dev) ^ drivers/pci/host/vmd.c:739:12: warning: ‘vmd_resume’ defined but not used [-Wunused-function] static int vmd_resume(struct device *dev) ^ Signed-off-by: Borislav Petkov <bp@suse.de> Signed-off-by: Bjorn Helgaas <helgaas@kernel.org> Reviewed-by: Keith Busch <keith.busch@intel.com>
2016-11-27 01:29:57 +07:00
#ifdef CONFIG_PM_SLEEP
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
static int vmd_suspend(struct device *dev)
{
struct pci_dev *pdev = to_pci_dev(dev);
PCI: vmd: Free up IRQs on suspend path Free up the IRQs we request on the suspend path and reallocate them on the resume path. Fixes this error: CPU 111 disable failed: CPU has 9 vectors assigned and there are only 0 available. Error taking CPU111 down: -34 Non-boot CPUs are not disabled Enabling non-boot CPUs ... Signed-off-by: Scott Bauer <scott.bauer@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2017-08-12 03:54:32 +07:00
struct vmd_dev *vmd = pci_get_drvdata(pdev);
int i;
for (i = 0; i < vmd->msix_count; i++)
PCI: Fix indentation Adjust indentation from spaces to tab (+optional two spaces) as in coding style with command like: $ sed -e 's/^ /\t/' -i */Kconfig [bhelgaas: do same in vmd.c] Link: https://lore.kernel.org/r/20191120134036.14502-1-krzk@kernel.org Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-11-20 20:40:36 +07:00
devm_free_irq(dev, pci_irq_vector(pdev, i), &vmd->irqs[i]);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
pci_save_state(pdev);
return 0;
}
static int vmd_resume(struct device *dev)
{
struct pci_dev *pdev = to_pci_dev(dev);
PCI: vmd: Free up IRQs on suspend path Free up the IRQs we request on the suspend path and reallocate them on the resume path. Fixes this error: CPU 111 disable failed: CPU has 9 vectors assigned and there are only 0 available. Error taking CPU111 down: -34 Non-boot CPUs are not disabled Enabling non-boot CPUs ... Signed-off-by: Scott Bauer <scott.bauer@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2017-08-12 03:54:32 +07:00
struct vmd_dev *vmd = pci_get_drvdata(pdev);
int err, i;
for (i = 0; i < vmd->msix_count; i++) {
err = devm_request_irq(dev, pci_irq_vector(pdev, i),
vmd_irq, IRQF_NO_THREAD,
"vmd", &vmd->irqs[i]);
if (err)
return err;
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
pci_restore_state(pdev);
return 0;
}
#endif
static SIMPLE_DEV_PM_OPS(vmd_dev_pm_ops, vmd_suspend, vmd_resume);
static const struct pci_device_id vmd_ids[] = {
PCI: Add Intel VMD devices to pci ids Add the Intel VMD device ids to the pci id database and update the VMD driver. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:27:58 +07:00
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_201D),},
PCI: vmd: Add an additional VMD device id to driver device id table Allow VMD devices with PCI id 8086:28c0 to bind to VMD driver. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> [lorenzo.pieralisi@arm.com: updated commit subject] Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2018-05-19 02:28:02 +07:00
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_28C0),
.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW |
VMD_FEAT_HAS_BUS_RESTRICTIONS,},
PCI: vmd: Add two VMD Device IDs Add new VMD device IDs that require the bus restriction mode. Signed-off-by: Sushma Kalakota <sushmax.kalakota@intel.com> Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2020-01-09 05:05:10 +07:00
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x467f),
.driver_data = VMD_FEAT_HAS_BUS_RESTRICTIONS,},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x4c3d),
.driver_data = VMD_FEAT_HAS_BUS_RESTRICTIONS,},
PCI: vmd: Add device id for VMD device 8086:9A0B This patch adds support for this VMD device which supports the bus restriction mode. Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
2019-11-12 19:47:53 +07:00
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B),
.driver_data = VMD_FEAT_HAS_BUS_RESTRICTIONS,},
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-13 03:18:10 +07:00
{0,}
};
MODULE_DEVICE_TABLE(pci, vmd_ids);
static struct pci_driver vmd_drv = {
.name = "vmd",
.id_table = vmd_ids,
.probe = vmd_probe,
.remove = vmd_remove,
.driver = {
.pm = &vmd_dev_pm_ops,
},
};
module_pci_driver(vmd_drv);
MODULE_AUTHOR("Intel Corporation");
MODULE_LICENSE("GPL v2");
MODULE_VERSION("0.6");
Reference in New Issue Copy Permalink