linux_dsm_epyc7002/Documentation/mic/mic_overview.txt

67 lines
4.3 KiB
Plaintext
Raw Normal View History

An Intel MIC X100 device is a PCIe form factor add-in coprocessor
card based on the Intel Many Integrated Core (MIC) architecture
that runs a Linux OS. It is a PCIe endpoint in a platform and therefore
implements the three required standard address spaces i.e. configuration,
memory and I/O. The host OS loads a device driver as is typical for
PCIe devices. The card itself runs a bootstrap after reset that
misc: mic: Enable OSPM suspend and resume support. This patch enables support for OSPM suspend and resume in the MIC driver. During a host suspend event, the driver performs an orderly shutdown of the cards if they are online. Upon resume, any cards that were previously online before suspend are rebooted. The driver performs an orderly shutdown of the card primarily to ensure that applications in the card are terminated and mounted devices are safely un-mounted before the card is powered down in the event of an OSPM suspend. The driver makes use of the MIC daemon to accomplish OSPM suspend and resume. The driver registers a PM notifier per MIC device. The devices get notified synchronously during PM_SUSPEND_PREPARE and PM_POST_SUSPEND phases. During the PM_SUSPEND_PREPARE phase, the driver performs one of the following three tasks. 1) If the card is 'offline', the driver sets the card to a 'suspended' state and returns. 2) If the card is 'online', the driver initiates card shutdown by setting the card state to suspending. This notifies the MIC daemon which invokes shutdown and sets card state to 'suspended'. The driver returns after the shutdown is complete. 3) If the card is already being shutdown, possibly by a host user space application, the driver sets the card state to 'suspended' and returns after the shutdown is complete. During the PM_POST_SUSPEND phase, the driver simply notifies the daemon and returns. The daemon boots those cards that were previously online during the suspend phase. Signed-off-by: Ashutosh Dixit <ashutosh.dixit@intel.com> Signed-off-by: Nikhil Rao <nikhil.rao@intel.com> Signed-off-by: Harshavardhan R Kharche <harshavardhan.r.kharche@intel.com> Signed-off-by: Sudeep Dutt <sudeep.dutt@intel.com> Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli@intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2013-10-04 08:06:23 +07:00
transfers control to the card OS downloaded from the host driver. The
host driver supports OSPM suspend and resume operations. It shuts down
the card during suspend and reboots the card OS during resume.
The card OS as shipped by Intel is a Linux kernel with modifications
for the X100 devices.
Since it is a PCIe card, it does not have the ability to host hardware
devices for networking, storage and console. We provide these devices
on X100 coprocessors thus enabling a self-bootable equivalent environment
for applications. A key benefit of our solution is that it leverages
the standard virtio framework for network, disk and console devices,
though in our case the virtio framework is used across a PCIe bus.
MIC PCIe card has a dma controller with 8 channels. These channels are
shared between the host s/w and the card s/w. 0 to 3 are used by host
and 4 to 7 by card. As the dma device doesn't show up as PCIe device,
a virtual bus called mic bus is created and virtual dma devices are
created on it by the host/card drivers. On host the channels are private
and used only by the host driver to transfer data for the virtio devices.
Here is a block diagram of the various components described above. The
virtio backends are situated on the host rather than the card given better
single threaded performance for the host compared to MIC, the ability of
the host to initiate DMA's to/from the card using the MIC DMA engine and
the fact that the virtio block storage backend can only be on the host.
|
+----------+ | +----------+
| Card OS | | | Host OS |
+----------+ | +----------+
|
+-------+ +--------+ +------+ | +---------+ +--------+ +--------+
| Virtio| |Virtio | |Virtio| | |Virtio | |Virtio | |Virtio |
| Net | |Console | |Block | | |Net | |Console | |Block |
| Driver| |Driver | |Driver| | |backend | |backend | |backend |
+-------+ +--------+ +------+ | +---------+ +--------+ +--------+
| | | | | | |
| | | |User | | |
| | | |------|------------|---------|-------
+-------------------+ |Kernel +--------------------------+
| | | Virtio over PCIe IOCTLs |
| | +--------------------------+
+-----------+ | | | +-----------+
| MIC DMA | | | | | MIC DMA |
| Driver | | | | | Driver |
+-----------+ | | | +-----------+
| | | | |
+---------------+ | | | +----------------+
|MIC virtual Bus| | | | |MIC virtual Bus |
+---------------+ | | | +----------------+
| | | | |
| +--------------+ | +---------------+ |
| |Intel MIC | | |Intel MIC | |
+---|Card Driver | | |Host Driver | |
+--------------+ | +---------------+-----+
| | |
+-------------------------------------------------------------+
| |
| PCIe Bus |
+-------------------------------------------------------------+