From 9a202d80f7b60fcf6caf059570bfd2d0b0cbe8a4 Mon Sep 17 00:00:00 2001 Message-Id: <9a202d80f7b60fcf6caf059570bfd2d0b0cbe8a4@dist-git> From: Erik Skultety Date: Thu, 18 May 2017 14:02:55 +0200 Subject: [PATCH] docs: Document the mediated devices within the nodedev driver https://bugzilla.redhat.com/show_bug.cgi?id=1452072 Signed-off-by: Erik Skultety (cherry picked from commit a0a0b3cf71cbee1df40dd0fdebd7bb6a84682f09) Signed-off-by: Erik Skultety Signed-off-by: Jiri Denemark --- docs/drvnodedev.html.in | 168 +++++++++++++++++++++++++++++++++++++++++++++++- tools/virsh.pod | 7 +- 2 files changed, 171 insertions(+), 4 deletions(-) diff --git a/docs/drvnodedev.html.in b/docs/drvnodedev.html.in index 0a3870343..26c52dd0d 100644 --- a/docs/drvnodedev.html.in +++ b/docs/drvnodedev.html.in @@ -9,7 +9,7 @@ (historically also referred to as node devices) like USB, PCI, SCSI, and network devices. This also includes various virtualization capabilities which the aforementioned devices provide for utilization, for example - SR-IOV, NPIV, DRM, etc. + SR-IOV, NPIV, MDEV, DRM, etc.

@@ -75,6 +75,7 @@ storage (Since 1.0.4), scsi_generic (Since 1.0.7), drm (Since 3.1.0), and + mdev (Since 3.4.0). This element can be nested in which case it further specifies a device's capability. Refer to specific device types to see more values for the type attribute which are exclusive. @@ -185,5 +186,170 @@ ... <device> +

MDEV capability

+

+ A PCI device capable of creating mediated devices will include a nested + capability mdev_types which enumerates all supported mdev + types on the physical device, along with the type attributes available + through sysfs: +

+ +
+
type
+
+ This element describes a mediated device type which acts as an + abstract template defining a resource allocation for instances of this + device type. The element has one attribute id which holds + an official vendor-supplied identifier for the type. + Since 3.4.0 +
+ +
name
+
+ The name element holds a vendor-supplied code name for + the given mediated device type. This is an optional element. + Since 3.4.0 +
+ +
deviceAPI
+
+ The value of this element describes how an instance of the given type + will be presented to the guest by the VFIO framework. + Since 3.4.0 +
+ +
availableInstances
+
+ This element reports the current state of resource allocation. In other + words, how many instances of the given type can still be successfully + created on the physical device. + Since 3.4.0 +
+
+ +

+ For a more info about mediated devices, refer to the + paragraph below. +

+ +
+<device>
+...
+  <driver>
+    <name>nvidia</name>
+  </driver>
+  <capability type='pci'>
+...
+    <capability type='mdev_types'>
+      <type id='nvidia-11'>
+        <name>GRID M60-0B</name>
+        <deviceAPI>vfio-pci</deviceAPI>
+        <availableInstances>16</availableInstances>
+      </type>
+      <!-- Here would come the rest of the available mdev types -->
+    </capability>
+...
+  </capability>
+</device>
+ +

Mediated devices (MDEVs)

+

+ Mediated devices (Since 3.2.0) are software + devices defining resource allocation on the backing physical device which + in turn allows the parent physical device's resources to be divided into + several mediated devices, thus sharing the physical device's performance + among multiple guests. Unlike SR-IOV however, where a PCIe device appears + as multiple separate PCIe devices on the host's PCI bus, mediated devices + only appear on the mdev virtual bus. Therefore, no detach/reattach + procedure from/to the host driver procedure is involved even though + mediated devices are used in a direct device assignment manner. +

+ +

+ The following sub-elements and attributes are exposed within the + capability element: +

+ +
+
type
+
+ This element describes a mediated device type which acts as an + abstract template defining a resource allocation for instances of this + device type. The element has one attribute id which holds + an official vendor-supplied identifier for the type. + Since 3.4.0 +
+ +
iommuGroup
+
+ This element supports a single attribute number which holds + the IOMMU group number the mediated device belongs to. + Since 3.4.0 +
+
+ +

Example of a mediated device

+ 
+<device>
+  <name>mdev_4b20d080_1b54_4048_85b3_a6a62d165c01</name>
+  <path>/sys/devices/pci0000:00/0000:00:02.0/4b20d080-1b54-4048-85b3-a6a62d165c01</path>
+  <parent>pci_0000_06_00_0</parent>
+  <driver>
+    <name>vfio_mdev</name>
+  </driver>
+  <capability type='mdev'>
+    <type id='nvidia-11'/>
+    <iommuGroup number='12'/>
+  <capability/>
+<device/>
+ +

+ The support of mediated device's framework in libvirt's node device driver + covers the following features: +

+ +
    +
  • + list available mediated devices on the host + (Since 3.4.0) +
    • +
  • + display device details + (Since 3.4.0) +
    • +
+ +

+ Because mediated devices are instantiated from vendor specific templates, + simply called 'types', information describing these types is contained + within the parent device's capabilities + (see the example in PCI host devices). +

+ +

+ To see the supported mediated device types on a specific physical device + use the following: +

+ + 
+$ ls /sys/class/mdev_bus/<device>/mdev_supported_types
+ +

+ To manually instantiate a mediated device, use one of the following as a + reference: +

+ + 
+$ uuidgen > /sys/class/mdev_bus/<device>/mdev_supported_types/<type>/create
+...
+$ echo <UUID> > /sys/class/mdev_bus/<device>/mdev_supported_types/<type>/create
+ +

+ Manual removal of a mediated device is then performed as follows: +

+ + 
+$ echo 1 > /sys/bus/mdev/devices/<uuid>/remove
+ diff --git a/tools/virsh.pod b/tools/virsh.pod index 0dd1fc7b2..11bacefa9 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -3155,10 +3155,11 @@ for HBA). List all of the devices available on the node that are known by libvirt. I is used to filter the list by capability types, the types must be -separated by comma, e.g. --cap pci,scsi, valid capability types include +separated by comma, e.g. --cap pci,scsi. Valid capability types include 'system', 'pci', 'usb_device', 'usb', 'net', 'scsi_host', 'scsi_target', -'scsi', 'storage', 'fc_host', 'vports', 'scsi_generic', 'drm'.If I<--tree> -is used, the output is formatted in a tree representing parents of each +'scsi', 'storage', 'fc_host', 'vports', 'scsi_generic', 'drm', 'mdev', +'mdev_types'. +If I<--tree> is used, the output is formatted in a tree representing parents of each node. I and I<--tree> are mutually exclusive. =item B I -- 2.13.0