yeahuh / rpms / qemu-kvm

Forked from rpms/qemu-kvm 2 years ago
Clone
6e7d01
From f9d332b1280cd3f6009b59323719548a36a7c52b Mon Sep 17 00:00:00 2001
6e7d01
From: Daniel Henrique Barboza <dbarboza@redhat.com>
6e7d01
Date: Mon, 21 Jun 2021 14:40:24 -0400
6e7d01
Subject: [PATCH 2/4] spapr: Fix EEH capability issue on KVM guest for PCI
6e7d01
 passthru
6e7d01
6e7d01
RH-Author: Daniel Henrique Barboza <dbarboza@redhat.com>
6e7d01
Message-id: <20210621144024.199732-2-dbarboza@redhat.com>
6e7d01
Patchwork-id: 101740
6e7d01
O-Subject: [RHEL-8.5.0 qemu-kvm PATCH 1/1] spapr: Fix EEH capability issue on KVM guest for PCI passthru
6e7d01
Bugzilla: 1957866
6e7d01
RH-Acked-by: Laurent Vivier <lvivier@redhat.com>
6e7d01
RH-Acked-by: Greg Kurz <gkurz@redhat.com>
6e7d01
RH-Acked-by: David Gibson <dgibson@redhat.com>
6e7d01
6e7d01
From: Mahesh Salgaonkar <mahesh@linux.ibm.com>
6e7d01
6e7d01
With upstream kernel, especially after commit 98ba956f6a389
6e7d01
("powerpc/pseries/eeh: Rework device EEH PE determination") we see that KVM
6e7d01
guest isn't able to enable EEH option for PCI pass-through devices anymore.
6e7d01
6e7d01
[root@atest-guest ~]# dmesg | grep EEH
6e7d01
[    0.032337] EEH: pSeries platform initialized
6e7d01
[    0.298207] EEH: No capable adapters found: recovery disabled.
6e7d01
[root@atest-guest ~]#
6e7d01
6e7d01
So far the linux kernel was assuming pe_config_addr equal to device's
6e7d01
config_addr and using it to enable EEH on the PE through ibm,set-eeh-option
6e7d01
RTAS call. Which wasn't the correct way as per PAPR. The linux kernel
6e7d01
commit 98ba956f6a389 fixed this flow. With that fixed, linux now uses PE
6e7d01
config address returned by ibm,get-config-addr-info2 RTAS call to enable
6e7d01
EEH option per-PE basis instead of per-device basis. However this has
6e7d01
uncovered a bug in qemu where ibm,set-eeh-option is treating PE config
6e7d01
address as per-device config address.
6e7d01
6e7d01
Hence in qemu guest with recent kernel the ibm,set-eeh-option RTAS call
6e7d01
fails with -3 return value indicating that there is no PCI device exist for
6e7d01
the specified PE config address. The rtas_ibm_set_eeh_option call uses
6e7d01
pci_find_device() to get the PC device that matches specific bus and devfn
6e7d01
extracted from PE config address passed as argument. Thus it tries to map
6e7d01
the PE config address to a single specific PCI device 'bus->devices[devfn]'
6e7d01
which always results into checking device on slot 0 'bus->devices[0]'.
6e7d01
This succeeds when there is a pass-through device (vfio-pci) present on
6e7d01
slot 0. But in cases where there is no pass-through device present in slot
6e7d01
0, but present in non-zero slots, ibm,set-eeh-option call fails to enable
6e7d01
the EEH capability.
6e7d01
6e7d01
hw/ppc/spapr_pci_vfio.c: spapr_phb_vfio_eeh_set_option()
6e7d01
   case RTAS_EEH_ENABLE: {
6e7d01
        PCIHostState *phb;
6e7d01
        PCIDevice *pdev;
6e7d01
6e7d01
        /*
6e7d01
         * The EEH functionality is enabled on basis of PCI device,
6e7d01
         * instead of PE. We need check the validity of the PCI
6e7d01
         * device address.
6e7d01
         */
6e7d01
        phb = PCI_HOST_BRIDGE(sphb);
6e7d01
        pdev = pci_find_device(phb->bus,
6e7d01
                               (addr >> 16) & 0xFF, (addr >> 8) & 0xFF);
6e7d01
        if (!pdev || !object_dynamic_cast(OBJECT(pdev), "vfio-pci")) {
6e7d01
            return RTAS_OUT_PARAM_ERROR;
6e7d01
        }
6e7d01
6e7d01
hw/pci/pci.c:pci_find_device()
6e7d01
6e7d01
PCIDevice *pci_find_device(PCIBus *bus, int bus_num, uint8_t devfn)
6e7d01
{
6e7d01
    bus = pci_find_bus_nr(bus, bus_num);
6e7d01
6e7d01
    if (!bus)
6e7d01
        return NULL;
6e7d01
6e7d01
    return bus->devices[devfn];
6e7d01
}
6e7d01
6e7d01
This patch fixes ibm,set-eeh-option to check for presence of any PCI device
6e7d01
(vfio-pci) under specified bus and enable the EEH if found. The current
6e7d01
code already makes sure that all the devices on that bus are from same
6e7d01
iommu group (within same PE) and fail very early if it does not.
6e7d01
6e7d01
After this fix guest is able to find EEH capable devices and enable EEH
6e7d01
recovery on it.
6e7d01
6e7d01
[root@atest-guest ~]# dmesg | grep EEH
6e7d01
[    0.048139] EEH: pSeries platform initialized
6e7d01
[    0.405115] EEH: Capable adapter found: recovery enabled.
6e7d01
[root@atest-guest ~]#
6e7d01
6e7d01
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
6e7d01
Signed-off-by: Mahesh Salgaonkar <mahesh@linux.ibm.com>
6e7d01
Message-Id: <162158429107.145117.5843504911924013125.stgit@jupiter>
6e7d01
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
6e7d01
(cherry picked from commit ac9ef668321ebb6eb871a0c4dd380fa7d7891b4e)
6e7d01
Signed-off-by: Daniel Henrique Barboza <dbarboza@redhat.com>
6e7d01
Signed-off-by: Danilo C. L. de Paula <ddepaula@redhat.com>
6e7d01
---
6e7d01
 hw/ppc/spapr_pci_vfio.c | 40 +++++++++++++++++++++++++++++++++-------
6e7d01
 1 file changed, 33 insertions(+), 7 deletions(-)
6e7d01
6e7d01
diff --git a/hw/ppc/spapr_pci_vfio.c b/hw/ppc/spapr_pci_vfio.c
6e7d01
index ecb34aaade..a411b08d60 100644
6e7d01
--- a/hw/ppc/spapr_pci_vfio.c
6e7d01
+++ b/hw/ppc/spapr_pci_vfio.c
6e7d01
@@ -48,6 +48,16 @@ void spapr_phb_vfio_reset(DeviceState *qdev)
6e7d01
     spapr_phb_vfio_eeh_reenable(SPAPR_PCI_HOST_BRIDGE(qdev));
6e7d01
 }
6e7d01
 
6e7d01
+static void spapr_eeh_pci_find_device(PCIBus *bus, PCIDevice *pdev,
6e7d01
+                                      void *opaque)
6e7d01
+{
6e7d01
+    bool *found = opaque;
6e7d01
+
6e7d01
+    if (object_dynamic_cast(OBJECT(pdev), "vfio-pci")) {
6e7d01
+        *found = true;
6e7d01
+    }
6e7d01
+}
6e7d01
+
6e7d01
 int spapr_phb_vfio_eeh_set_option(SpaprPhbState *sphb,
6e7d01
                                   unsigned int addr, int option)
6e7d01
 {
6e7d01
@@ -60,17 +70,33 @@ int spapr_phb_vfio_eeh_set_option(SpaprPhbState *sphb,
6e7d01
         break;
6e7d01
     case RTAS_EEH_ENABLE: {
6e7d01
         PCIHostState *phb;
6e7d01
-        PCIDevice *pdev;
6e7d01
+        bool found = false;
6e7d01
 
6e7d01
         /*
6e7d01
-         * The EEH functionality is enabled on basis of PCI device,
6e7d01
-         * instead of PE. We need check the validity of the PCI
6e7d01
-         * device address.
6e7d01
+         * The EEH functionality is enabled per sphb level instead of
6e7d01
+         * per PCI device. We have already identified this specific sphb
6e7d01
+         * based on buid passed as argument to ibm,set-eeh-option rtas
6e7d01
+         * call. Now we just need to check the validity of the PCI
6e7d01
+         * pass-through devices (vfio-pci) under this sphb bus.
6e7d01
+         * We have already validated that all the devices under this sphb
6e7d01
+         * are from same iommu group (within same PE) before comming here.
6e7d01
+         *
6e7d01
+         * Prior to linux commit 98ba956f6a389 ("powerpc/pseries/eeh:
6e7d01
+         * Rework device EEH PE determination") kernel would call
6e7d01
+         * eeh-set-option for each device in the PE using the device's
6e7d01
+         * config_address as the argument rather than the PE address.
6e7d01
+         * Hence if we check validity of supplied config_addr whether
6e7d01
+         * it matches to this PHB will cause issues with older kernel
6e7d01
+         * versions v5.9 and older. If we return an error from
6e7d01
+         * eeh-set-option when the argument isn't a valid PE address
6e7d01
+         * then older kernels (v5.9 and older) will interpret that as
6e7d01
+         * EEH not being supported.
6e7d01
          */
6e7d01
         phb = PCI_HOST_BRIDGE(sphb);
6e7d01
-        pdev = pci_find_device(phb->bus,
6e7d01
-                               (addr >> 16) & 0xFF, (addr >> 8) & 0xFF);
6e7d01
-        if (!pdev || !object_dynamic_cast(OBJECT(pdev), "vfio-pci")) {
6e7d01
+        pci_for_each_device(phb->bus, (addr >> 16) & 0xFF,
6e7d01
+                            spapr_eeh_pci_find_device, &found);
6e7d01
+
6e7d01
+        if (!found) {
6e7d01
             return RTAS_OUT_PARAM_ERROR;
6e7d01
         }
6e7d01
 
6e7d01
-- 
6e7d01
2.27.0
6e7d01