26ba25
From 1b487f2a77c49129267e4f779956dd549684d8d9 Mon Sep 17 00:00:00 2001
26ba25
From: Stefan Hajnoczi <stefanha@redhat.com>
26ba25
Date: Mon, 23 Jul 2018 16:36:29 +0200
26ba25
Subject: [PATCH 256/268] throttle-groups: fix hang when group member leaves
26ba25
26ba25
RH-Author: Stefan Hajnoczi <stefanha@redhat.com>
26ba25
Message-id: <20180723163629.7600-2-stefanha@redhat.com>
26ba25
Patchwork-id: 81469
26ba25
O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/1] throttle-groups: fix hang when group member leaves
26ba25
Bugzilla: 1535914
26ba25
RH-Acked-by: Fam Zheng <famz@redhat.com>
26ba25
RH-Acked-by: Laurent Vivier <lvivier@redhat.com>
26ba25
RH-Acked-by: Miroslav Rezanina <mrezanin@redhat.com>
26ba25
26ba25
Throttle groups consist of members sharing one throttling state
26ba25
(including bps/iops limits).  Round-robin scheduling is used to ensure
26ba25
fairness.  If a group member already has a timer pending then other
26ba25
groups members do not schedule their own timers.  The next group member
26ba25
will have its turn when the existing timer expires.
26ba25
26ba25
A hang may occur when a group member leaves while it had a timer
26ba25
scheduled.  Although the code carefully removes the group member from
26ba25
the round-robin list, it does not schedule the next member.  Therefore
26ba25
remaining members continue to wait for the removed member's timer to
26ba25
expire.
26ba25
26ba25
This patch schedules the next request if a timer is pending.
26ba25
Unfortunately the actual bug is a race condition that I've been unable
26ba25
to capture in a test case.
26ba25
26ba25
Sometimes drive2 hangs when drive1 is removed from the throttling group:
26ba25
26ba25
  $ qemu ... -drive if=none,id=drive1,cache=none,format=qcow2,file=data1.qcow2,iops=100,group=foo \
26ba25
             -device virtio-blk-pci,id=virtio-blk-pci0,drive=drive1 \
26ba25
             -drive if=none,id=drive2,cache=none,format=qcow2,file=data2.qcow2,iops=10,group=foo \
26ba25
             -device virtio-blk-pci,id=virtio-blk-pci1,drive=drive2
26ba25
  (guest-console1)# fio -filename /dev/vda 4k-seq-read.job
26ba25
  (guest-console2)# fio -filename /dev/vdb 4k-seq-read.job
26ba25
  (qmp) {"execute": "block_set_io_throttle", "arguments": {"device": "drive1","bps": 0,"bps_rd": 0,"bps_wr": 0,"iops": 0,"iops_rd": 0,"iops_wr": 0}}
26ba25
26ba25
Reported-by: Nini Gu <ngu@redhat.com>
26ba25
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
26ba25
Message-id: 20180704145410.794-1-stefanha@redhat.com
26ba25
Cc: Alberto Garcia <berto@igalia.com>
26ba25
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
26ba25
Signed-off-by: Miroslav Rezanina <mrezanin@redhat.com>
26ba25
---
26ba25
 block/throttle-groups.c | 4 ++++
26ba25
 1 file changed, 4 insertions(+)
26ba25
26ba25
diff --git a/block/throttle-groups.c b/block/throttle-groups.c
26ba25
index 36cc043..e297b04 100644
26ba25
--- a/block/throttle-groups.c
26ba25
+++ b/block/throttle-groups.c
26ba25
@@ -564,6 +564,10 @@ void throttle_group_unregister_tgm(ThrottleGroupMember *tgm)
26ba25
 
26ba25
     qemu_mutex_lock(&tg->lock);
26ba25
     for (i = 0; i < 2; i++) {
26ba25
+        if (timer_pending(tgm->throttle_timers.timers[i])) {
26ba25
+            tg->any_timer_armed[i] = false;
26ba25
+            schedule_next_request(tgm, i);
26ba25
+        }
26ba25
         if (tg->tokens[i] == tgm) {
26ba25
             token = throttle_group_next_tgm(tgm);
26ba25
             /* Take care of the case where this is the last tgm in the group */
26ba25
-- 
26ba25
1.8.3.1
26ba25