Blame SOURCES/kvm-qcow2-Give-the-refcount-cache-the-minimum-possible-s.patch

26ba25
From f9faa15ed2a819c8fcf1eaf3534d7162f9cb8290 Mon Sep 17 00:00:00 2001
26ba25
From: Kevin Wolf <kwolf@redhat.com>
26ba25
Date: Thu, 6 Dec 2018 17:12:26 +0000
26ba25
Subject: [PATCH 01/15] qcow2: Give the refcount cache the minimum possible
26ba25
 size by default
26ba25
26ba25
RH-Author: Kevin Wolf <kwolf@redhat.com>
26ba25
Message-id: <20181206171240.5674-2-kwolf@redhat.com>
26ba25
Patchwork-id: 83284
26ba25
O-Subject: [RHEL-8.0 qemu-kvm PATCH 01/15] qcow2: Give the refcount cache the minimum possible size by default
26ba25
Bugzilla: 1656507
26ba25
RH-Acked-by: Max Reitz <mreitz@redhat.com>
26ba25
RH-Acked-by: John Snow <jsnow@redhat.com>
26ba25
RH-Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
26ba25
26ba25
From: Alberto Garcia <berto@igalia.com>
26ba25
26ba25
The L2 and refcount caches have default sizes that can be overridden
26ba25
using the l2-cache-size and refcount-cache-size (an additional
26ba25
parameter named cache-size sets the combined size of both caches).
26ba25
26ba25
Unless forced by one of the aforementioned parameters, QEMU will set
26ba25
the unspecified sizes so that the L2 cache is 4 times larger than the
26ba25
refcount cache.
26ba25
26ba25
This is based on the premise that the refcount metadata needs to be
26ba25
only a fourth of the L2 metadata to cover the same amount of disk
26ba25
space. This is incorrect for two reasons:
26ba25
26ba25
 a) The amount of disk covered by an L2 table depends solely on the
26ba25
    cluster size, but in the case of a refcount block it depends on
26ba25
    the cluster size *and* the width of each refcount entry.
26ba25
    The 4/1 ratio is only valid with 16-bit entries (the default).
26ba25
26ba25
 b) When we talk about disk space and L2 tables we are talking about
26ba25
    guest space (L2 tables map guest clusters to host clusters),
26ba25
    whereas refcount blocks are used for host clusters (including
26ba25
    L1/L2 tables and the refcount blocks themselves). On a fully
26ba25
    populated (and uncompressed) qcow2 file, image size > virtual size
26ba25
    so there are more refcount entries than L2 entries.
26ba25
26ba25
Problem (a) could be fixed by adjusting the algorithm to take into
26ba25
account the refcount entry width. Problem (b) could be fixed by
26ba25
increasing a bit the refcount cache size to account for the clusters
26ba25
used for qcow2 metadata.
26ba25
26ba25
However this patch takes a completely different approach and instead
26ba25
of keeping a ratio between both cache sizes it assigns as much as
26ba25
possible to the L2 cache and the remainder to the refcount cache.
26ba25
26ba25
The reason is that L2 tables are used for every single I/O request
26ba25
from the guest and the effect of increasing the cache is significant
26ba25
and clearly measurable. Refcount blocks are however only used for
26ba25
cluster allocation and internal snapshots and in practice are accessed
26ba25
sequentially in most cases, so the effect of increasing the cache is
26ba25
negligible (even when doing random writes from the guest).
26ba25
26ba25
So, make the refcount cache as small as possible unless the user
26ba25
explicitly asks for a larger one.
26ba25
26ba25
Signed-off-by: Alberto Garcia <berto@igalia.com>
26ba25
Reviewed-by: Eric Blake <eblake@redhat.com>
26ba25
Reviewed-by: Max Reitz <mreitz@redhat.com>
26ba25
Message-id: 9695182c2eb11b77cb319689a1ebaa4e7c9d6591.1523968389.git.berto@igalia.com
26ba25
Signed-off-by: Max Reitz <mreitz@redhat.com>
26ba25
(cherry picked from commit 52253998ec3e523c9e20ae81e2a6431d8ff733ba)
26ba25
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
26ba25
Signed-off-by: Danilo C. L. de Paula <ddepaula@redhat.com>
26ba25
---
26ba25
 block/qcow2.c              | 31 +++++++++++++++++++------------
26ba25
 block/qcow2.h              |  4 ----
26ba25
 tests/qemu-iotests/137.out |  2 +-
26ba25
 3 files changed, 20 insertions(+), 17 deletions(-)
26ba25
26ba25
diff --git a/block/qcow2.c b/block/qcow2.c
26ba25
index 36d1152..4b65e4c 100644
26ba25
--- a/block/qcow2.c
26ba25
+++ b/block/qcow2.c
26ba25
@@ -809,23 +809,30 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts,
26ba25
         } else if (refcount_cache_size_set) {
26ba25
             *l2_cache_size = combined_cache_size - *refcount_cache_size;
26ba25
         } else {
26ba25
-            *refcount_cache_size = combined_cache_size
26ba25
-                                 / (DEFAULT_L2_REFCOUNT_SIZE_RATIO + 1);
26ba25
-            *l2_cache_size = combined_cache_size - *refcount_cache_size;
26ba25
+            uint64_t virtual_disk_size = bs->total_sectors * BDRV_SECTOR_SIZE;
26ba25
+            uint64_t max_l2_cache = virtual_disk_size / (s->cluster_size / 8);
26ba25
+            uint64_t min_refcount_cache =
26ba25
+                (uint64_t) MIN_REFCOUNT_CACHE_SIZE * s->cluster_size;
26ba25
+
26ba25
+            /* Assign as much memory as possible to the L2 cache, and
26ba25
+             * use the remainder for the refcount cache */
26ba25
+            if (combined_cache_size >= max_l2_cache + min_refcount_cache) {
26ba25
+                *l2_cache_size = max_l2_cache;
26ba25
+                *refcount_cache_size = combined_cache_size - *l2_cache_size;
26ba25
+            } else {
26ba25
+                *refcount_cache_size =
26ba25
+                    MIN(combined_cache_size, min_refcount_cache);
26ba25
+                *l2_cache_size = combined_cache_size - *refcount_cache_size;
26ba25
+            }
26ba25
         }
26ba25
     } else {
26ba25
-        if (!l2_cache_size_set && !refcount_cache_size_set) {
26ba25
+        if (!l2_cache_size_set) {
26ba25
             *l2_cache_size = MAX(DEFAULT_L2_CACHE_BYTE_SIZE,
26ba25
                                  (uint64_t)DEFAULT_L2_CACHE_CLUSTERS
26ba25
                                  * s->cluster_size);
26ba25
-            *refcount_cache_size = *l2_cache_size
26ba25
-                                 / DEFAULT_L2_REFCOUNT_SIZE_RATIO;
26ba25
-        } else if (!l2_cache_size_set) {
26ba25
-            *l2_cache_size = *refcount_cache_size
26ba25
-                           * DEFAULT_L2_REFCOUNT_SIZE_RATIO;
26ba25
-        } else if (!refcount_cache_size_set) {
26ba25
-            *refcount_cache_size = *l2_cache_size
26ba25
-                                 / DEFAULT_L2_REFCOUNT_SIZE_RATIO;
26ba25
+        }
26ba25
+        if (!refcount_cache_size_set) {
26ba25
+            *refcount_cache_size = MIN_REFCOUNT_CACHE_SIZE * s->cluster_size;
26ba25
         }
26ba25
     }
26ba25
 
26ba25
diff --git a/block/qcow2.h b/block/qcow2.h
26ba25
index 43163b2..3d92cdb 100644
26ba25
--- a/block/qcow2.h
26ba25
+++ b/block/qcow2.h
26ba25
@@ -77,10 +77,6 @@
26ba25
 #define DEFAULT_L2_CACHE_CLUSTERS 8 /* clusters */
26ba25
 #define DEFAULT_L2_CACHE_BYTE_SIZE 1048576 /* bytes */
26ba25
 
26ba25
-/* The refblock cache needs only a fourth of the L2 cache size to cover as many
26ba25
- * clusters */
26ba25
-#define DEFAULT_L2_REFCOUNT_SIZE_RATIO 4
26ba25
-
26ba25
 #define DEFAULT_CLUSTER_SIZE 65536
26ba25
 
26ba25
 
26ba25
diff --git a/tests/qemu-iotests/137.out b/tests/qemu-iotests/137.out
26ba25
index e28e1ea..96724a6 100644
26ba25
--- a/tests/qemu-iotests/137.out
26ba25
+++ b/tests/qemu-iotests/137.out
26ba25
@@ -22,7 +22,7 @@ refcount-cache-size may not exceed cache-size
26ba25
 L2 cache size too big
26ba25
 L2 cache entry size must be a power of two between 512 and the cluster size (65536)
26ba25
 L2 cache entry size must be a power of two between 512 and the cluster size (65536)
26ba25
-L2 cache size too big
26ba25
+Refcount cache size too big
26ba25
 Conflicting values for qcow2 options 'overlap-check' ('constant') and 'overlap-check.template' ('all')
26ba25
 Unsupported value 'blubb' for qcow2 option 'overlap-check'. Allowed are any of the following: none, constant, cached, all
26ba25
 Unsupported value 'blubb' for qcow2 option 'overlap-check'. Allowed are any of the following: none, constant, cached, all
26ba25
-- 
26ba25
1.8.3.1
26ba25