From f3b6556eedcc1f8278e45ed809b06f243fe99ced Mon Sep 17 00:00:00 2001

Message-Id: <f3b6556eedcc1f8278e45ed809b06f243fe99ced.1386348946.git.jdenemar@redhat.com>

From: "Daniel P. Berrange" <berrange@redhat.com>

Date: Mon, 2 Dec 2013 13:36:29 +0000

Subject: [PATCH] Improve cgroups docs to cover systemd integration

For

  https://bugzilla.redhat.com/show_bug.cgi?id=1004340

As of libvirt 1.1.1 and systemd 205, the cgroups layout used by

libvirt has some changes. Update the 'cgroups.html' file from

the website to describe how it works in a systemd world.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

(cherry picked from commit 7f2b173febaefda73b486337b6c53f5c2127070f)

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>

---

 docs/cgroups.html.in | 212 +++++++++++++++++++++++++++++++++++++++++----------

 1 file changed, 172 insertions(+), 40 deletions(-)

diff --git a/docs/cgroups.html.in b/docs/cgroups.html.in

index 77656b2..f7c2450 100644

--- a/docs/cgroups.html.in

+++ b/docs/cgroups.html.in

@@ -47,17 +47,121 @@

       As of libvirt 1.0.5 or later, the cgroups layout created by libvirt has been

       simplified, in order to facilitate the setup of resource control policies by

-      administrators / management applications. The layout is based on the concepts of

-      "partitions" and "consumers". Each virtual machine or container is a consumer,

-      and has a corresponding cgroup named $VMNAME.libvirt-{qemu,lxc}.

-      Each consumer is associated with exactly one partition, which also have a

-      corresponding cgroup usually named $PARTNAME.partition. The

-      exceptions to this naming rule are the three top level default partitions,

-      named /system (for system services), /user (for

-      user login sessions) and /machine (for virtual machines and

-      containers). By default every consumer will of course be associated with

-      the /machine partition. This leads to a hierarchy that looks

-      like

+      administrators / management applications. The new layout is based on the concepts

+      of "partitions" and "consumers". A "consumer" is a cgroup which holds the

+      processes for a single virtual machine or container. A "partition" is a cgroup

+      which does not contain any processes, but can have resource controls applied.

+      A "partition" will have zero or more child directories which may be either

+      "consumer" or "partition".

+    

+

+    

+      As of libvirt 1.1.1 or later, the cgroups layout will have some slight

+      differences when running on a host with systemd 205 or later. The overall

+      tree structure is the same, but there are some differences in the naming

+      conventions for the cgroup directories. Thus the following docs split

+      in two, one describing systemd hosts and the other non-systemd hosts.

+    

+

+    
Systemd cgroups integration

+

+    

+      On hosts which use systemd, each consumer maps to a systemd scope unit,

+      while partitions map to a system slice unit.

+    

+

+    
Systemd scope naming

+

+    

+      The systemd convention is for the scope name of virtual machines / containers

+      to be of the general format machine-$NAME.scope. Libvirt forms the

+      $NAME part of this by concatenating the driver type with the name

+      of the guest, and then escaping any systemd reserved characters.

+      So for a guest demo running under the lxc driver,

+      we get a $NAME of lxc-demo which when escaped is

+      lxc\x2ddemo. So the complete scope name is machine-lxc\x2ddemo.scope.

+      The scope names map directly to the cgroup directory names.

+    

+

+    
Systemd slice naming

+

+    

+      The systemd convention for slice naming is that a slice should include the

+      name of all of its parents prepended on its own name. So for a libvirt

+      partition /machine/engineering/testing, the slice name will

+      be machine-engineering-testing.slice. Again the slice names

+      map directly to the cgroup directory names. Systemd creates three top level

+      slices by default, system.slice user.slice and

+      machine.slice. All virtual machines or containers created

+      by libvirt will be associated with machine.slice by default.

+    

+

+    
Systemd cgroup layout

+

+    

+      Given this, a possible systemd cgroups layout involving 3 qemu guests,

+      3 lxc containers and 3 custom child slices, would be:

+    

+

+    

+$ROOT

+  |

+  +- system.slice

+  |   |

+  |   +- libvirtd.service

+  |

+  +- machine.slice

+      |

+      +- machine-qemu\x2dvm1.scope

+      |   |

+      |   +- emulator

+      |   +- vcpu0

+      |   +- vcpu1

+      |

+      +- machine-qemu\x2dvm2.scope

+      |   |

+      |   +- emulator

+      |   +- vcpu0

+      |   +- vcpu1

+      |

+      +- machine-qemu\x2dvm3.scope

+      |   |

+      |   +- emulator

+      |   +- vcpu0

+      |   +- vcpu1

+      |

+      +- machine-engineering.slice

+      |   |

+      |   +- machine-engineering-testing.slice

+      |   |   |

+      |   |   +- machine-lxc\x2dcontainer1.scope

+      |   |

+      |   +- machine-engineering-production.slice

+      |       |

+      |       +- machine-lxc\x2dcontainer2.scope

+      |

+      +- machine-marketing.slice

+          |

+          +- machine-lxc\x2dcontainer3.scope

+    

+

+    
Non-systemd cgroups layout

+

+    

+      On hosts which do not use systemd, each consumer has a corresponding cgroup

+      named $VMNAME.libvirt-{qemu,lxc}. Each consumer is associated

+      with exactly one partition, which also have a corresponding cgroup usually

+      named $PARTNAME.partition. The exceptions to this naming rule

+      are the three top level default partitions, named /system (for

+      system services), /user (for user login sessions) and

+      /machine (for virtual machines and containers). By default

+      every consumer will of course be associated with the /machine

+      partition.

+    

+

+    

+      Given this, a possible systemd cgroups layout involving 3 qemu guests,

+      3 lxc containers and 2 custom child slices, would be:

@@ -87,23 +191,21 @@ $ROOT

       |   +- vcpu0

       |   +- vcpu1

       |

-      +- container1.libvirt-lxc

-      |

-      +- container2.libvirt-lxc

+      +- engineering.partition

+      |   |

+      |   +- testing.partition

+      |   |   |

+      |   |   +- container1.libvirt-lxc

+      |   |

+      |   +- production.partition

+      |       |

+      |       +- container2.libvirt-lxc

       |

-      +- container3.libvirt-lxc

+      +- marketing.partition

+          |

+          +- container3.libvirt-lxc

-    

-      The default cgroups layout ensures that, when there is contention for

-      CPU time, it is shared equally between system services, user sessions

-      and virtual machines / containers. This prevents virtual machines from

-      locking the administrator out of the host, or impacting execution of

-      system services. Conversely, when there is no contention from

-      system services / user sessions, it is possible for virtual machines

-      to fully utilize the host CPUs.

-    

-

     
Using custom partitions

@@ -127,12 +229,54 @@ $ROOT

+      Note that the partition names in the guest XML are using a

+      generic naming format, not the low level naming convention

+      required by the underlying host OS. That is, you should not include

+      any of the .partition or .slice

+      suffixes in the XML config. Given a partition name

+      /machine/production, libvirt will automatically

+      apply the platform specific translation required to get

+      /machine/production.partition (non-systemd)

+      or /machine.slice/machine-production.slice

+      (systemd) as the underlying cgroup name

+    

+

+    

       Libvirt will not auto-create the cgroups directory to back

       this partition. In the future, libvirt / virsh will provide

       APIs / commands to create custom partitions, but currently

-      this is left as an exercise for the administrator. For

-      example, given the XML config above, the admin would need

-      to create a cgroup named '/machine/production.partition'

+      this is left as an exercise for the administrator.

+    

+

+    

+      Note: the ability to place guests in custom

+      partitions is only available with libvirt >= 1.0.5, using

+      the new cgroup layout. The legacy cgroups layout described

+      later in this document did not support customization per guest.

+    

+

+    
Creating custom partitions (systemd)

+

+    

+      Given the XML config above, the admin on a systemd based host would

+      need to create a unit file /etc/systemd/system/machine-production.slice

+    

+

+    

+# cat > /etc/systemd/system/machine-testing.slice <<EOF

+[Unit]

+Description=VM testing slice

+Before=slices.target

+Wants=machine.slice

+EOF

+# systemctl start machine-testing.slice

+    

+

+    
Creating custom partitions (non-systemd)

+

+    

+      Given the XML config above, the admin on a non-systemd based host

+      would need to create a cgroup named '/machine/production.partition'

@@ -147,18 +291,6 @@ $ROOT

   done

-    

-      Note: the cgroups directory created as a ".partition"

-      suffix, but the XML config does not require this suffix.

-    

-

-    

-      Note: the ability to place guests in custom

-      partitions is only available with libvirt >= 1.0.5, using

-      the new cgroup layout. The legacy cgroups layout described

-      later did not support customization per guest.

-    

-

     
Resource management APIs/commands

-- 

1.8.4.5