From 789668deb3e6f8584ffab964d2204ddcb75f0a06 Mon Sep 17 00:00:00 2001

From: Frederick Grose <fgrose@sugarlabs.org>

Date: Mon, 20 Aug 2018 09:47:50 -0400

Subject: [PATCH] dmsquash-live: Support a flattened squashfs.img

A simplified root filesystem structure may be provided for OverlayFS

overlays by squashing the root filesystem directly instead of squashing

an embedded image file at /LiveOS/rootfs.img.  Detect and configure

such a squashed root filesystem for live booting.

For OverlayFS boots, avoid the read-only Device-mapper linear device

  at /dev/mapper/live-base.

Create a consistent device link at /dev/live-base for the read-only

  base loop device for all overlayed live root filesystems.

Consistently provide a link at /dev/root for wait_for_dev.

Update documentation.

---

 dracut.cmdline.7.asc                            | 126 +++++++++++++++---------

 modules.d/90dmsquash-live/dmsquash-live-root.sh |  51 ++++++----

 2 files changed, 110 insertions(+), 67 deletions(-)

diff --git a/dracut.cmdline.7.asc b/dracut.cmdline.7.asc

index df633e5d..882035af 100644

--- a/dracut.cmdline.7.asc

+++ b/dracut.cmdline.7.asc

@@ -838,23 +838,24 @@ Booting live images

 Dracut offers multiple options for live booted images:

 =====================

-SquashFS with read-only filesystem image::: The system will boot with a read

-only filesystem from the SquashFS and apply a writable device-mapper snapshot

-over the read only filesystem.  Using this method ensures a relatively fast

-boot and lower RAM usage. Users **must be careful** to avoid writing too many

-blocks to the snapshot volume.  Once the blocks of the snapshot overlay are

-exhausted, the root filesystem becomes read only and may cause application

-failures.  The overlay file is marked 'Overflow', and a difficult recovery is

-required to repair and enlarge the overlay offline.  Non-persistent overlays

-are sparse files in RAM that only consume content space as required blocks are

-allocated.  They default to an apparent size of 32 GiB in RAM.  The size can be

-adjusted with the **rd.live.overlay.size=** kernel command line option.

+SquashFS with read-only filesystem image::: The system will boot with a

+read-only filesystem from the SquashFS and apply a writable Device-mapper

+snapshot or an OverlayFS overlay mount for the read-only base filesystem.  This

+method ensures a relatively fast boot and lower RAM usage. Users **must be

+careful** to avoid writing too many blocks to a snapshot volume.  Once the

+blocks of the snapshot overlay are exhausted, the root filesystem becomes

+read-only and may cause application failures.  The snapshot overlay file is

+marked 'Overflow', and a difficult recovery is required to repair and enlarge

+the overlay offline.  Non-persistent overlays are sparse files in RAM that only

+consume content space as required blocks are allocated.  They default to an

+apparent size of 32 GiB in RAM.  The size can be adjusted with the

+**rd.live.overlay.size=** kernel command line option.

 +

-The filesystem structure is expected to be:

+The filesystem structure is traditionally expected to be:

 +

 [listing]

 --

-squashfs.img          |  Squashfs from LiveCD .iso downloaded via network

+squashfs.img          |  SquashFS from LiveCD .iso

    !(mount)

    /LiveOS

        |- rootfs.img  |  Filesystem image to mount read-only

@@ -865,21 +866,35 @@ squashfs.img          |  Squashfs from LiveCD .iso downloaded via network

             ...       |

 --

 +

-Dracut uses this method of live booting by default.  No additional command line

-options are required other than **root=live:<URL>** to specify the location

-of your squashed filesystem.

+For OverlayFS mount overlays, the filesystem structure may also be a direct

+compression of the root filesystem:

++

+[listing]

+--

+squashfs.img          |  SquashFS from LiveCD .iso

+   !(mount)

+   /bin               |  Live filesystem

+   /boot              |

+   /dev               |

+   ...                |

+--

++

+Dracut uses one of the overlay methods of live booting by default.  No

+additional command line options are required other than **root=live:<URL>** to

+specify the location of your squashed filesystem.

 +

 - The compressed SquashFS image can be copied during boot to RAM at

 `/run/initramfs/squashed.img` by using the **rd.live.ram=1** option.

-- A device with a persistent overlay can be booted read only by using the

+- A device with a persistent overlay can be booted read-only by using the

 **rd.live.overlay.readonly** option on the kernel command line.  This will

-cause a temporary, writable overlay to be stacked over a read-only snapshot

-of the root filesystem.

+either cause a temporary, writable overlay to be stacked over a read-only

+snapshot of the root filesystem or the OverlayFS mount will use an additional

+lower layer with the root filesystem.

 +

 Uncompressed live filesystem image:::

 When the live system was installed with the '--skipcompress' option of the

 __livecd-iso-to-disk__ installation script for Live USB devices, the root

-filesystem image, `rootfs.img`, is expanded on installation and no SquashFS

+filesystem image, __rootfs.img__, is expanded on installation and no SquashFS

 is involved during boot.

 +

 - If **rd.live.ram=1** is used in this situation, the full, uncompressed

@@ -887,12 +902,12 @@ root filesystem is copied during boot to `/run/initramfs/rootfs.img` in the

 `/run` tmpfs.

 +

 - If **rd.live.overlay=none** is provided as a kernel command line option,

-a writable, linear device-mapper target is created on boot with no overlay.

+a writable, linear Device-mapper target is created on boot with no overlay.

-writable filesystem image:::

+Writable filesystem image:::

 The system will retrieve a compressed filesystem image, extract it to

 `/run/initramfs/fsimg/rootfs.img`, connect it to a loop device, create a

-writable, linear device-mapper target at `/dev/mapper/live-rw`, and mount that

+writable, linear Device-mapper target at `/dev/mapper/live-rw`, and mount that

 as a writable volume at `/`.  More RAM is required during boot but the live

 filesystem is easier to manage if it becomes full.  Users can make a filesystem

 image of any size and that size will be maintained when the system boots. There

@@ -902,7 +917,7 @@ The filesystem structure is expected to be:

 +

 [listing]

 --

-rootfs.tgz            |  Compressed tarball containing fileystem image

+rootfs.tgz            |  Compressed tarball containing filesystem image

    !(unpack)

    /rootfs.img        |  Filesystem image at /run/initramfs/fsimg/

       !(mount)

@@ -930,11 +945,11 @@ NOTE: There must be enough free RAM available to hold the complete image.

 This method is very suitable for diskless boots.

 **root=**live:__<url>__::

-Boots a live image retrieved from __<url>__.  Requires dracut 'livenet' module.

-Valid handlers: __http, https, ftp, torrent, tftp__.

+Boots a live image retrieved from __<url>__.  Requires the dracut 'livenet'

+module.  Valid handlers: __http, https, ftp, torrent, tftp__.

 +

 [listing]

-.Example

+.Examples

 --

 root=live:http://example.com/liveboot.img

 root=live:ftp://ftp.example.com/liveboot.img

@@ -946,7 +961,7 @@ Enables debug output from the live boot process.

 **rd.live.dir=**__<path>__::

 Specifies the directory within the boot device where the squashfs.img or

-rootfs.img can be found.  By default, this is __LiveOS__.

+rootfs.img can be found.  By default, this is `/LiveOS`.

 **rd.live.squashimg=**__<filename of SquashFS image>__::

 Specifies the filename for a SquashFS image of the root filesystem.

@@ -954,35 +969,52 @@ By default, this is __squashfs.img__.

 **rd.live.ram=**1::

 Copy the complete image to RAM and use this for booting. This is useful

-when the image resides on, i.e., a DVD which needs to be ejected later on.

+when the image resides on, e.g., a DVD which needs to be ejected later on.

 **rd.live.overlay={**__<devspec>__[:__{<pathspec>|auto}__]|__none__}::

-Allow the usage of a permanent overlay.

-- _<devspec>_ specifies the path to a device with a mountable filesystem.

-- _<pathspec>_ is the path to a file within that filesystem, which shall be

+Manage the usage of a permanent overlay.

++

+--

+* _<devspec>_ specifies the path to a device with a mountable filesystem.

+* _<pathspec>_ is the path to a file within that filesystem, which shall be

 used to persist the changes made to the device specified by the

 **root=live:__<url>__** option.

-- _none_ specifies no overlay when an uncompressed live root filesystem is

-available.

-If a persistent overlay is detected at the standard LiveOS path, the overlay &

-overlay type detected (whether Device-mapper or OverlayFS) will be used.

++

+The default _pathspec_, when _auto_ or no _:<pathspec>_ is given, is

+`/<+++rd.live.dir+++>/overlay-<label>-<uuid>`, where _<label>_ is the

+device LABEL, and _<uuid>_ is the device UUID.

+* _none_ (the word itself) specifies that no overlay will be used, such as when

+an uncompressed, writable live root filesystem is available.

++

+If a persistent overlay __is detected__ at the standard LiveOS path, the

+overlay & overlay type detected, whether Device-mapper or OverlayFS, will be

+used.

+--

 +

 [listing]

-.Example

+.Examples

 --

 rd.live.overlay=/dev/sdb1:persistent-overlay.img

+rd.live.overlay=UUID=99440c1f-8daa-41bf-b965-b7240a8996f4

 --

 **rd.live.overlay.size=**__<size_MiB>__::

-Specifies a non-persistent overlay size in MiB.  The default is _32768_.

+Specifies a non-persistent Device-mapper overlay size in MiB.  The default is

+_32768_.

 **rd.live.overlay.readonly=**1::

-Specifies a non-persistent, writable snapshot overlay to be stacked over a

-read-only snapshot of the root filesystem, `/dev/mapper/live-ro`, or a read-

-only loop device of a writable `rootfs.img`.

+This is used to boot with a normally read-write persistent overlay in a

+read-only mode.  With this option, either an additional, non-persistent,

+writable snapshot overlay will be stacked over a read-only snapshot,

+`/dev/mapper/live‑ro`, of the base filesystem with the persistent overlay, or a

+read-only loop device, in the case of a writable __rootfs.img__, or an OverlayFS

+mount will use the persistent overlay directory linked at `/run/overlayfs‑r` as

+an additional lower layer along with the base root filesystem and apply a

+transient, writable upper directory overlay, in order to complete the booted

+root filesystem.

 **rd.live.overlay.reset=**1::

-Specifies that a persistent overlay should be reset on boot.  All root

+Specifies that a persistent overlay should be reset on boot.  All previous root

 filesystem changes are vacated by this action.

 **rd.live.overlay.thin=**1::

@@ -993,25 +1025,25 @@ that memory is given back to the kernel when the filesystem does not claim it

 anymore.

 **rd.live.overlay.overlayfs=**1::

-Enables the use of the **OverlayFS** kernel module, if available, to provide a

+Enables the use of the *OverlayFS* kernel module, if available, to provide a

 copy-on-write union directory for the root filesystem.  OverlayFS overlays are

 directories of the files that have changed on the read-only base (lower)

 filesystem.  The root filesystem is provided through a special overlay type

 mount that merges the lower and upper directories.  If an OverlayFS upper

 directory is not present on the boot device, a tmpfs directory will be created

-at /run/overlayfs to provide temporary storage.  Persistent storage can be

+at `/run/overlayfs` to provide temporary storage.  Persistent storage can be

 provided on vfat or msdos formatted devices by supplying the OverlayFS upper

 directory within an embedded filesystem that supports the creation of trusted.*

 extended attributes and provides a valid d_type in readdir responses, such as

 with ext4 and xfs.  On non-vfat-formatted devices, a persistent OverlayFS

 overlay can extend the available root filesystem storage up to the capacity of

-the LiveOS device.

+the LiveOS disk device.

 +

 If a persistent overlay is detected at the standard LiveOS path, the overlay &

-overlay type detected (whether OverlayFS or Device-mapper) will be used.

+overlay type detected, whether OverlayFS or Device-mapper, will be used.

 +

 The **rd.live.overlay.readonly** option, which allows a persistent overlayfs to

-be mounted read only through a higher level transient overlay directory, has

+be mounted read-only through a higher level transient overlay directory, has

 been implemented through the multiple lower layers feature of OverlayFS.

diff --git a/modules.d/90dmsquash-live/dmsquash-live-root.sh b/modules.d/90dmsquash-live/dmsquash-live-root.sh

index ac603408..0c5e010e 100755

--- a/modules.d/90dmsquash-live/dmsquash-live-root.sh

+++ b/modules.d/90dmsquash-live/dmsquash-live-root.sh

@@ -250,13 +250,11 @@ do_live_overlay() {

         echo 0 $sz snapshot $base $over PO 8 | dmsetup create live-rw

     fi

-    # Create a device that always points to a ro base image

-    if [ -n "$overlayfs" ]; then

-        BASE_LOOPDUP=$(losetup -f --show -r $BASE_LOOPDEV)

-        echo 0 $sz linear $BASE_LOOPDUP 0 | dmsetup create --readonly live-base

-    else

+    # Create a device for the ro base of overlayed file systems.

+    if [ -z "$overlayfs" ]; then

         echo 0 $sz linear $BASE_LOOPDEV 0 | dmsetup create --readonly live-base

     fi

+    ln -s $BASE_LOOPDEV /dev/live-base

 }

 # we might have a genMinInstDelta delta file for anaconda to take advantage of

@@ -291,10 +289,21 @@ if [ -e "$SQUASHED" ]; then

     mkdir -m 0755 -p /run/initramfs/squashfs

     mount -n -t squashfs -o ro $SQUASHED_LOOPDEV /run/initramfs/squashfs

-    if [ -f /run/initramfs/squashfs/LiveOS/rootfs.img ]; then

-        FSIMG="/run/initramfs/squashfs/LiveOS/rootfs.img"

-    elif [ -f /run/initramfs/squashfs/LiveOS/ext3fs.img ]; then

-        FSIMG="/run/initramfs/squashfs/LiveOS/ext3fs.img"

+    if [ -d /run/initramfs/squashfs/LiveOS ]; then

+        if [ -f /run/initramfs/squashfs/LiveOS/rootfs.img ]; then

+            FSIMG="/run/initramfs/squashfs/LiveOS/rootfs.img"

+        elif [ -f /run/initramfs/squashfs/LiveOS/ext3fs.img ]; then

+            FSIMG="/run/initramfs/squashfs/LiveOS/ext3fs.img"

+        fi

+    elif [ -d /run/initramfs/squashfs/proc ]; then

+        FSIMG=$SQUASHED

+        if [ -z "$overlayfs" ]; then

+            overlayfs="yes"

+            [ -n "$DRACUT_SYSTEMD" ] && reloadsysrootmountunit="yes"

+        fi

+    else

+        die "Failed to find a root filesystem in $SQUASHED."

+        exit 1

     fi

 else

     # we might have an embedded fs image to use as rootfs (uncompressed live)

@@ -312,8 +321,8 @@ else

     fi

 fi

-if [ -n "$FSIMG" ] ; then

-    if [ -n "$writable_fsimg" ] ; then

+if [ -n "$FSIMG" ]; then

+    if [ -n "$writable_fsimg" ]; then

         # mount the provided filesystem read/write

         echo "Unpacking live filesystem (may take some time)" > /dev/kmsg

         mkdir -m 0755 /run/initramfs/fsimg/

@@ -336,9 +345,13 @@ if [ -n "$FSIMG" ] ; then

             setup=yes

         fi

     fi

-    BASE_LOOPDEV=$(losetup -f --show $opt $FSIMG)

-    sz=$(blockdev --getsz $BASE_LOOPDEV)

-    if [ "$setup" == rw ]; then

+    if [ "$FSIMG" = "$SQUASHED" ]; then

+        BASE_LOOPDEV=$SQUASHED_LOOPDEV

+    else

+        BASE_LOOPDEV=$(losetup -f --show $opt $FSIMG)

+        sz=$(blockdev --getsz $BASE_LOOPDEV)

+    fi

+    if [ "$setup" = rw ]; then

         echo 0 $sz linear $BASE_LOOPDEV 0 | dmsetup create live-rw

     else

         # Add a DM snapshot or OverlayFS for writes.

@@ -346,8 +359,6 @@ if [ -n "$FSIMG" ] ; then

     fi

 fi

-[ -e "$SQUASHED" ] && [ -z "$overlayfs" ] && umount -l /run/initramfs/squashfs

-

 if [ -b "$OSMIN_LOOPDEV" ]; then

     # set up the devicemapper snapshot device, which will merge

     # the normal live fs image, and the delta, into a minimzied fs image

@@ -380,17 +391,17 @@ if [ -n "$overlayfs" ]; then

         'lowerdir=/run/rootfsbase,upperdir=/run/overlayfs,workdir=/run/ovlwork' \

         "$NEWROOT" > $hookdir/mount/01-$$-live.sh

     fi

-    _dev=/run/rootfsbase

 else

-    _dev=/dev/mapper/live-rw

     if [ -z "$DRACUT_SYSTEMD" ]; then

         [ -n "$ROOTFLAGS" ] && ROOTFLAGS="-o $ROOTFLAGS"

         printf 'mount %s /dev/mapper/live-rw %s\n' "$ROOTFLAGS" "$NEWROOT" > $hookdir/mount/01-$$-live.sh

     fi

-    ln -s $BASE_LOOPDEV /run/rootfsbase

 fi

-ln -s $_dev /dev/root

+[ -e "$SQUASHED" ] && umount -l /run/initramfs/squashfs

+

+ln -s null /dev/root

 need_shutdown

 exit 0

+