[PATCH 0/5] vm: boot sub-VMs from dm-verity roots and enable IPE

[PATCH 0/5] vm: boot sub-VMs from dm-verity roots and enable IPE

[colbyt]

This series changes the app VM and net VM root images to boot through
dm-verity, then enables IPE kernel support for the sub-VMs and the
Spectrum host rootfs.

Each sub-VM disk now has a verity metadata partition and an EROFS root
partition.  The build installs the root hash next to the disk image, and
the VM boot paths pass that hash to a shared initramfs.  The initramfs
uses the hash to find both partitions, opens /dev/mapper/root-verity, and
switches into it read-only.

The last two patches only enable kernel support for IPE.  They do not
install an IPE policy.  With the existing generated kernel config,
enabling SECURITY_IPE also enables the dm-verity root-hash provider, so
later policy work can match files from verified roots.

This is based on current main and does not depend on the Cloud Hypervisor
v52 series.  If the v52 series lands first, the only expected overlap is
the crosvm run target context in patch 3.

Tested:

- nix-build tools -A tests.tests -o /tmp/spectrum-ipe-standalone-tools-tests --no-build-output
- nix-build vm/sys/net -o /tmp/spectrum-ipe-standalone-netvm --no-build-output
- nix-build img/app -o /tmp/spectrum-ipe-standalone-appvm --no-build-output
- built app VM, net VM, and host kernel configfiles and confirmed:
  CONFIG_SECURITY_IPE=y
  CONFIG_IPE_PROP_DM_VERITY=y
  CONFIG_DM_VERITY=m
  CONFIG_LSM="landlock,lockdown,yama,loadpin,safesetid,ipe,bpf"
- booted the net VM root image with QEMU and confirmed:
  LSM: initializing lsm=capability,landlock,yama,ipe,bpf
  device-mapper: verity: sha256 using "sha256-lib"
  erofs (device dm-0): mounted
  s6-linux-init version 1.1.3.0

colbyt (5):
  vm: generate dm-verity metadata for sub-VM roots
  vm: add shared initramfs for verified roots
  vm: boot sub-VMs through verified roots
  vm: enable IPE in sub-VM kernels
  host/rootfs: enable IPE in the host kernel

 host/rootfs/default.nix                   |   6 +-
 img/app/Makefile                          |  41 ++++++--
 img/app/default.nix                       |  14 ++-
 pkgs/default.nix                          |   1 +
 tools/start-vmm/ch.rs                     |   3 +-
 tools/start-vmm/lib.rs                    |  15 ++-
 tools/start-vmm/tests/vm_command-basic.rs |  18 +++-
 vm-lib/make-vm.nix                        |   2 +
 vm/initramfs/Makefile                     |  37 +++++++
 vm/initramfs/default.nix                  | 113 ++++++++++++++++++++++
 vm/initramfs/etc/fstab                    |   5 +
 vm/initramfs/etc/getuuids                 |  18 ++++
 vm/initramfs/etc/init                     |  58 +++++++++++
 vm/initramfs/etc/mdev.conf                |   4 +
 vm/initramfs/etc/probe                    |  17 ++++
 vm/sys/net/Makefile                       |  41 ++++++--
 vm/sys/net/default.nix                    |  14 ++-
 17 files changed, 380 insertions(+), 27 deletions(-)
 create mode 100644 vm/initramfs/Makefile
 create mode 100644 vm/initramfs/default.nix
 create mode 100644 vm/initramfs/etc/fstab
 create mode 100755 vm/initramfs/etc/getuuids
 create mode 100755 vm/initramfs/etc/init
 create mode 100644 vm/initramfs/etc/mdev.conf
 create mode 100755 vm/initramfs/etc/probe

-- 
2.54.0

[PATCH 1/5] vm: generate dm-verity metadata for sub-VM roots

[colbyt]

Generate dm-verity metadata for the app VM and net VM root filesystems,
and install the root hash next to each VM disk image.

The disk layout now has separate verity and root partitions. Both
partition UUIDs are derived from the root hash, matching the host rootfs
approach and giving the initramfs enough information to find the two
partitions without hard-coded UUIDs.

Signed-off-by: colbyt <colby@colbyt.com>
---
 img/app/Makefile       | 21 +++++++++++++++++++--
 img/app/default.nix    |  6 +++++-
 vm/sys/net/Makefile    | 21 +++++++++++++++++++--
 vm/sys/net/default.nix |  6 +++++-
 4 files changed, 48 insertions(+), 6 deletions(-)

diff --git a/img/app/Makefile b/img/app/Makefile
index fd75e6e..6e47570 100644
--- a/img/app/Makefile
+++ b/img/app/Makefile
@@ -15,6 +15,7 @@ VMM = cloud-hypervisor
 
 HOST_BUILD_FILES = \
 	$(imgdir)/appvm/blk/root.img \
+	$(imgdir)/appvm/blk/rootfs.verity.roothash \
 	$(imgdir)/appvm/vmlinux
 
 all: $(HOST_BUILD_FILES)
@@ -24,10 +25,26 @@ $(imgdir)/appvm/vmlinux: $(KERNEL)
 	mkdir -p $$(dirname $@)
 	cp $(KERNEL) $@
 
-$(imgdir)/appvm/blk/root.img: ../../scripts/make-gpt.sh ../../scripts/sfdisk-field.awk build/rootfs.erofs
+build/rootfs.verity-timestamp: build/rootfs.erofs
+	mkdir -p build
+	$(VERITYSETUP) format \
+		--root-hash-file build/rootfs.verity.roothash \
+		-- build/rootfs.erofs build/rootfs.verity.superblock
+	echo >> build/rootfs.verity.roothash
+	awk 'length == 64 && !/[^0-9a-f]/ { valid = 1 } END { exit !valid }' \
+		build/rootfs.verity.roothash
+	touch -- $@
+build/rootfs.verity.roothash build/rootfs.verity.superblock: build/rootfs.verity-timestamp
+
+$(imgdir)/appvm/blk/rootfs.verity.roothash: build/rootfs.verity.roothash
+	mkdir -p $$(dirname $@)
+	cp build/rootfs.verity.roothash $@
+
+$(imgdir)/appvm/blk/root.img: ../../scripts/format-uuid.sh ../../scripts/make-gpt.sh ../../scripts/sfdisk-field.awk build/rootfs.erofs build/rootfs.verity.roothash build/rootfs.verity.superblock
 	mkdir -p $$(dirname $@)
 	../../scripts/make-gpt.sh $@.tmp \
-	    build/rootfs.erofs:root:5460386f-2203-4911-8694-91400125c604:root
+	    build/rootfs.verity.superblock:verity:$$(../../scripts/format-uuid.sh "$$(dd if=build/rootfs.verity.roothash bs=32 skip=1 count=1 status=none)"):verity \
+	    build/rootfs.erofs:root:$$(../../scripts/format-uuid.sh "$$(head -c 32 build/rootfs.verity.roothash)"):root
 	mv $@.tmp $@
 
 DIRS = dev host run mnt proc sys tmp \
diff --git a/img/app/default.nix b/img/app/default.nix
index 71e6fa0..ebd37e7 100644
--- a/img/app/default.nix
+++ b/img/app/default.nix
@@ -4,6 +4,7 @@
 import ../../lib/call-package.nix (
 { spectrum-app-tools, spectrum-build-tools, src, terminfo
 , lib, appimageTools, buildFHSEnv, runCommand, stdenvNoCC, writeClosure
+, cryptsetup
 , erofs-utils, jq, s6-rc, util-linux, xorg
 , cacert, linux_latest
 }:
@@ -104,6 +105,7 @@ stdenvNoCC.mkDerivation {
     fileset = lib.fileset.intersection src (lib.fileset.unions [
       ./.
       ../../lib/common.mk
+      ../../scripts/format-uuid.sh
       ../../scripts/make-erofs.sh
       ../../scripts/make-gpt.sh
       ../../scripts/sfdisk-field.awk
@@ -111,7 +113,9 @@ stdenvNoCC.mkDerivation {
   };
   sourceRoot = "source/img/app";
 
-  nativeBuildInputs = [ erofs-utils jq spectrum-build-tools s6-rc util-linux ];
+  nativeBuildInputs = [
+    cryptsetup erofs-utils jq spectrum-build-tools s6-rc util-linux
+  ];
 
   env = {
     KERNEL = "${kernel}/${baseNameOf kernelTarget}";
diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile
index 653ef13..c6cf7f2 100644
--- a/vm/sys/net/Makefile
+++ b/vm/sys/net/Makefile
@@ -14,6 +14,7 @@ VMM = cloud-hypervisor
 
 HOST_BUILD_FILES = \
 	$(vmdir)/netvm/blk/root.img \
+	$(vmdir)/netvm/blk/rootfs.verity.roothash \
 	$(vmdir)/netvm/vmlinux
 
 all: $(HOST_BUILD_FILES)
@@ -23,10 +24,26 @@ $(vmdir)/netvm/vmlinux: $(KERNEL)
 	mkdir -p $$(dirname $@)
 	cp $(KERNEL) $@
 
-$(vmdir)/netvm/blk/root.img: ../../../scripts/make-gpt.sh ../../../scripts/sfdisk-field.awk build/rootfs.erofs
+build/rootfs.verity-timestamp: build/rootfs.erofs
+	mkdir -p build
+	$(VERITYSETUP) format \
+		--root-hash-file build/rootfs.verity.roothash \
+		-- build/rootfs.erofs build/rootfs.verity.superblock
+	echo >> build/rootfs.verity.roothash
+	awk 'length == 64 && !/[^0-9a-f]/ { valid = 1 } END { exit !valid }' \
+		build/rootfs.verity.roothash
+	touch -- $@
+build/rootfs.verity.roothash build/rootfs.verity.superblock: build/rootfs.verity-timestamp
+
+$(vmdir)/netvm/blk/rootfs.verity.roothash: build/rootfs.verity.roothash
+	mkdir -p $$(dirname $@)
+	cp build/rootfs.verity.roothash $@
+
+$(vmdir)/netvm/blk/root.img: ../../../scripts/format-uuid.sh ../../../scripts/make-gpt.sh ../../../scripts/sfdisk-field.awk build/rootfs.erofs build/rootfs.verity.roothash build/rootfs.verity.superblock
 	mkdir -p $$(dirname $@)
 	../../../scripts/make-gpt.sh $@.tmp \
-	    build/rootfs.erofs:root:ea21da27-0391-48da-9235-9d2ab2ca7844:root
+	    build/rootfs.verity.superblock:verity:$$(../../../scripts/format-uuid.sh "$$(dd if=build/rootfs.verity.roothash bs=32 skip=1 count=1 status=none)"):verity \
+	    build/rootfs.erofs:root:$$(../../../scripts/format-uuid.sh "$$(head -c 32 build/rootfs.verity.roothash)"):root
 	mv $@.tmp $@
 
 DIRS = dev etc/s6-linux-init/env proc run sys
diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix
index a722b02..8549bf3 100644
--- a/vm/sys/net/default.nix
+++ b/vm/sys/net/default.nix
@@ -6,6 +6,7 @@ import ../../../lib/call-package.nix (
 pkgsMusl.callPackage (
 
 { lib, stdenvNoCC, nixos, runCommand, writeClosure
+, cryptsetup
 , erofs-utils, jq, s6-rc, util-linux, xorg
 , busybox, dbus, execline, iwd, kmod, linuxPackagesFor, linux_latest
 , mdevd, nftables, s6, s6-linux-init, spectrum-driver-tools, xdp-tools
@@ -110,6 +111,7 @@ stdenvNoCC.mkDerivation {
     fileset = lib.fileset.intersection src (lib.fileset.unions [
       ./.
       ../../../lib/common.mk
+      ../../../scripts/format-uuid.sh
       ../../../scripts/make-erofs.sh
       ../../../scripts/make-gpt.sh
       ../../../scripts/sfdisk-field.awk
@@ -117,7 +119,9 @@ stdenvNoCC.mkDerivation {
   };
   sourceRoot = "source/vm/sys/net";
 
-  nativeBuildInputs = [ erofs-utils jq spectrum-build-tools s6-rc util-linux ];
+  nativeBuildInputs = [
+    cryptsetup erofs-utils jq spectrum-build-tools s6-rc util-linux
+  ];
 
   env = {
     KERNEL = "${kernel}/${baseNameOf kernelTarget}";
-- 
2.54.0

[PATCH 2/5] vm: add shared initramfs for verified roots

[colbyt]

Add a shared initramfs for the app VM and net VM. It loads dm-mod and
dm-verity, finds the root and verity partitions from the supplied root
hash, opens root-verity, and switches into it as the read-only root.

Doing this in the initramfs lets the VM kernels keep dm-verity modular
instead of forcing it built in.

Signed-off-by: colbyt <colby@colbyt.com>
---
 img/app/Makefile           |   5 ++
 img/app/default.nix        |   7 ++-
 pkgs/default.nix           |   1 +
 vm/initramfs/Makefile      |  37 ++++++++++++
 vm/initramfs/default.nix   | 113 +++++++++++++++++++++++++++++++++++++
 vm/initramfs/etc/fstab     |   5 ++
 vm/initramfs/etc/getuuids  |  18 ++++++
 vm/initramfs/etc/init      |  58 +++++++++++++++++++
 vm/initramfs/etc/mdev.conf |   4 ++
 vm/initramfs/etc/probe     |  17 ++++++
 vm/sys/net/Makefile        |   5 ++
 vm/sys/net/default.nix     |   7 ++-
 12 files changed, 275 insertions(+), 2 deletions(-)
 create mode 100644 vm/initramfs/Makefile
 create mode 100644 vm/initramfs/default.nix
 create mode 100644 vm/initramfs/etc/fstab
 create mode 100755 vm/initramfs/etc/getuuids
 create mode 100755 vm/initramfs/etc/init
 create mode 100644 vm/initramfs/etc/mdev.conf
 create mode 100755 vm/initramfs/etc/probe

diff --git a/img/app/Makefile b/img/app/Makefile
index 6e47570..3f60ef8 100644
--- a/img/app/Makefile
+++ b/img/app/Makefile
@@ -14,6 +14,7 @@ imgdir = $(libdir)/spectrum/img
 VMM = cloud-hypervisor
 
 HOST_BUILD_FILES = \
+	$(imgdir)/appvm/initramfs \
 	$(imgdir)/appvm/blk/root.img \
 	$(imgdir)/appvm/blk/rootfs.verity.roothash \
 	$(imgdir)/appvm/vmlinux
@@ -21,6 +22,10 @@ HOST_BUILD_FILES = \
 all: $(HOST_BUILD_FILES)
 .PHONY: all
 
+$(imgdir)/appvm/initramfs: $(INITRAMFS)
+	mkdir -p $$(dirname $@)
+	cp $(INITRAMFS) $@
+
 $(imgdir)/appvm/vmlinux: $(KERNEL)
 	mkdir -p $$(dirname $@)
 	cp $(KERNEL) $@
diff --git a/img/app/default.nix b/img/app/default.nix
index ebd37e7..23487e3 100644
--- a/img/app/default.nix
+++ b/img/app/default.nix
@@ -2,7 +2,7 @@
 # SPDX-FileCopyrightText: 2021-2025 Alyssa Ross <hi@alyssa.is>
 
 import ../../lib/call-package.nix (
-{ spectrum-app-tools, spectrum-build-tools, src, terminfo
+{ callSpectrumPackage, spectrum-app-tools, spectrum-build-tools, src, terminfo
 , lib, appimageTools, buildFHSEnv, runCommand, stdenvNoCC, writeClosure
 , cryptsetup
 , erofs-utils, jq, s6-rc, util-linux, xorg
@@ -46,6 +46,10 @@ let
     ];
   });
 
+  vmInitramfs = callSpectrumPackage ../../vm/initramfs {
+    kernel = kernel.modules;
+  };
+
   appimageFhsenv = (buildFHSEnv (appimageTools.defaultFhsEnvArgs // {
     name = "vm-fhs-env";
     targetPkgs = pkgs: appimageTools.defaultFhsEnvArgs.targetPkgs pkgs ++ [
@@ -118,6 +122,7 @@ stdenvNoCC.mkDerivation {
   ];
 
   env = {
+    INITRAMFS = vmInitramfs;
     KERNEL = "${kernel}/${baseNameOf kernelTarget}";
     PACKAGES = runCommand "packages" {} ''
       printf "%s\n/\n" ${packagesSysroot} >$out
diff --git a/pkgs/default.nix b/pkgs/default.nix
index 4dbdfee..6d179b5 100644
--- a/pkgs/default.nix
+++ b/pkgs/default.nix
@@ -75,6 +75,7 @@ let
         ../tools/mount-flatpak/target
         ../tools/router/target
         ../tools/xdg-desktop-portal-spectrum-host/target
+        ../vm/initramfs/build
         ../vm/sys/net/build
       ]));
 
diff --git a/vm/initramfs/Makefile b/vm/initramfs/Makefile
new file mode 100644
index 0000000..03ae1b4
--- /dev/null
+++ b/vm/initramfs/Makefile
@@ -0,0 +1,37 @@
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Colby Russell <colby@colbyt.com>
+
+.POSIX:
+
+include ../../lib/common.mk
+
+dest = build/initramfs
+
+$(dest): build/local.cpio $(PACKAGES_CPIO)
+	cat build/local.cpio $(PACKAGES_CPIO) | gzip -9n > $@
+
+# etc/init isn't included in ETC_FILES, because it gets installed to
+# the root.
+ETC_FILES = etc/getuuids etc/probe etc/fstab etc/mdev.conf
+MOUNTPOINTS = dev mnt/root proc sys tmp
+
+build/local.cpio: $(ETC_FILES) etc/init build/mountpoints
+	printf "%s\n" $(ETC_FILES) | \
+	    awk '{while (length) { print; sub("/?[^/]*$$", "") }}' | \
+	    sort -u | \
+	    $(CPIO) -o $(CPIOFLAGS) > $@
+	cd etc && echo init | $(CPIO) -o $(CPIOFLAGS) -AF ../$@
+	cd build/mountpoints && printf "%s\n" $(MOUNTPOINTS) | \
+	    awk '{while (length) { print; sub("/?[^/]*$$", "") }}' | \
+	    sort -u | \
+	    $(CPIO) -o $(CPIOFLAGS) -AF ../../$@
+
+build/mountpoints:
+	rm -rf build/mountpoints
+	mkdir -p build/mountpoints
+	cd build/mountpoints && mkdir -p $(MOUNTPOINTS)
+	find build/mountpoints -mindepth 1 -exec touch -d @0 {} ';'
+
+clean:
+	rm -rf build
+.PHONY: clean
diff --git a/vm/initramfs/default.nix b/vm/initramfs/default.nix
new file mode 100644
index 0000000..59b01f0
--- /dev/null
+++ b/vm/initramfs/default.nix
@@ -0,0 +1,113 @@
+# SPDX-License-Identifier: MIT
+# SPDX-FileCopyrightText: 2026 Colby Russell <colby@colbyt.com>
+
+import ../../lib/call-package.nix (
+{ src, spectrum-build-tools, kernel ? null
+, lib, stdenvNoCC, makeModulesClosure, runCommand, writeClosure, pkgsStatic
+, busybox, cpio
+}:
+
+pkgsStatic.callPackage ({ execline, kmod, mdevd, cryptsetup, util-linuxMinimal }:
+
+let
+  inherit (lib) concatMapStringsSep filter foldl isString last split tail;
+
+  emptyFirmware = runCommand "empty-firmware" {} ''
+    mkdir -p $out/lib/firmware
+  '';
+
+  modules = if kernel == null then null else makeModulesClosure {
+    firmware = emptyFirmware;
+    inherit kernel;
+    rootModules = [ "dm-mod" "dm-verity" ];
+  };
+
+  packages = [
+    execline kmod mdevd
+
+    (cryptsetup.override {
+      programs = {
+        cryptsetup = false;
+        cryptsetup-reencrypt = false;
+        integritysetup = false;
+      };
+    })
+
+    (busybox.override {
+      enableStatic = true;
+      extraConfig = ''
+        CONFIG_DEPMOD n
+        CONFIG_FINDFS n
+        CONFIG_INSMOD n
+        CONFIG_LSMOD n
+        CONFIG_MODINFO n
+        CONFIG_MODPROBE n
+        CONFIG_RMMOD n
+      '';
+    })
+  ];
+
+  packagesSysroot = runCommand "vm-initramfs-packages-sysroot" {} ''
+    mkdir -p $out/bin
+    ln -s ${concatMapStringsSep " " (p: "${p}/bin/*") packages} $out/bin
+    ln -s /bin $out/sbin
+    ${lib.optionalString (modules != null) "cp -R ${modules}/lib $out"}
+
+    # TODO: this is a hack and we should just build the util-linux
+    # programs we want.
+    # https://lore.kernel.org/util-linux/87zgrl6ufb.fsf@alyssa.is/
+    cp ${util-linuxMinimal}/bin/{findfs,lsblk} $out/bin
+  '';
+
+  storeComponents = tail (filter isString (split "/" builtins.storeDir));
+
+  packagesCpio = runCommand "vm-initramfs-packages.cpio" {
+    nativeBuildInputs = [ cpio ];
+
+    storePaths = writeClosure [ packagesSysroot ];
+    storePrefixes = foldl
+      (acc: elem: acc ++ [ "${if acc == [] then "" else last acc}/${elem}" ])
+      []
+      storeComponents;
+
+    __structuredAttrs = true;
+    unsafeDiscardReferences = { out = true; };
+    dontFixup = true;
+  } ''
+    cd ${packagesSysroot}
+    (printf "%s\n" "''${storePrefixes[@]}" && find . $(< $storePaths)) |
+        cpio -o -H newc -R +0:+0 --reproducible > $out
+  '';
+in
+
+stdenvNoCC.mkDerivation {
+  name = "spectrum-vm-initramfs";
+
+  src = lib.fileset.toSource {
+    root = ../..;
+    fileset = lib.fileset.intersection src (lib.fileset.unions [
+      ./.
+      ../../lib/common.mk
+    ]);
+  };
+  sourceRoot = "source/vm/initramfs";
+
+  env = {
+    PACKAGES_CPIO = packagesCpio;
+  };
+
+  nativeBuildInputs = [ cpio spectrum-build-tools ];
+
+  makeFlags = [ "dest=$(out)" ];
+
+  dontInstall = true;
+
+  enableParallelBuilding = true;
+
+  __structuredAttrs = true;
+  unsafeDiscardReferences = { out = true; };
+  dontFixup = true;
+
+  passthru = { inherit packagesSysroot; };
+}
+) {}) (_: {})
diff --git a/vm/initramfs/etc/fstab b/vm/initramfs/etc/fstab
new file mode 100644
index 0000000..4387224
--- /dev/null
+++ b/vm/initramfs/etc/fstab
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: CC0-1.0
+# SPDX-FileCopyrightText: 2026 Colby Russell <colby@colbyt.com>
+devtmpfs	/dev	devtmpfs	nosuid			0	0
+proc		/proc	proc		nosuid,nodev,noexec	0	0
+sysfs		/sys	sysfs		nosuid,nodev,noexec	0	0
diff --git a/vm/initramfs/etc/getuuids b/vm/initramfs/etc/getuuids
new file mode 100755
index 0000000..62f8935
--- /dev/null
+++ b/vm/initramfs/etc/getuuids
@@ -0,0 +1,18 @@
+#!/bin/awk -f
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2021, 2023 Alyssa Ross <hi@alyssa.is>
+
+function to_uuid(hex) {
+    return substr(hex, 1, 8) "-" substr(hex, 9, 4) "-" substr(hex, 13, 4) "-" \
+        substr(hex, 17, 4) "-" substr(hex, 21)
+}
+
+BEGIN {
+    if (length(ENVIRON["roothash"]) != 64) {
+	system("echo 'roothash invalid or missing' >&2")
+	exit 1
+    }
+
+    print to_uuid(substr(ENVIRON["roothash"], 1, 32))
+    print to_uuid(substr(ENVIRON["roothash"], 33, 32))
+}
diff --git a/vm/initramfs/etc/init b/vm/initramfs/etc/init
new file mode 100755
index 0000000..4e2df2b
--- /dev/null
+++ b/vm/initramfs/etc/init
@@ -0,0 +1,58 @@
+#!/bin/execlineb -WS0
+# SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2026 Colby Russell <colby@colbyt.com>
+# SPDX-License-Identifier: EUPL-1.2+
+
+export PATH /bin
+
+if { mount -a }
+
+if { mkdir -p /sys/kernel/security }
+foreground { mount -t securityfs securityfs /sys/kernel/security }
+
+piperw 3 4
+if { fdmove 1 4 /etc/getuuids }
+fdclose 4
+# head -1 would be clearer, but it might use buffered I/O and consume
+# too much from the fifo.  Ideally we'd have line(1) from illumos.
+backtick ROOTFS_UUID { fdmove 0 3 dd count=1 bs=37 status=none }
+backtick VERITY_UUID { fdmove 0 3 dd count=1 bs=37 status=none }
+fdclose 3
+
+if { mkfifo /dev/rootfs.poll }
+
+background {
+  fdclose 3
+  mdevd -C -b134217728
+}
+importas -iu mdevd_pid !
+
+if { modprobe dm-mod }
+if { modprobe dm-verity }
+
+# Do a blocking read on the fifo to wait for mdevd to find the
+# partition.
+if {
+  redirfd -r 0 /dev/rootfs.poll
+  redirfd -w 1 /dev/null
+  head -c 1
+}
+background { kill $mdevd_pid }
+background { rm /dev/rootfs.poll }
+
+if {
+  importas -Si roothash
+  veritysetup open /dev/rootfs root-verity /dev/verity $roothash
+}
+
+background { rm /dev/rootfs /dev/verity }
+
+if { mount -o ro,nosuid,nodev /dev/mapper/root-verity /mnt/root }
+wait { $mdevd_pid }
+
+if { mount --move /proc /mnt/root/proc }
+if { mount --move /sys /mnt/root/sys }
+if { mount --move /dev /mnt/root/dev }
+
+switch_root /mnt/root
+/etc/init
diff --git a/vm/initramfs/etc/mdev.conf b/vm/initramfs/etc/mdev.conf
new file mode 100644
index 0000000..0f4655d
--- /dev/null
+++ b/vm/initramfs/etc/mdev.conf
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2021 Alyssa Ross <hi@alyssa.is>
+#
+$DEVTYPE=partition 0:0 660 +/etc/probe
diff --git a/vm/initramfs/etc/probe b/vm/initramfs/etc/probe
new file mode 100755
index 0000000..2ebeb8b
--- /dev/null
+++ b/vm/initramfs/etc/probe
@@ -0,0 +1,17 @@
+#!/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2021 Alyssa Ross <hi@alyssa.is>
+
+# Check whether we now have all the partitions we need to boot.
+
+importas -i rootfs_uuid ROOTFS_UUID
+importas -i verity_uuid VERITY_UUID
+
+backtick -E rootfs_dev { findfs PARTUUID=${rootfs_uuid} }
+backtick -E verity_dev { findfs PARTUUID=${verity_uuid} }
+
+if { rm -f /dev/rootfs /dev/verity }
+if { ln -s $rootfs_dev /dev/rootfs }
+if { ln -s $verity_dev /dev/verity }
+
+redirfd -w -nb 1 /dev/rootfs.poll echo
diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile
index c6cf7f2..d01d766 100644
--- a/vm/sys/net/Makefile
+++ b/vm/sys/net/Makefile
@@ -13,6 +13,7 @@ vmdir = $(libdir)/spectrum/vm
 VMM = cloud-hypervisor
 
 HOST_BUILD_FILES = \
+	$(vmdir)/netvm/initramfs \
 	$(vmdir)/netvm/blk/root.img \
 	$(vmdir)/netvm/blk/rootfs.verity.roothash \
 	$(vmdir)/netvm/vmlinux
@@ -20,6 +21,10 @@ HOST_BUILD_FILES = \
 all: $(HOST_BUILD_FILES)
 .PHONY: all
 
+$(vmdir)/netvm/initramfs: $(INITRAMFS)
+	mkdir -p $$(dirname $@)
+	cp $(INITRAMFS) $@
+
 $(vmdir)/netvm/vmlinux: $(KERNEL)
 	mkdir -p $$(dirname $@)
 	cp $(KERNEL) $@
diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix
index 8549bf3..7f077a9 100644
--- a/vm/sys/net/default.nix
+++ b/vm/sys/net/default.nix
@@ -2,7 +2,7 @@
 # SPDX-FileCopyrightText: 2021-2023 Alyssa Ross <hi@alyssa.is>
 
 import ../../../lib/call-package.nix (
-{ spectrum-build-tools, src, terminfo, pkgsMusl }:
+{ callSpectrumPackage, spectrum-build-tools, src, terminfo, pkgsMusl }:
 pkgsMusl.callPackage (
 
 { lib, stdenvNoCC, nixos, runCommand, writeClosure
@@ -52,6 +52,10 @@ let
     ];
   });
 
+  vmInitramfs = callSpectrumPackage ../../initramfs {
+    kernel = kernel.modules;
+  };
+
   packages = [
     dbus execline kmod mdevd s6 s6-linux-init s6-rc xdp-tools
 
@@ -124,6 +128,7 @@ stdenvNoCC.mkDerivation {
   ];
 
   env = {
+    INITRAMFS = vmInitramfs;
     KERNEL = "${kernel}/${baseNameOf kernelTarget}";
     PACKAGES = runCommand "packages" {} ''
       printf "%s\n/\n" ${packagesSysroot} >$out
-- 
2.54.0

[PATCH 3/5] vm: boot sub-VMs through verified roots

[colbyt]

Pass the generated initramfs and root hash through the app VM and net VM
boot paths for QEMU, Cloud Hypervisor, crosvm, and start-vmm.

For start-vmm, include the initramfs in the payload config and validate
the root hash before adding roothash= to the kernel command line.

Signed-off-by: colbyt <colby@colbyt.com>
---
 img/app/Makefile                          | 15 +++++++++------
 tools/start-vmm/ch.rs                     |  3 ++-
 tools/start-vmm/lib.rs                    | 15 ++++++++++++---
 tools/start-vmm/tests/vm_command-basic.rs | 18 ++++++++++++++++--
 vm-lib/make-vm.nix                        |  2 ++
 vm/sys/net/Makefile                       | 15 +++++++++------
 6 files changed, 50 insertions(+), 18 deletions(-)

diff --git a/img/app/Makefile b/img/app/Makefile
index 3f60ef8..96ba2aa 100644
--- a/img/app/Makefile
+++ b/img/app/Makefile
@@ -112,10 +112,11 @@ start-virtiofsd: scripts/start-virtiofsd.elb
 	scripts/start-virtiofsd.elb
 .PHONY: start-virtiofsd
 
-run-qemu: $(imgdir)/appvm/blk/root.img start-vhost-user-net start-virtiofsd
+run-qemu: $(imgdir)/appvm/initramfs $(imgdir)/appvm/blk/root.img $(imgdir)/appvm/blk/rootfs.verity.roothash start-vhost-user-net start-virtiofsd
 	@../../scripts/run-qemu.sh -m 256 -cpu max -kernel $(KERNEL) -vga none \
+	    -initrd $(imgdir)/appvm/initramfs \
 	    -drive file=$(imgdir)/appvm/blk/root.img,if=virtio,format=raw,readonly=on \
-	    -append "root=PARTLABEL=root nokaslr" \
+	    -append "ro roothash=$$(< $(imgdir)/appvm/blk/rootfs.verity.roothash) nokaslr" \
 	    -gdb unix:build/gdb.sock,server,nowait \
 	    -chardev socket,id=vhost-user-net,path=build/vhost-user-net.sock \
 	    -netdev vhost-user,id=net0,chardev=vhost-user-net \
@@ -133,7 +134,7 @@ run-qemu: $(imgdir)/appvm/blk/root.img start-vhost-user-net start-virtiofsd
 	    -device virtconsole,chardev=virtiocon0
 .PHONY: run-qemu
 
-run-cloud-hypervisor: $(imgdir)/appvm/blk/root.img start-vhost-user-gpu start-vhost-user-net start-virtiofsd
+run-cloud-hypervisor: $(imgdir)/appvm/initramfs $(imgdir)/appvm/blk/root.img $(imgdir)/appvm/blk/rootfs.verity.roothash start-vhost-user-gpu start-vhost-user-net start-virtiofsd
 	rm -f build/vmm.sock build/vsock.sock
 	@../../scripts/run-cloud-hypervisor.sh \
 	    --api-socket path=build/vmm.sock \
@@ -144,15 +145,17 @@ run-cloud-hypervisor: $(imgdir)/appvm/blk/root.img start-vhost-user-gpu start-vh
 	    --vsock cid=3,socket=build/vsock.sock \
 	    --net mac=02:00:00:00:00:01,vhost_user=on,socket=build/vhost-user-net.sock \
 	    --kernel $(KERNEL) \
-	    --cmdline "root=PARTLABEL=root" \
+	    --initramfs $(imgdir)/appvm/initramfs \
+	    --cmdline "ro roothash=$$(< $(imgdir)/appvm/blk/rootfs.verity.roothash)" \
 	    --console tty \
 	    --serial file=build/serial.log
 .PHONY: run-cloud-hypervisor
 
-run-crosvm: $(imgdir)/appvm/blk/root.img start-vhost-user-gpu start-virtiofsd
+run-crosvm: $(imgdir)/appvm/initramfs $(imgdir)/appvm/blk/root.img $(imgdir)/appvm/blk/rootfs.verity.roothash start-vhost-user-gpu start-virtiofsd
 	../../scripts/with-taps.elb $(CROSVM_RUN) \
 	    -b path=$(imgdir)/appvm/blk/root.img,ro=true \
-	    -p "console=ttyS0 root=PARTLABEL=root" \
+	    --initrd $(imgdir)/appvm/initramfs \
+	    -p "console=ttyS0 ro roothash=$$(< $(imgdir)/appvm/blk/rootfs.verity.roothash)" \
 	    --net tap-name=tap0,mac=02:00:00:00:00:01 \
 	    --vhost-user fs,socket=build/virtiofsd.sock \
 	    --vhost-user gpu,socket=build/vhost-user-gpu.sock \
diff --git a/tools/start-vmm/ch.rs b/tools/start-vmm/ch.rs
index 239d08e..714c929 100644
--- a/tools/start-vmm/ch.rs
+++ b/tools/start-vmm/ch.rs
@@ -56,7 +56,8 @@ pub struct MemoryConfig {
 #[derive(Serialize)]
 pub struct PayloadConfig {
     pub kernel: String,
-    pub cmdline: &'static str,
+    pub initramfs: String,
+    pub cmdline: String,
 }
 
 #[derive(Serialize)]
diff --git a/tools/start-vmm/lib.rs b/tools/start-vmm/lib.rs
index 0fa6352..99a6a43 100644
--- a/tools/start-vmm/lib.rs
+++ b/tools/start-vmm/lib.rs
@@ -9,7 +9,7 @@ mod s6;
 use std::borrow::Cow;
 use std::env::args_os;
 use std::ffi::OsStr;
-use std::fs::File;
+use std::fs::{File, read_to_string};
 use std::hash::{Hash, Hasher};
 use std::io::ErrorKind;
 use std::path::Path;
@@ -43,8 +43,16 @@ pub fn vm_config(vm_dir: &Path) -> Result<VmConfig, String> {
 
     let config_dir = vm_dir.join("config");
     let blk_dir = config_dir.join("blk");
+    let initramfs_path = config_dir.join("initramfs");
     let kernel_path = config_dir.join("vmlinux");
     let net_providers_dir = config_dir.join("providers/net");
+    let roothash_path = blk_dir.join("rootfs.verity.roothash");
+    let roothash = read_to_string(&roothash_path)
+        .map_err(|e| format!("reading {roothash_path:?}: {e}"))?;
+    let roothash = roothash.trim();
+    if roothash.len() != 64 || !roothash.bytes().all(|b| b.is_ascii_hexdigit()) {
+        return Err(format!("invalid dm-verity roothash in {roothash_path:?}"));
+    }
 
     Ok(VmConfig {
         console: ConsoleConfig {
@@ -155,10 +163,11 @@ pub fn vm_config(vm_dir: &Path) -> Result<VmConfig, String> {
         },
         payload: PayloadConfig {
             kernel: kernel_path.to_str().unwrap().to_string(),
+            initramfs: initramfs_path.to_str().unwrap().to_string(),
             #[cfg(target_arch = "x86_64")]
-            cmdline: "console=ttyS0 root=PARTLABEL=root",
+            cmdline: format!("console=ttyS0 ro roothash={roothash}"),
             #[cfg(not(target_arch = "x86_64"))]
-            cmdline: "root=PARTLABEL=root",
+            cmdline: format!("ro roothash={roothash}"),
         },
         serial: ConsoleConfig {
             mode: "File",
diff --git a/tools/start-vmm/tests/vm_command-basic.rs b/tools/start-vmm/tests/vm_command-basic.rs
index 2e9ad0c..dd4e0c9 100644
--- a/tools/start-vmm/tests/vm_command-basic.rs
+++ b/tools/start-vmm/tests/vm_command-basic.rs
@@ -11,13 +11,20 @@ fn main() -> std::io::Result<()> {
     let tmp_dir = TempDir::new()?;
 
     let vm_dir = tmp_dir.path().join("testvm");
+    let initramfs_path = vm_dir.join("config/initramfs");
     let kernel_path = vm_dir.join("config/vmlinux");
     let image_path = vm_dir.join("config/blk/root.img");
+    let roothash_path = vm_dir.join("config/blk/rootfs.verity.roothash");
 
     create_dir_all(kernel_path.parent().unwrap())?;
     create_dir_all(image_path.parent().unwrap())?;
+    File::create(&initramfs_path)?;
     File::create(&kernel_path)?;
     File::create(&image_path)?;
+    std::fs::write(
+        &roothash_path,
+        "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef\n",
+    )?;
 
     let mut config = vm_config(&vm_dir).unwrap();
 
@@ -33,10 +40,17 @@ fn main() -> std::io::Result<()> {
         "/run/service/vm-services/instance/testvm/data/service/vhost-user-fs/env/virtiofsd.sock";
     assert_eq!(fs1.socket, expected);
     assert_eq!(PathBuf::from(config.payload.kernel), kernel_path);
+    assert_eq!(PathBuf::from(config.payload.initramfs), initramfs_path);
     #[cfg(target_arch = "x86_64")]
-    assert_eq!(config.payload.cmdline, "console=ttyS0 root=PARTLABEL=root");
+    assert_eq!(
+        config.payload.cmdline,
+        "console=ttyS0 ro roothash=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+    );
     #[cfg(not(target_arch = "x86_64"))]
-    assert_eq!(config.payload.cmdline, "root=PARTLABEL=root");
+    assert_eq!(
+        config.payload.cmdline,
+        "ro roothash=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+    );
     assert_eq!(config.memory.size, 0x40000000);
     assert!(config.memory.shared);
     assert_eq!(config.serial.mode, "File");
diff --git a/vm-lib/make-vm.nix b/vm-lib/make-vm.nix
index 7847883..3a2d941 100644
--- a/vm-lib/make-vm.nix
+++ b/vm-lib/make-vm.nix
@@ -54,6 +54,8 @@ runCommand "spectrum-vm" {
   popd
 
   ln -s /usr/lib/spectrum/img/appvm/blk/root.img "$out/blk"
+  ln -s /usr/lib/spectrum/img/appvm/blk/rootfs.verity.roothash "$out/blk"
+  ln -s /usr/lib/spectrum/img/appvm/initramfs "$out"
   ln -s /usr/lib/spectrum/img/appvm/vmlinux "$out"
 ''
 ) {}
diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile
index d01d766..1e07850 100644
--- a/vm/sys/net/Makefile
+++ b/vm/sys/net/Makefile
@@ -86,10 +86,11 @@ start-vhost-user-net:
 	mkdir -p build
 	../../../scripts/start-passt.elb
 
-run-qemu: $(vmdir)/netvm/blk/root.img
+run-qemu: $(vmdir)/netvm/initramfs $(vmdir)/netvm/blk/root.img $(vmdir)/netvm/blk/rootfs.verity.roothash
 	@../../../scripts/run-qemu.sh -m 256 -cpu max -kernel $(KERNEL) -vga none \
+	    -initrd $(vmdir)/netvm/initramfs \
 	    -drive file=$(vmdir)/netvm/blk/root.img,if=virtio,format=raw,readonly=on \
-	    -append "root=PARTLABEL=root nokaslr" \
+	    -append "ro roothash=$$(< $(vmdir)/netvm/blk/rootfs.verity.roothash) nokaslr" \
 	    -gdb unix:build/gdb.sock,server,nowait \
 	    -netdev user,id=net0 \
 	    -device e1000e,netdev=net0 \
@@ -101,7 +102,7 @@ run-qemu: $(vmdir)/netvm/blk/root.img
 	    -device virtconsole,chardev=virtiocon0
 .PHONY: run-qemu
 
-run-cloud-hypervisor: $(vmdir)/netvm/blk/root.img start-vhost-user-net
+run-cloud-hypervisor: $(vmdir)/netvm/initramfs $(vmdir)/netvm/blk/root.img $(vmdir)/netvm/blk/rootfs.verity.roothash start-vhost-user-net
 	rm -f build/vmm.sock
 	@../../../scripts/with-taps.elb \
 	    ../../../scripts/run-cloud-hypervisor.sh \
@@ -110,15 +111,17 @@ run-cloud-hypervisor: $(vmdir)/netvm/blk/root.img start-vhost-user-net
 	    --disk path=$(vmdir)/netvm/blk/root.img,readonly=on \
 	    --net vhost_user=on,socket=build/vhost-user-net.sock tap=tap1,mac=02:01:00:00:00:01 \
 	    --kernel $(KERNEL) \
-	    --cmdline "root=PARTLABEL=root" \
+	    --initramfs $(vmdir)/netvm/initramfs \
+	    --cmdline "ro roothash=$$(< $(vmdir)/netvm/blk/rootfs.verity.roothash)" \
 	    --console tty \
 	    --serial file=build/serial.log
 .PHONY: run-cloud-hypervisor
 
-run-crosvm: $(vmdir)/netvm/blk/root.img
+run-crosvm: $(vmdir)/netvm/initramfs $(vmdir)/netvm/blk/root.img $(vmdir)/netvm/blk/rootfs.verity.roothash
 	../../../scripts/with-taps.elb $(CROSVM_RUN) \
 	    -b path=$(vmdir)/netvm/blk/root.img,ro=true \
-	    -p "console=ttyS0 root=PARTLABEL=root" \
+	    --initrd $(vmdir)/netvm/initramfs \
+	    -p "console=ttyS0 ro roothash=$$(< $(vmdir)/netvm/blk/rootfs.verity.roothash)" \
 	    --net tap-name=tap0 \
 	    --net tap-name=tap1,mac=02:01:00:00:00:01 \
 	    --serial type=file,hardware=serial,path=build/serial.log \
-- 
2.54.0

[PATCH 4/5] vm: enable IPE in sub-VM kernels

[colbyt]

Enable the IPE LSM in the app VM and net VM kernels.

The generated kernel config already enables audit, audit syscall support,
securityfs, and an LSM order containing ipe. Enabling SECURITY_IPE also
selects the dm-verity root-hash provider, which lets later policy work
match files that come from the verified root.

This does not install an IPE policy yet.

Signed-off-by: colbyt <colby@colbyt.com>
---
 img/app/default.nix    | 1 +
 vm/sys/net/default.nix | 1 +
 2 files changed, 2 insertions(+)

diff --git a/img/app/default.nix b/img/app/default.nix
index 23487e3..8496755 100644
--- a/img/app/default.nix
+++ b/img/app/default.nix
@@ -33,6 +33,7 @@ let
       FRAMEBUFFER_CONSOLE_DEFERRED_TAKEOVER = lib.mkForce unset;
       FRAMEBUFFER_CONSOLE_DETECT_PRIMARY = lib.mkForce unset;
       FRAMEBUFFER_CONSOLE_ROTATION = lib.mkForce unset;
+      SECURITY_IPE = yes;
       RC_CORE = lib.mkForce unset;
       VIRTIO = yes;
       VIRTIO_BLK = yes;
diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix
index 7f077a9..2f7e31d 100644
--- a/vm/sys/net/default.nix
+++ b/vm/sys/net/default.nix
@@ -39,6 +39,7 @@ let
       FRAMEBUFFER_CONSOLE_DEFERRED_TAKEOVER = lib.mkForce unset;
       FRAMEBUFFER_CONSOLE_DETECT_PRIMARY = lib.mkForce unset;
       FRAMEBUFFER_CONSOLE_ROTATION = lib.mkForce unset;
+      SECURITY_IPE = yes;
       RC_CORE = lib.mkForce unset;
       VIRTIO = yes;
       VIRTIO_BLK = yes;
-- 
2.54.0

[PATCH 5/5] host/rootfs: enable IPE in the host kernel

[colbyt]

Enable the IPE LSM in the host kernel used by the Spectrum rootfs,
including the QEMU host-rootfs run path.

The generated kernel config already enables audit, audit syscall support,
securityfs, and an LSM order containing ipe. Enabling SECURITY_IPE also
selects the dm-verity root-hash provider for the host rootfs.

This only enables kernel support. It does not install an IPE policy.

Signed-off-by: colbyt <colby@colbyt.com>
---
 host/rootfs/default.nix | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix
index 6bfeefb..79baea4 100644
--- a/host/rootfs/default.nix
+++ b/host/rootfs/default.nix
@@ -51,7 +51,11 @@ let
     system.stateVersion = trivial.release;
   });
 
-  kernel = linux_latest;
+  kernel = linux_latest.override {
+    structuredExtraConfig = with lib.kernel; {
+      SECURITY_IPE = yes;
+    };
+  };
 
   appvm = callSpectrumPackage ../../img/app { inherit (foot) terminfo; };
   netvm = callSpectrumPackage ../../vm/sys/net { inherit (foot) terminfo; };
-- 
2.54.0

Re: [PATCH 0/5] vm: boot sub-VMs from dm-verity roots and enable IPE

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 1459 bytes --]

colbyt <colby@colbyt.com> writes:

> This series changes the app VM and net VM root images to boot through
> dm-verity, then enables IPE kernel support for the sub-VMs and the
> Spectrum host rootfs.
>
> Each sub-VM disk now has a verity metadata partition and an EROFS root
> partition.  The build installs the root hash next to the disk image, and
> the VM boot paths pass that hash to a shared initramfs.  The initramfs
> uses the hash to find both partitions, opens /dev/mapper/root-verity, and
> switches into it read-only.
>
> The last two patches only enable kernel support for IPE.  They do not
> install an IPE policy.  With the existing generated kernel config,
> enabling SECURITY_IPE also enables the dm-verity root-hash provider, so
> later policy work can match files from verified roots.

Would you mind explaining your motivation / the direction you envisage
here in a bit more detail?  Given that the VM gets its root file system
and the corresponding verity hash from the same place (the host file
system), there's no immediate security benefit from using dm-verity,
right?  Is the idea rather to enable some sort of cool IPE thing for
which dm-verity is a requirement, but that's not implemented yet as part
of this series?

(I really appreciate the quality of your patches BTW — you've obviously
gone to some effort to identify and follow all the tiny little
conventions we have in the codebase.  Thank you!)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: [PATCH 0/5] vm: boot sub-VMs from dm-verity roots and enable IPE

[colby]

Yes this is me laying the groundwork for/ trying to solve 

"Look into Integrity Policy Enhancement LSM
Apparently this enforces that executables must be authenticated by dm-verity. Could we use this?"

I may have forgotten to push all the files but I got it implemented and working --- only I did not want to push actual IPE policies in a patch, as that has a substantial possibility to break things.



On Friday, June 26th, 2026 at 8:15 AM, Alyssa Ross <hi@alyssa.is> wrote:

> colbyt <colby@colbyt.com> writes:
> 
> > This series changes the app VM and net VM root images to boot through
> > dm-verity, then enables IPE kernel support for the sub-VMs and the
> > Spectrum host rootfs.
> >
> > Each sub-VM disk now has a verity metadata partition and an EROFS root
> > partition.  The build installs the root hash next to the disk image, and
> > the VM boot paths pass that hash to a shared initramfs.  The initramfs
> > uses the hash to find both partitions, opens /dev/mapper/root-verity, and
> > switches into it read-only.
> >
> > The last two patches only enable kernel support for IPE.  They do not
> > install an IPE policy.  With the existing generated kernel config,
> > enabling SECURITY_IPE also enables the dm-verity root-hash provider, so
> > later policy work can match files from verified roots.
> 
> Would you mind explaining your motivation / the direction you envisage
> here in a bit more detail?  Given that the VM gets its root file system
> and the corresponding verity hash from the same place (the host file
> system), there's no immediate security benefit from using dm-verity,
> right?  Is the idea rather to enable some sort of cool IPE thing for
> which dm-verity is a requirement, but that's not implemented yet as part
> of this series?
> 
> (I really appreciate the quality of your patches BTW — you've obviously
> gone to some effort to identify and follow all the tiny little
> conventions we have in the codebase.  Thank you!)
>

Re: [PATCH 0/5] vm: boot sub-VMs from dm-verity roots and enable IPE

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 2227 bytes --]

colby@colbyt.com writes:

> Yes this is me laying the groundwork for/ trying to solve 
>
> "Look into Integrity Policy Enhancement LSM
> Apparently this enforces that executables must be authenticated by dm-verity. Could we use this?"
>
> I may have forgotten to push all the files but I got it implemented and working --- only I did not want to push actual IPE policies in a patch, as that has a substantial possibility to break things.

Ah, yes.  I meant IPE on the host when I wrote that.  I'll update the
text to make that clearer.  I don't immediately see any value in IPE in
VMs.

> On Friday, June 26th, 2026 at 8:15 AM, Alyssa Ross <hi@alyssa.is> wrote:
>
>> colbyt <colby@colbyt.com> writes:
>> 
>> > This series changes the app VM and net VM root images to boot through
>> > dm-verity, then enables IPE kernel support for the sub-VMs and the
>> > Spectrum host rootfs.
>> >
>> > Each sub-VM disk now has a verity metadata partition and an EROFS root
>> > partition.  The build installs the root hash next to the disk image, and
>> > the VM boot paths pass that hash to a shared initramfs.  The initramfs
>> > uses the hash to find both partitions, opens /dev/mapper/root-verity, and
>> > switches into it read-only.
>> >
>> > The last two patches only enable kernel support for IPE.  They do not
>> > install an IPE policy.  With the existing generated kernel config,
>> > enabling SECURITY_IPE also enables the dm-verity root-hash provider, so
>> > later policy work can match files from verified roots.
>> 
>> Would you mind explaining your motivation / the direction you envisage
>> here in a bit more detail?  Given that the VM gets its root file system
>> and the corresponding verity hash from the same place (the host file
>> system), there's no immediate security benefit from using dm-verity,
>> right?  Is the idea rather to enable some sort of cool IPE thing for
>> which dm-verity is a requirement, but that's not implemented yet as part
>> of this series?
>> 
>> (I really appreciate the quality of your patches BTW — you've obviously
>> gone to some effort to identify and follow all the tiny little
>> conventions we have in the codebase.  Thank you!)
>> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: [PATCH 5/5] host/rootfs: enable IPE in the host kernel

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 1139 bytes --]

colbyt <colby@colbyt.com> writes:

> Enable the IPE LSM in the host kernel used by the Spectrum rootfs,
> including the QEMU host-rootfs run path.
>
> The generated kernel config already enables audit, audit syscall support,
> securityfs, and an LSM order containing ipe. Enabling SECURITY_IPE also
> selects the dm-verity root-hash provider for the host rootfs.
>
> This only enables kernel support. It does not install an IPE policy.
>
> Signed-off-by: colbyt <colby@colbyt.com>
> ---
>  host/rootfs/default.nix | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix
> index 6bfeefb..79baea4 100644
> --- a/host/rootfs/default.nix
> +++ b/host/rootfs/default.nix
> @@ -51,7 +51,11 @@ let
>      system.stateVersion = trivial.release;
>    });
>  
> -  kernel = linux_latest;
> +  kernel = linux_latest.override {
> +    structuredExtraConfig = with lib.kernel; {
> +      SECURITY_IPE = yes;
> +    };
> +  };

Just noting that we'll get this for free when we update Nixpkgs, as it's
now enabled in Nixpkgs' default kernel config.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: [PATCH 0/5] vm: boot sub-VMs from dm-verity roots and enable IPE

[colby]

So instinctively the path I took there was to do IPE on apple end to end from host to VM in an accountability chain, but not sure of the validity of my taste there -- I just like treating internal assumptions as hostile instead of trusted -- I was hoping to achieve an inter-vm attestation system which would use things like this, but tbh did not think deeply.



On Thursday, July 2nd, 2026 at 7:22 AM, Alyssa Ross <hi@alyssa.is> wrote:

> colby@colbyt.com writes:
> 
> > Yes this is me laying the groundwork for/ trying to solve 
> >
> > "Look into Integrity Policy Enhancement LSM
> > Apparently this enforces that executables must be authenticated by dm-verity. Could we use this?"
> >
> > I may have forgotten to push all the files but I got it implemented and working --- only I did not want to push actual IPE policies in a patch, as that has a substantial possibility to break things.
> 
> Ah, yes.  I meant IPE on the host when I wrote that.  I'll update the
> text to make that clearer.  I don't immediately see any value in IPE in
> VMs.
> 
> > On Friday, June 26th, 2026 at 8:15 AM, Alyssa Ross <hi@alyssa.is> wrote:
> >
> >> colbyt <colby@colbyt.com> writes:
> >> 
> >> > This series changes the app VM and net VM root images to boot through
> >> > dm-verity, then enables IPE kernel support for the sub-VMs and the
> >> > Spectrum host rootfs.
> >> >
> >> > Each sub-VM disk now has a verity metadata partition and an EROFS root
> >> > partition.  The build installs the root hash next to the disk image, and
> >> > the VM boot paths pass that hash to a shared initramfs.  The initramfs
> >> > uses the hash to find both partitions, opens /dev/mapper/root-verity, and
> >> > switches into it read-only.
> >> >
> >> > The last two patches only enable kernel support for IPE.  They do not
> >> > install an IPE policy.  With the existing generated kernel config,
> >> > enabling SECURITY_IPE also enables the dm-verity root-hash provider, so
> >> > later policy work can match files from verified roots.
> >> 
> >> Would you mind explaining your motivation / the direction you envisage
> >> here in a bit more detail?  Given that the VM gets its root file system
> >> and the corresponding verity hash from the same place (the host file
> >> system), there's no immediate security benefit from using dm-verity,
> >> right?  Is the idea rather to enable some sort of cool IPE thing for
> >> which dm-verity is a requirement, but that's not implemented yet as part
> >> of this series?
> >> 
> >> (I really appreciate the quality of your patches BTW — you've obviously
> >> gone to some effort to identify and follow all the tiny little
> >> conventions we have in the codebase.  Thank you!)
> >> 
>

Re: [PATCH 0/5] vm: boot sub-VMs from dm-verity roots and enable IPE

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 3184 bytes --]

colby@colbyt.com writes:

> So instinctively the path I took there was to do IPE on apple end to
> end from host to VM in an accountability chain, but not sure of the
> validity of my taste there -- I just like treating internal
> assumptions as hostile instead of trusted -- I was hoping to achieve
> an inter-vm attestation system which would use things like this, but
> tbh did not think deeply.
>

I would love to see something like that.  Just needs some more design
thought.  As it stands I don't think this is giving any more assurance
to VMs than they had before, but we definitely could — e.g. we could
prove to them that they're running on an unmodified (and therefore
presumably trustworthy) Spectrum host.

>
> On Thursday, July 2nd, 2026 at 7:22 AM, Alyssa Ross <hi@alyssa.is> wrote:
>
>> colby@colbyt.com writes:
>> 
>> > Yes this is me laying the groundwork for/ trying to solve 
>> >
>> > "Look into Integrity Policy Enhancement LSM
>> > Apparently this enforces that executables must be authenticated by dm-verity. Could we use this?"
>> >
>> > I may have forgotten to push all the files but I got it implemented and working --- only I did not want to push actual IPE policies in a patch, as that has a substantial possibility to break things.
>> 
>> Ah, yes.  I meant IPE on the host when I wrote that.  I'll update the
>> text to make that clearer.  I don't immediately see any value in IPE in
>> VMs.
>> 
>> > On Friday, June 26th, 2026 at 8:15 AM, Alyssa Ross <hi@alyssa.is> wrote:
>> >
>> >> colbyt <colby@colbyt.com> writes:
>> >> 
>> >> > This series changes the app VM and net VM root images to boot through
>> >> > dm-verity, then enables IPE kernel support for the sub-VMs and the
>> >> > Spectrum host rootfs.
>> >> >
>> >> > Each sub-VM disk now has a verity metadata partition and an EROFS root
>> >> > partition.  The build installs the root hash next to the disk image, and
>> >> > the VM boot paths pass that hash to a shared initramfs.  The initramfs
>> >> > uses the hash to find both partitions, opens /dev/mapper/root-verity, and
>> >> > switches into it read-only.
>> >> >
>> >> > The last two patches only enable kernel support for IPE.  They do not
>> >> > install an IPE policy.  With the existing generated kernel config,
>> >> > enabling SECURITY_IPE also enables the dm-verity root-hash provider, so
>> >> > later policy work can match files from verified roots.
>> >> 
>> >> Would you mind explaining your motivation / the direction you envisage
>> >> here in a bit more detail?  Given that the VM gets its root file system
>> >> and the corresponding verity hash from the same place (the host file
>> >> system), there's no immediate security benefit from using dm-verity,
>> >> right?  Is the idea rather to enable some sort of cool IPE thing for
>> >> which dm-verity is a requirement, but that's not implemented yet as part
>> >> of this series?
>> >> 
>> >> (I really appreciate the quality of your patches BTW — you've obviously
>> >> gone to some effort to identify and follow all the tiny little
>> >> conventions we have in the codebase.  Thank you!)
>> >> 
>> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]