[PATCH v2 1/7] host/rootfs: make fs root directories shared

[PATCH v2 1/7] host/rootfs: make fs root directories shared

[Alyssa Ross]

We want every mount under here to show up inside the virtiofsd
sandbox.

Signed-off-by: Alyssa Ross <hi@alyssa.is>
---
v2: no change

 host/rootfs/image/usr/bin/create-vm-dependencies | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/host/rootfs/image/usr/bin/create-vm-dependencies b/host/rootfs/image/usr/bin/create-vm-dependencies
index 9e123fcf..6bf12d03 100755
--- a/host/rootfs/image/usr/bin/create-vm-dependencies
+++ b/host/rootfs/image/usr/bin/create-vm-dependencies
@@ -30,8 +30,8 @@ if {
 
   # Needs to be shared so that additional mounts under config/ (e.g. from
   # mount-flatpak) will be propagated into the virtiofsd sandbox.
-  if { mount --make-shared --rbind -o nofail /proc/self/fd/3/fs /run/fs/${1}/config }
-  if { mount --rbind -o ro /run/fs/${1} /run/fs/${1} }
+  if { mount --make-shared --rbind -o ro /run/fs/${1} /run/fs/${1} }
+  if { mount --rbind -o nofail /proc/self/fd/3/fs /run/fs/${1}/config }
 
   # Needs to be shared so that when xdg-document-portal mounts its fuse
   # filesystem at /run/doc/${1}/doc, it will propagate to /run/fs/${1}/doc.

base-commit: 227a3ea149281b6dddb0c1ba70008fffb7404c1f
-- 
2.51.0

[PATCH v2 2/7] host/rootfs: give VMs a disk-backed directory

[Alyssa Ross]

This is the first step towards persistent VM data.  For now, these
directories are never reused, and also don't get cleaned up.  Both are
things to work on in future.  Even without persistence, it's good to
not have to store everything a VM might write into its home directory
in RAM.

For AppImage and Flatpak VMs, disk-backed directories are stored on
the partition containing the AppImage or Flatpak, in the new Spectrum/
hierarchy.  This will enable locating them for later reuse.  System
VMs, on the other hand, don't have a natural partition to store data
on — there may not even be a writable partition at the time they're
launched.  Since we don't expect persistence from system VMs, we just
use a tmpfs for their "disk-backed" directories, which is reset on
every boot.  Doing it this way avoids the need for multiple routes in
img/app — it can always assume it gets a "disk" directory from the
host.  For manually imported VMs, it's up to the user where these
directories should be, or whether they should have the same behavior
as system VMs.

Signed-off-by: Alyssa Ross <hi@alyssa.is>
---
v2: reset "disk-backed" directories of system VMs between boots
v1: https://spectrum-os.org/lists/archives/spectrum-devel/20251213161637.510752-2-hi@alyssa.is/

 .../using-spectrum/creating-custom-vms.adoc   | 10 ++++--
 .../image/usr/bin/create-vm-dependencies      |  1 +
 host/rootfs/image/usr/bin/run-appimage        | 22 ++++++++++---
 host/rootfs/image/usr/bin/run-flatpak         | 15 ++++++++-
 host/rootfs/image/usr/bin/run-vmm             | 32 ++++++++++++++++++-
 host/rootfs/image/usr/bin/vm-import           | 17 +++++++++-
 img/app/Makefile                              |  2 +-
 img/app/image/etc/fstab                       |  1 -
 img/app/image/etc/s6-rc/app/run               |  7 ++--
 img/app/scripts/start-virtiofsd.elb           |  2 +-
 release/checks/integration/lib.c              |  3 +-
 release/checks/integration/networking.c       |  2 +-
 release/checks/integration/portal.c           |  2 +-
 13 files changed, 97 insertions(+), 19 deletions(-)

diff --git a/Documentation/using-spectrum/creating-custom-vms.adoc b/Documentation/using-spectrum/creating-custom-vms.adoc
index 229c0140..68213c89 100644
--- a/Documentation/using-spectrum/creating-custom-vms.adoc
+++ b/Documentation/using-spectrum/creating-custom-vms.adoc
@@ -13,9 +13,13 @@ configurations are directories under a dedicated parent directory, and
 the name of each configuration directory determines the name of
 the VM.  After mounting the persistent storage partition, the
 configured VMs can be made available by running `vm-import user
-/media/4e43cdc2-82b2-4d94-8a90-b6c6189312d2/vms`, replacing
-/media/4e43cdc2-82b2-4d94-8a90-b6c6189312d2/vms with the directory
-containing the VM definitions.
+/media/4e43cdc2-82b2-4d94-8a90-b6c6189312d2/vms
+/media/4e43cdc2-82b2-4d94-8a90-b6c6189312d2/Spectrum/data/spectrum/storage`,
+replacing /media/4e43cdc2-82b2-4d94-8a90-b6c6189312d2/vms with the
+directory containing the VM definitions, and
+/media/4e43cdc2-82b2-4d94-8a90-b6c6189312d2/Spectrum/data/spectrum/storage
+with the directory where disk-backed directories for the VMs should be
+created.
 
 The directory can contain the following files:
 
diff --git a/host/rootfs/image/usr/bin/create-vm-dependencies b/host/rootfs/image/usr/bin/create-vm-dependencies
index 6bf12d03..b7545425 100755
--- a/host/rootfs/image/usr/bin/create-vm-dependencies
+++ b/host/rootfs/image/usr/bin/create-vm-dependencies
@@ -6,6 +6,7 @@ if {
   mkdir -p
     /run/doc/${1}/doc
     /run/fs/${1}/config
+    /run/fs/${1}/disk
     /run/fs/${1}/doc
     /run/vm/by-id/${1}/ns
 }
diff --git a/host/rootfs/image/usr/bin/run-appimage b/host/rootfs/image/usr/bin/run-appimage
index b9464f8b..f62844f8 100755
--- a/host/rootfs/image/usr/bin/run-appimage
+++ b/host/rootfs/image/usr/bin/run-appimage
@@ -25,6 +25,18 @@ if {
   if { create-vm-dependencies $id }
 }
 
+backtick diskdir {
+  s6-setuidgid fs
+
+  backtick -E mountpoint {
+    importas -Siu 1
+    findmnt -no TARGET -T $1
+  }
+
+  if { mkdir -p -- ${mountpoint}/Spectrum/data/spectrum/storage }
+  mktemp -d -- ${mountpoint}/Spectrum/data/spectrum/storage/tmp.XXXXXX
+}
+
 if {
   s6-envuidgid fs
   s6-applyuidgid -Uzu 0
@@ -32,15 +44,17 @@ if {
   multisubstitute {
     importas -Siu id
     importas -Siu 1
+    importas -Siu diskdir
   }
 
   nsenter --preserve-credentials -S0
     --mount=/run/vm/by-id/${id}/ns/mnt
     --user=/run/vm/by-id/${id}/ns/user
-  cd /run/fs/${id}/config
-  if { redirfd -w 1 type echo appimage }
-  if { touch run }
-  mount --bind $1 run
+  cd /run/fs/${id}
+  if { redirfd -w 1 config/type echo appimage }
+  if { touch config/run }
+  if { mount --bind $1 config/run }
+  mount --bind -- $diskdir disk
 }
 
 importas -Siu id
diff --git a/host/rootfs/image/usr/bin/run-flatpak b/host/rootfs/image/usr/bin/run-flatpak
index 2d3e7ea0..9a7ffa33 100755
--- a/host/rootfs/image/usr/bin/run-flatpak
+++ b/host/rootfs/image/usr/bin/run-flatpak
@@ -25,21 +25,34 @@ if {
   if { create-vm-dependencies $id }
 }
 
+backtick diskdir {
+  s6-setuidgid fs
+
+  importas -Siu 1
+
+  if { mkdir -p -- ${1}/Spectrum/data/spectrum/storage }
+  mktemp -d -- ${1}/Spectrum/data/spectrum/storage/tmp.XXXXXX
+}
+
 if {
   s6-envuidgid fs
   s6-applyuidgid -Uzu 0
 
   multisubstitute {
     importas -Siu id
+    importas -Siu diskdir
     elgetpositionals
   }
 
   nsenter --preserve-credentials -S0
     --mount=/run/vm/by-id/${id}/ns/mnt
     --user=/run/vm/by-id/${id}/ns/user
+
   cd /run/fs/${id}/config
   if { redirfd -w 1 type echo flatpak }
-  mount-flatpak $@
+  if { mount-flatpak $@ }
+
+  mount --bind -- $diskdir /run/fs/${id}/disk
 }
 
 importas -Siu id
diff --git a/host/rootfs/image/usr/bin/run-vmm b/host/rootfs/image/usr/bin/run-vmm
index 7c2b9af5..4661f5f5 100755
--- a/host/rootfs/image/usr/bin/run-vmm
+++ b/host/rootfs/image/usr/bin/run-vmm
@@ -1,7 +1,37 @@
-#!/bin/execlineb -s0
+#!/bin/execlineb
 # SPDX-License-Identifier: EUPL-1.2+
 # SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
 
+if {
+  backtick -D "" mnt {
+    importas -Siu 1
+    nsenter --mount=/run/vm/by-id/${1}/ns/mnt
+    findmnt -no FSTYPE,SOURCE /run/fs/${1}/disk
+  }
+
+  multisubstitute {
+    importas -Siu mnt
+    importas -Siu 1
+  }
+
+  case $mnt {
+    "^$|^tmpfs fallback$" {
+      s6-envuidgid fs
+      s6-applyuidgid -Uzu 0
+      nsenter --preserve-credentials -S0
+        --mount=/run/vm/by-id/${1}/ns/mnt
+        --user=/run/vm/by-id/${1}/ns/user
+      foreground {
+        redirfd -w 2 /dev/null
+        umount -- /run/fs/${1}/disk
+      }
+      mount -t tmpfs -o mode=0700 -- fallback /run/fs/${1}/disk
+    }
+  }
+}
+
+elgetpositionals
+
 s6-ipcserver-socketbinder -B /run/vm/by-id/${1}/vmm
 
 getpid -E vmm_pid
diff --git a/host/rootfs/image/usr/bin/vm-import b/host/rootfs/image/usr/bin/vm-import
index 014eab87..e931ddd7 100755
--- a/host/rootfs/image/usr/bin/vm-import
+++ b/host/rootfs/image/usr/bin/vm-import
@@ -1,4 +1,4 @@
-#!/bin/execlineb -S2
+#!/bin/execlineb -S3
 # SPDX-License-Identifier: EUPL-1.2+
 # SPDX-FileCopyrightText: 2023-2024 Alyssa Ross <hi@alyssa.is>
 
@@ -20,4 +20,19 @@ if { ln -s -- /run/service/vmm/instance/${id} /run/vm/by-id/${id}/service }
 
 if { create-vm-dependencies $id }
 
+if {
+  case $# {
+    3 {
+      s6-envuidgid fs
+      s6-applyuidgid -Uzu 0
+      nsenter --preserve-credentials -S0
+	--mount=/run/vm/by-id/${id}/ns/mnt
+	--user=/run/vm/by-id/${id}/ns/user
+
+      if { mkdir -p -- ${3}/${name} }
+      mount --bind -- ${3}/${name} /run/fs/${id}/disk
+    }
+  }
+}
+
 s6-instance-create -- /run/service/vmm $id
diff --git a/img/app/Makefile b/img/app/Makefile
index 7e3d05b2..2e720a91 100644
--- a/img/app/Makefile
+++ b/img/app/Makefile
@@ -30,7 +30,7 @@ $(imgdir)/appvm/blk/root.img: ../../scripts/make-gpt.sh ../../scripts/sfdisk-fie
 	    build/rootfs.erofs:root:5460386f-2203-4911-8694-91400125c604:root
 	mv $@.tmp $@
 
-DIRS = dev home/user host run mnt proc sys tmp \
+DIRS = dev host run mnt proc sys tmp \
 	etc/s6-linux-init/run-image/pipewire \
 	etc/s6-linux-init/run-image/service \
 	etc/s6-linux-init/run-image/user \
diff --git a/img/app/image/etc/fstab b/img/app/image/etc/fstab
index 5f78ab87..f51eace0 100644
--- a/img/app/image/etc/fstab
+++ b/img/app/image/etc/fstab
@@ -5,4 +5,3 @@ devpts	/dev/pts	devpts	nosuid,noexec,gid=5,mode=620	0	0
 tmpfs	/dev/shm	tmpfs	nosuid,nodev				0	0
 sysfs	/sys		sysfs	nosuid,nodev,noexec			0	0
 tmpfs	/tmp		tmpfs	nosuid,nodev				0	0
-tmpfs	/home/user	tmpfs	nodev,mode=0700,uid=1000,gid=1000	0	0
diff --git a/img/app/image/etc/s6-rc/app/run b/img/app/image/etc/s6-rc/app/run
index f91877d4..f36d153c 100755
--- a/img/app/image/etc/s6-rc/app/run
+++ b/img/app/image/etc/s6-rc/app/run
@@ -4,11 +4,12 @@
 
 export TMPDIR /run
 
-export HOME /home/user
-cd /home/user
-
 if { /etc/mdev/wait virtiofs-host }
 
+if { install -do user -g user /host/disk/home }
+export HOME /host/disk/home
+cd /host/disk/home
+
 foreground {
   redirfd -r 0 /host/config/type
   withstdinas -E type
diff --git a/img/app/scripts/start-virtiofsd.elb b/img/app/scripts/start-virtiofsd.elb
index 9efb436b..d861a22b 100755
--- a/img/app/scripts/start-virtiofsd.elb
+++ b/img/app/scripts/start-virtiofsd.elb
@@ -7,7 +7,7 @@ background {
   if { mkdir -p build/fs }
   unshare -rUm
   if { mount -t tmpfs -o nosuid,nodev fs build/fs }
-  if { mkdir build/fs/config }
+  if { mkdir build/fs/config build/fs/disk }
   if { importas -Si CONFIG mount --rbind -- ${CONFIG}/fs build/fs/config }
   unshare --map-user 1000 --map-group 1000
   importas -SsD virtiofsd VIRTIOFSD
diff --git a/release/checks/integration/lib.c b/release/checks/integration/lib.c
index 51f6bae7..3dcce471 100644
--- a/release/checks/integration/lib.c
+++ b/release/checks/integration/lib.c
@@ -195,6 +195,7 @@ struct vm *start_qemu(struct config c)
 		"-drive", nullptr,
 		"-drive", nullptr,
 		"-smbios", nullptr,
+		"-snapshot",
 		"-m", "4G",
 		"-nodefaults",
 		"-machine", "virtualization=on",
@@ -242,7 +243,7 @@ struct vm *start_qemu(struct config c)
 
 		if (asprintf(efi_arg, "file=%s,format=raw,if=pflash,readonly=true", c.drives.efi) == -1 ||
 		    asprintf(img_arg, "file=%s,format=raw,if=virtio,readonly=true", c.drives.img) == -1 ||
-		    asprintf(user_data_arg, "file=%s,format=raw,if=virtio,readonly=true", c.drives.user_data) == -1 ||
+		    asprintf(user_data_arg, "file=%s,format=raw,if=virtio", c.drives.user_data) == -1 ||
 		    asprintf(console_arg, "type=11,value=io.systemd.stub.kernel-cmdline-extra=%s%s",
 		             c.serial.console ? "console=" : "",
 		             c.serial.console ? c.serial.console : "") == -1) {
diff --git a/release/checks/integration/networking.c b/release/checks/integration/networking.c
index 078e31fc..d581b647 100644
--- a/release/checks/integration/networking.c
+++ b/release/checks/integration/networking.c
@@ -151,7 +151,7 @@ void test(struct config c)
 	          "mkdir /run/mnt && "
 	          "mount \"$(findfs UUID=a7834806-2f82-4faf-8ac4-4f8fd8a474ca)\" /run/mnt && "
 	          "s6-rc -bu change vmm-env && "
-	          "vm-import user /run/mnt/vms && "
+	          "vm-import user /run/mnt/vms /run/mnt/storage && "
 	          "vm-start \"$(basename \"$(readlink /run/vm/by-name/user.nc)\")\" && "
 	          "tail -Fc +0 /run/log/current /run/*.log &\n",
 	          vm_console_writer(vm)) == EOF) {
diff --git a/release/checks/integration/portal.c b/release/checks/integration/portal.c
index 6ba5654a..dc459791 100644
--- a/release/checks/integration/portal.c
+++ b/release/checks/integration/portal.c
@@ -16,7 +16,7 @@ void test(struct config c)
 	          "mkdir /run/mnt && "
 	          "mount \"$(findfs UUID=a7834806-2f82-4faf-8ac4-4f8fd8a474ca)\" /run/mnt && "
 	          "s6-rc -bu change vmm-env && "
-	          "vm-import user /run/mnt/vms && "
+	          "vm-import user /run/mnt/vms /run/mnt/storage && "
 	          "(tail -Fc +0 /run/*.log &) && "
 	          "s6-svc -O /run/vm/by-name/user.portal/service && "
 	          "vm-start \"$(basename \"$(readlink /run/vm/by-name/user.portal)\")\" && "
-- 
2.51.0

[PATCH v2 3/7] host/rootfs: clean up obsolete tmp dirs on VM exit

[Alyssa Ross]

Signed-off-by: Alyssa Ross <hi@alyssa.is>
---
v2: no change

 host/rootfs/image/usr/bin/run-appimage | 7 +++++--
 host/rootfs/image/usr/bin/run-flatpak  | 7 +++++--
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/host/rootfs/image/usr/bin/run-appimage b/host/rootfs/image/usr/bin/run-appimage
index f62844f8..de851c52 100755
--- a/host/rootfs/image/usr/bin/run-appimage
+++ b/host/rootfs/image/usr/bin/run-appimage
@@ -57,7 +57,10 @@ if {
   mount --bind -- $diskdir disk
 }
 
-importas -Siu id
+multisubstitute {
+  importas -Siu diskdir
+  importas -Siu id
+}
 
 piperw 4 3
 background {
@@ -76,4 +79,4 @@ fdclose 3
 if { s6-instance-delete /run/service/vm-services $id }
 
 if { umount -R /run/vm/by-id/${id}/ns }
-rm -r /run/vm/by-id/${id} /run/configs/${id}
+rm -r -- $diskdir /run/vm/by-id/${id} /run/configs/${id}
diff --git a/host/rootfs/image/usr/bin/run-flatpak b/host/rootfs/image/usr/bin/run-flatpak
index 9a7ffa33..b47204c9 100755
--- a/host/rootfs/image/usr/bin/run-flatpak
+++ b/host/rootfs/image/usr/bin/run-flatpak
@@ -55,7 +55,10 @@ if {
   mount --bind -- $diskdir /run/fs/${id}/disk
 }
 
-importas -Siu id
+multisubstitute {
+  importas -Siu diskdir
+  importas -Siu id
+}
 
 if {
   piperw 4 3
@@ -75,4 +78,4 @@ if {
 if { s6-instance-delete -- /run/service/vm-services $id }
 
 if { umount -R /run/vm/by-id/${id}/ns }
-rm -r /run/vm/by-id/${id} /run/configs/${id}
+rm -r -- $diskdir /run/vm/by-id/${id} /run/configs/${id}
-- 
2.51.0

[PATCH v2 4/7] host/rootfs: clean up obsolete tmp dirs on mount

[Alyssa Ross]

In the ideal case, these will be cleaned up when the VM running them
exits, but there's always the possibility that there are some left
over, in which case mount time is the perfect time to clean up, when
we can be reasonably sure nothing else is still accessing them.

Signed-off-by: Alyssa Ross <hi@alyssa.is>
---
v2: no change, but thanks to rebase no longer fails to exit after
    outputting error message as identified last time.
v1: https://spectrum-os.org/lists/archives/spectrum-devel/20251213161637.510752-4-hi@alyssa.is/

 host/rootfs/image/usr/bin/mount-userdata | 42 ++++++++++++++++--------
 1 file changed, 28 insertions(+), 14 deletions(-)

diff --git a/host/rootfs/image/usr/bin/mount-userdata b/host/rootfs/image/usr/bin/mount-userdata
index 71f12c55..4b9dc8a1 100755
--- a/host/rootfs/image/usr/bin/mount-userdata
+++ b/host/rootfs/image/usr/bin/mount-userdata
@@ -7,21 +7,35 @@ backtick -D "" uuid {
   blkid -o value -s UUID -- $1
 }
 
-multisubstitute {
-  importas -Siu 0
-  importas -Siu 1
-  importas -Siu uuid
-}
-
-case $uuid {
-  "" {
-    if {
-      fdmove -c 1 2
-      printf "%s: '%s' does not have a UUID\n" $0 $1
-    }
-    exit 1
+if {
+  multisubstitute {
+    importas -Siu 0
+    importas -Siu 1
+    importas -Siu uuid
   }
+
+  case $uuid {
+    "" {
+      if {
+        fdmove -c 1 2
+        printf "%s: '%s' does not have a UUID\n" $0 $1
+      }
+      exit 1
+    }
+  }
+
+  mount -m -t btrfs -o nosuid,nodev,noexec,nosymfollow -- $1 /media/${uuid}
+}
+
+importas -Siu uuid
+
+foreground {
+  if -t { test -d /media/${uuid}/Spectrum/data/spectrum/storage }
+  find /media/${uuid}/Spectrum/data/spectrum/storage
+    -mindepth 1
+    -maxdepth 1
+    -name tmp.*
+    -exec rm -rf -- {} ;
 }
 
-if { mount -m -t btrfs -o nosuid,nodev,noexec,nosymfollow -- $1 /media/${uuid} }
 printf "%s\n" /media/${uuid}
-- 
2.51.0

[PATCH v2 5/7] tools/vm-set-persist.c: init

[Alyssa Ross]

This allows the disk-backed directory of a running VM to be made
persistent, with a user-provided name.  This is done by renaming the
directory to have a "persist." prefix rather than the "tmp." one the
cleaner will look for.  Since the VM's virtiofsd accesses the
directory via a bind mount, this rename will be unnoticeable to the
guest.

musl has quite a bit of catching up to do with the APIs used here,
which requires the use of a lot of raw syscalls.  This even applies to
some syscalls musl has wrappers for, like mkdirat(2), because musl's
mkdirat() comes from <sys/stat.h>, which defines a struct statx that's
missing the stx_mnt_id member we need.  There doesn't even seem to be
a SYS_statmount, so we use __NR_ constants throughout for consistency.

Signed-off-by: Alyssa Ross <hi@alyssa.is>
---
v2: new this round

 tools/default.nix      |   1 +
 tools/meson.build      |   4 +
 tools/vm-set-persist.c | 179 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 184 insertions(+)
 create mode 100644 tools/vm-set-persist.c

diff --git a/tools/default.nix b/tools/default.nix
index 56f41cd9..f094594f 100644
--- a/tools/default.nix
+++ b/tools/default.nix
@@ -78,6 +78,7 @@ stdenv.mkDerivation (finalAttrs: {
       ./start-vmm
       ./subprojects
       ./updates-dir-check.c
+      ./vm-set-persist.c
     ] ++ lib.optionals driverSupport [
       ./xdp-forwarder
     ]));
diff --git a/tools/meson.build b/tools/meson.build
index 666483b3..06aa24d7 100644
--- a/tools/meson.build
+++ b/tools/meson.build
@@ -37,6 +37,10 @@ if get_option('host')
   executable('updates-dir-check', 'updates-dir-check.c',
     c_args : '-D_GNU_SOURCE',
     install: true)
+
+  executable('vm-set-persist', 'vm-set-persist.c',
+    c_args : '-D_GNU_SOURCE',
+    install: true)
 endif
 
 if get_option('build')
diff --git a/tools/vm-set-persist.c b/tools/vm-set-persist.c
new file mode 100644
index 00000000..ac759504
--- /dev/null
+++ b/tools/vm-set-persist.c
@@ -0,0 +1,179 @@
+// SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+// SPDX-License-Identifier: EUPL-1.2+
+
+#include <err.h>
+#include <fcntl.h>
+#include <libgen.h>
+#include <unistd.h>
+#include <sched.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+
+// No <sys/stat.h> until musl declares stx_mnt_id.
+#include <sys/syscall.h>
+
+#include <linux/fs.h>
+#include <linux/mount.h>
+#include <linux/openat2.h>
+#include <linux/stat.h>
+#include <linux/unistd.h>
+
+// Including trailing NUL bytes.
+static const int MNT_ROOT_MAX_LEN = 43;
+static const int SOURCE_MAX_LEN = 28;
+
+static void set_mount_namespace(const char vm_id[static 1])
+{
+	char ns_path[28];
+	int r = snprintf(ns_path, sizeof ns_path,
+	                 "/run/vm/by-id/%s/ns/mnt", vm_id);
+
+	if (r == -1)
+		err(EXIT_FAILURE, "snprintf");
+	if ((size_t)r >= sizeof ns_path)
+		errx(EXIT_FAILURE, "VM ID unexpectedly long");
+
+	if ((r = open(ns_path, O_RDONLY | O_CLOEXEC)) == -1)
+		err(EXIT_FAILURE, "open");
+	if (setns(r, CLONE_NEWNS) == -1)
+		err(EXIT_FAILURE, "setns");
+	close(r);
+}
+
+static void do_statx(const char path[static 1],
+                     mode_t mode[static 1], uint64_t mnt_id[static 1])
+{
+	struct statx stx;
+
+	if (syscall(__NR_statx, AT_FDCWD, path, AT_SYMLINK_NOFOLLOW,
+	            STATX_MODE | STATX_MNT_ID_UNIQUE, &stx) == -1)
+		err(EXIT_FAILURE, "statx");
+
+	if (!(stx.stx_attributes & STATX_ATTR_MOUNT_ROOT)) {
+		if (stx.stx_attributes_mask & STATX_ATTR_MOUNT_ROOT)
+			errx(EXIT_FAILURE,
+			     "VM disk-backed directory not mounted");
+
+		errx(EXIT_FAILURE, "statx didn't return STATX_ATTR_MOUNT_ROOT");
+	}
+
+	if (!(stx.stx_mask & STATX_MNT_ID_UNIQUE))
+		errx(EXIT_FAILURE, "statx didn't return STATX_MNT_ID_UNIQUE");
+	if (!(stx.stx_mask & STATX_MODE))
+		errx(EXIT_FAILURE, "statx didn't return STATX_MODE");
+
+	*mode = stx.stx_mode;
+	*mnt_id = stx.stx_mnt_id;
+}
+
+static int do_mount(const char source[static 1])
+{
+	int mnt, fs = syscall(__NR_fsopen, "btrfs", FSOPEN_CLOEXEC);
+	if (fs == -1)
+		err(EXIT_FAILURE, "fsopen");
+	if (syscall(__NR_fsconfig, fs, FSCONFIG_SET_STRING,
+	            "source", source, 0) == -1)
+		err(EXIT_FAILURE, "FSCONFIG_SET_STRING source");
+	if (syscall(__NR_fsconfig, fs, FSCONFIG_SET_FLAG,
+	            "rw", nullptr, 0) == -1)
+		err(EXIT_FAILURE, "FSCONFIG_SET_FLAG rw");
+	if (syscall(__NR_fsconfig, fs, FSCONFIG_CMD_CREATE,
+	            nullptr, nullptr, 0) == -1)
+		err(EXIT_FAILURE, "FSCONFIG_CMD_CREATE");
+	if ((mnt = syscall(__NR_fsmount, fs, FSMOUNT_CLOEXEC,
+	                   MOUNT_ATTR_NOSUID | MOUNT_ATTR_NOSYMFOLLOW |
+	                   MOUNT_ATTR_NOEXEC | MOUNT_ATTR_NODEV)) == -1)
+		err(EXIT_FAILURE, "fsmount");
+	close(fs);
+	return mnt;
+}
+
+static void do_statmount(uint64_t mnt_id,
+                         char mnt_root[static MNT_ROOT_MAX_LEN],
+                         char source[static SOURCE_MAX_LEN])
+{
+	int r;
+	char sm_buf[sizeof(struct statmount) +
+	            MNT_ROOT_MAX_LEN + SOURCE_MAX_LEN];
+	struct statmount *sm = (struct statmount *)sm_buf;
+	struct mnt_id_req req = {
+		.size = sizeof req,
+		.mnt_id = mnt_id,
+		.param = STATMOUNT_MNT_ROOT | STATMOUNT_SB_SOURCE,
+	};
+
+	if (syscall(__NR_statmount, &req, sm, sizeof sm_buf, 0) == -1)
+		err(EXIT_FAILURE, "statmount");
+
+	r = snprintf(mnt_root, MNT_ROOT_MAX_LEN, "%s", sm->str + sm->mnt_root);
+	if (r == -1)
+		err(EXIT_FAILURE, "snprintf");
+	if (r >= MNT_ROOT_MAX_LEN)
+		errx(EXIT_FAILURE, "unexpectedly long mnt_root");
+
+	r = snprintf(source, SOURCE_MAX_LEN, "%s", sm->str + sm->sb_source);
+	if (r == -1)
+		err(EXIT_FAILURE, "snprintf");
+	if (r >= SOURCE_MAX_LEN)
+		errx(EXIT_FAILURE, "unexpectedly long sb_source");
+}
+
+static void do_rename(int mnt, const char dir_name[static 1],
+                      const char old_name[static 1],
+                      const char new_name[static 1], mode_t mode)
+{
+	struct open_how how = {
+		.flags = O_PATH | O_CLOEXEC | O_DIRECTORY | O_NOFOLLOW,
+		.resolve = RESOLVE_NO_MAGICLINKS | RESOLVE_IN_ROOT |
+		           RESOLVE_NO_SYMLINKS | RESOLVE_NO_XDEV,
+	};
+	int dir = syscall(__NR_openat2, mnt, dir_name, &how, sizeof how);
+	if (dir == -1)
+		err(EXIT_FAILURE, "openat2");
+
+	if (syscall(__NR_mkdirat, dir, new_name, mode) == -1)
+		err(EXIT_FAILURE, "mkdirat");
+	if (syscall(__NR_renameat2, dir, old_name, dir, new_name,
+	            RENAME_EXCHANGE) == -1)
+		err(EXIT_FAILURE, "renameat2");
+}
+
+int main(int argc, char *argv[])
+{
+	int mnt;
+	mode_t mode;
+	uint64_t mnt_id;
+	char *disk_path, *dir_name, *old_name, *new_name,
+	     mnt_root[MNT_ROOT_MAX_LEN], source[SOURCE_MAX_LEN];
+
+	if (argc != 3) {
+		fprintf(stderr, "Usage: vm-set-persist ID INSTANCE\n");
+		exit(EXIT_FAILURE);
+	}
+
+	if (strchr(argv[1], '/'))
+		errx(EXIT_FAILURE, "invalid VM ID");
+	if (strchr(argv[2], '/'))
+		errx(EXIT_FAILURE, "invalid persistent directory name");
+
+	if (asprintf(&disk_path, "/run/fs/%s/disk", argv[1]) == -1)
+		err(EXIT_FAILURE, "asprintf");
+	if (asprintf(&new_name, "persist.%s", argv[2]) == -1)
+		err(EXIT_FAILURE, "asprintf");
+
+	set_mount_namespace(argv[1]);
+
+	do_statx(disk_path, &mode, &mnt_id);
+	do_statmount(mnt_id, mnt_root, source);
+
+	if (!(dir_name = strdup(mnt_root)))
+		err(EXIT_FAILURE, "strdup");
+	dir_name = dirname(dir_name);
+	old_name = basename(mnt_root);
+
+	mnt = do_mount(source);
+
+	do_rename(mnt, dir_name, old_name, new_name, mode);
+}
-- 
2.51.0

[PATCH v2 6/7] host/rootfs: run transient VMs with persistence

[Alyssa Ross]

This allows run-appimage and run-flatpak to be given the name of a
previously saved persistent directory.  Further writes by the
application to that directory will also be persisted.

Signed-off-by: Alyssa Ross <hi@alyssa.is>
---
v2: new this round

 host/rootfs/image/usr/bin/run-appimage | 16 +++++++++++++---
 host/rootfs/image/usr/bin/run-flatpak  | 15 +++++++++++++--
 2 files changed, 26 insertions(+), 5 deletions(-)

diff --git a/host/rootfs/image/usr/bin/run-appimage b/host/rootfs/image/usr/bin/run-appimage
index de851c52..24621065 100755
--- a/host/rootfs/image/usr/bin/run-appimage
+++ b/host/rootfs/image/usr/bin/run-appimage
@@ -26,13 +26,23 @@ if {
 }
 
 backtick diskdir {
-  s6-setuidgid fs
-
-  backtick -E mountpoint {
+  backtick mountpoint {
     importas -Siu 1
     findmnt -no TARGET -T $1
   }
 
+  if -tn {
+    redirfd -w 2 /dev/null
+    multisubstitute {
+      importas -Siu mountpoint
+      importas -Siu 2
+    }
+    printf "%s/Spectrum/data/spectrum/storage/persist.%s\n" $mountpoint $2
+  }
+
+  s6-setuidgid fs
+
+  importas -Siu mountpoint
   if { mkdir -p -- ${mountpoint}/Spectrum/data/spectrum/storage }
   mktemp -d -- ${mountpoint}/Spectrum/data/spectrum/storage/tmp.XXXXXX
 }
diff --git a/host/rootfs/image/usr/bin/run-flatpak b/host/rootfs/image/usr/bin/run-flatpak
index b47204c9..4123d329 100755
--- a/host/rootfs/image/usr/bin/run-flatpak
+++ b/host/rootfs/image/usr/bin/run-flatpak
@@ -26,6 +26,15 @@ if {
 }
 
 backtick diskdir {
+  if -tn {
+    redirfd -w 2 /dev/null
+    multisubstitute {
+      importas -Siu 1
+      importas -Siu 4
+    }
+    printf "%s/Spectrum/data/spectrum/storage/persist.%s\n" $1 $4
+  }
+
   s6-setuidgid fs
 
   importas -Siu 1
@@ -41,7 +50,9 @@ if {
   multisubstitute {
     importas -Siu id
     importas -Siu diskdir
-    elgetpositionals
+    importas -Siu 1
+    importas -Siu 2
+    importas -Siu 3
   }
 
   nsenter --preserve-credentials -S0
@@ -50,7 +61,7 @@ if {
 
   cd /run/fs/${id}/config
   if { redirfd -w 1 type echo flatpak }
-  if { mount-flatpak $@ }
+  if { mount-flatpak $1 $2 $3 }
 
   mount --bind -- $diskdir /run/fs/${id}/disk
 }
-- 
2.51.0

[PATCH v2 7/7] Documentation: document persistence

[Alyssa Ross]

Signed-off-by: Alyssa Ross <hi@alyssa.is>
---
v2: new this round

 .../using-spectrum/vm-file-access.adoc        | 33 +++++++++++++++++--
 1 file changed, 30 insertions(+), 3 deletions(-)

diff --git a/Documentation/using-spectrum/vm-file-access.adoc b/Documentation/using-spectrum/vm-file-access.adoc
index 1546d57d..6189233f 100644
--- a/Documentation/using-spectrum/vm-file-access.adoc
+++ b/Documentation/using-spectrum/vm-file-access.adoc
@@ -2,11 +2,38 @@
 :page-parent: Using Spectrum
 :page-nav_order: 1
 
-// SPDX-FileCopyrightText: 2024 Alyssa Ross <hi@alyssa.is>
+// SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
 // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0
 
-Spectrum VMs start without any access to user data, but the user can
-grant VMs access to files while the VM is running.
+Spectrum VMs start without any access to user data, and with all
+application state being discarded when the VM exits, but these
+restrictions can be softened at runtime as required.
+
+== Persistent application data
+
+To make an application VM persistent, run, for example,
+`vm-set-persist gGKghi configured`, where "gGKghi" is the VM's ID (can
+be found using xref:running-vms.adoc#basic-vm-commands[`lsvm`]) and
+"configured" is the name to be given to this persistent application
+instance.  The VM's home directory will now be saved under the given
+name.  For now, names are scoped to only to a user data partition, not
+to an application, so you cannot create instances of different
+applications with the same name.
+
+Then, to start a VM with that persistent data in future, give the
+instance name to `run-appimage` or `run-flatpak` as an extra argument.
+This run of the application will also persist further changes to its
+home directory under the same name.
+
+Persistent application data is stored as directories prefixed with
+"persist." under Spectrum/data/spectrum/storage on the user data
+partition.  They can be manually renamed, and, as long as they are not
+being used by a currently running VM, removed.
+
+For manually configured VMs, persistence can optionally be enabled
+when the VM is imported, by providing the storage location as an extra
+argument to `vm-import`.  The name of each imported VM will be used as
+its instance name.
 
 == File chooser portal
 
-- 
2.51.0

Re: [PATCH v2 5/7] tools/vm-set-persist.c: init

[Demi Marie Obenour]

[-- Attachment #1.1.1: Type: text/plain, Size: 8813 bytes --]

On 12/13/25 20:42, Alyssa Ross wrote:
> This allows the disk-backed directory of a running VM to be made
> persistent, with a user-provided name.  This is done by renaming the
> directory to have a "persist." prefix rather than the "tmp." one the
> cleaner will look for.  Since the VM's virtiofsd accesses the
> directory via a bind mount, this rename will be unnoticeable to the
> guest.
> 
> musl has quite a bit of catching up to do with the APIs used here,
> which requires the use of a lot of raw syscalls.  This even applies to
> some syscalls musl has wrappers for, like mkdirat(2), because musl's
> mkdirat() comes from <sys/stat.h>, which defines a struct statx that's
> missing the stx_mnt_id member we need.  There doesn't even seem to be
> a SYS_statmount, so we use __NR_ constants throughout for consistency.
> 
> Signed-off-by: Alyssa Ross <hi@alyssa.is>
> ---
> v2: new this round
> 
>  tools/default.nix      |   1 +
>  tools/meson.build      |   4 +
>  tools/vm-set-persist.c | 179 +++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 184 insertions(+)
>  create mode 100644 tools/vm-set-persist.c
> 
> diff --git a/tools/default.nix b/tools/default.nix
> index 56f41cd9..f094594f 100644
> --- a/tools/default.nix
> +++ b/tools/default.nix
> @@ -78,6 +78,7 @@ stdenv.mkDerivation (finalAttrs: {
>        ./start-vmm
>        ./subprojects
>        ./updates-dir-check.c
> +      ./vm-set-persist.c
>      ] ++ lib.optionals driverSupport [
>        ./xdp-forwarder
>      ]));
> diff --git a/tools/meson.build b/tools/meson.build
> index 666483b3..06aa24d7 100644
> --- a/tools/meson.build
> +++ b/tools/meson.build
> @@ -37,6 +37,10 @@ if get_option('host')
>    executable('updates-dir-check', 'updates-dir-check.c',
>      c_args : '-D_GNU_SOURCE',
>      install: true)
> +
> +  executable('vm-set-persist', 'vm-set-persist.c',
> +    c_args : '-D_GNU_SOURCE',
> +    install: true)
>  endif
>  
>  if get_option('build')
> diff --git a/tools/vm-set-persist.c b/tools/vm-set-persist.c
> new file mode 100644
> index 00000000..ac759504
> --- /dev/null
> +++ b/tools/vm-set-persist.c
> @@ -0,0 +1,179 @@
> +// SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
> +// SPDX-License-Identifier: EUPL-1.2+
> +
> +#include <err.h>
> +#include <fcntl.h>
> +#include <libgen.h>
> +#include <unistd.h>
> +#include <sched.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <stdint.h>
> +#include <string.h>
> +
> +// No <sys/stat.h> until musl declares stx_mnt_id.
> +#include <sys/syscall.h>
> +
> +#include <linux/fs.h>
> +#include <linux/mount.h>
> +#include <linux/openat2.h>
> +#include <linux/stat.h>
> +#include <linux/unistd.h>
> +
> +// Including trailing NUL bytes.
> +static const int MNT_ROOT_MAX_LEN = 43;
> +static const int SOURCE_MAX_LEN = 28;
> +
> +static void set_mount_namespace(const char vm_id[static 1])
> +{
> +	char ns_path[28];
> +	int r = snprintf(ns_path, sizeof ns_path,
> +	                 "/run/vm/by-id/%s/ns/mnt", vm_id);
> +
> +	if (r == -1)
> +		err(EXIT_FAILURE, "snprintf");
> +	if ((size_t)r >= sizeof ns_path)
> +		errx(EXIT_FAILURE, "VM ID unexpectedly long");
> +
> +	if ((r = open(ns_path, O_RDONLY | O_CLOEXEC)) == -1)
> +		err(EXIT_FAILURE, "open");
> +	if (setns(r, CLONE_NEWNS) == -1)
> +		err(EXIT_FAILURE, "setns");
> +	close(r);
> +}
> +
> +static void do_statx(const char path[static 1],
> +                     mode_t mode[static 1], uint64_t mnt_id[static 1])
> +{
> +	struct statx stx;
> +
> +	if (syscall(__NR_statx, AT_FDCWD, path, AT_SYMLINK_NOFOLLOW,
> +	            STATX_MODE | STATX_MNT_ID_UNIQUE, &stx) == -1)
> +		err(EXIT_FAILURE, "statx");

Here (and below), I recommend using wrapper functions around syscalls
instead of raw syscalls.  This makes it much easier to check the
types used against the manpage.

This is also missing casts to long.

> +	if (!(stx.stx_attributes & STATX_ATTR_MOUNT_ROOT)) {
> +		if (stx.stx_attributes_mask & STATX_ATTR_MOUNT_ROOT)
> +			errx(EXIT_FAILURE,
> +			     "VM disk-backed directory not mounted");
> +
> +		errx(EXIT_FAILURE, "statx didn't return STATX_ATTR_MOUNT_ROOT");
> +	}
> +
> +	if (!(stx.stx_mask & STATX_MNT_ID_UNIQUE))
> +		errx(EXIT_FAILURE, "statx didn't return STATX_MNT_ID_UNIQUE");
> +	if (!(stx.stx_mask & STATX_MODE))
> +		errx(EXIT_FAILURE, "statx didn't return STATX_MODE");
> +
> +	*mode = stx.stx_mode;
> +	*mnt_id = stx.stx_mnt_id;
> +}
> +
> +static int do_mount(const char source[static 1])
> +{
> +	int mnt, fs = syscall(__NR_fsopen, "btrfs", FSOPEN_CLOEXEC);
> +	if (fs == -1)
> +		err(EXIT_FAILURE, "fsopen");
> +	if (syscall(__NR_fsconfig, fs, FSCONFIG_SET_STRING,
> +	            "source", source, 0) == -1)

This might be too paranoid, but if possible I would use openat2()
with RESOLVE_NO_SYMLINKS to get a file descriptor to the source,
and then use /proc/thread-self/fd/FD_NUMBER here.

> +		err(EXIT_FAILURE, "FSCONFIG_SET_STRING source");
> +	if (syscall(__NR_fsconfig, fs, FSCONFIG_SET_FLAG,
> +	            "rw", nullptr, 0) == -1)
> +		err(EXIT_FAILURE, "FSCONFIG_SET_FLAG rw");
> +	if (syscall(__NR_fsconfig, fs, FSCONFIG_CMD_CREATE,
> +	            nullptr, nullptr, 0) == -1)
> +		err(EXIT_FAILURE, "FSCONFIG_CMD_CREATE");
> +	if ((mnt = syscall(__NR_fsmount, fs, FSMOUNT_CLOEXEC,
> +	                   MOUNT_ATTR_NOSUID | MOUNT_ATTR_NOSYMFOLLOW |
> +	                   MOUNT_ATTR_NOEXEC | MOUNT_ATTR_NODEV)) == -1)
> +		err(EXIT_FAILURE, "fsmount");
> +	close(fs);
> +	return mnt;
> +}
> +
> +static void do_statmount(uint64_t mnt_id,
> +                         char mnt_root[static MNT_ROOT_MAX_LEN],
> +                         char source[static SOURCE_MAX_LEN])
> +{
> +	int r;
> +	char sm_buf[sizeof(struct statmount) +
> +	            MNT_ROOT_MAX_LEN + SOURCE_MAX_LEN];
> +	struct statmount *sm = (struct statmount *)sm_buf;
> +	struct mnt_id_req req = {
> +		.size = sizeof req,
> +		.mnt_id = mnt_id,
> +		.param = STATMOUNT_MNT_ROOT | STATMOUNT_SB_SOURCE,
> +	};
> +
> +	if (syscall(__NR_statmount, &req, sm, sizeof sm_buf, 0) == -1)
> +		err(EXIT_FAILURE, "statmount");
> +
> +	r = snprintf(mnt_root, MNT_ROOT_MAX_LEN, "%s", sm->str + sm->mnt_root);
> +	if (r == -1)
> +		err(EXIT_FAILURE, "snprintf");
> +	if (r >= MNT_ROOT_MAX_LEN)
> +		errx(EXIT_FAILURE, "unexpectedly long mnt_root");
> +
> +	r = snprintf(source, SOURCE_MAX_LEN, "%s", sm->str + sm->sb_source);
> +	if (r == -1)
> +		err(EXIT_FAILURE, "snprintf");
> +	if (r >= SOURCE_MAX_LEN)
> +		errx(EXIT_FAILURE, "unexpectedly long sb_source");
> +}

Here and elsewhere, I suggest a wrapper around snprintf().

> +static void do_rename(int mnt, const char dir_name[static 1],
> +                      const char old_name[static 1],
> +                      const char new_name[static 1], mode_t mode)
> +{
> +	struct open_how how = {
> +		.flags = O_PATH | O_CLOEXEC | O_DIRECTORY | O_NOFOLLOW,
> +		.resolve = RESOLVE_NO_MAGICLINKS | RESOLVE_IN_ROOT |
> +		           RESOLVE_NO_SYMLINKS | RESOLVE_NO_XDEV,
> +	};
> +	int dir = syscall(__NR_openat2, mnt, dir_name, &how, sizeof how);
> +	if (dir == -1)
> +		err(EXIT_FAILURE, "openat2");
> +
> +	if (syscall(__NR_mkdirat, dir, new_name, mode) == -1)
> +		err(EXIT_FAILURE, "mkdirat");
> +	if (syscall(__NR_renameat2, dir, old_name, dir, new_name,
> +	            RENAME_EXCHANGE) == -1)
> +		err(EXIT_FAILURE, "renameat2");
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +	int mnt;
> +	mode_t mode;
> +	uint64_t mnt_id;
> +	char *disk_path, *dir_name, *old_name, *new_name,
> +	     mnt_root[MNT_ROOT_MAX_LEN], source[SOURCE_MAX_LEN];
> +
> +	if (argc != 3) {
> +		fprintf(stderr, "Usage: vm-set-persist ID INSTANCE\n");
> +		exit(EXIT_FAILURE);
> +	}
> +
> +	if (strchr(argv[1], '/'))
> +		errx(EXIT_FAILURE, "invalid VM ID");

I'd check for ".", "..", empty string, and NAME_MAX (255) as well.

> +	if (strchr(argv[2], '/'))
> +		errx(EXIT_FAILURE, "invalid persistent directory name");> +	if (asprintf(&disk_path, "/run/fs/%s/disk", argv[1]) == -1)
> +		err(EXIT_FAILURE, "asprintf");
> +	if (asprintf(&new_name, "persist.%s", argv[2]) == -1)
> +		err(EXIT_FAILURE, "asprintf");

I'd check that this doesn't go beyond NAME_MAX.

> +	set_mount_namespace(argv[1]);
> +
> +	do_statx(disk_path, &mode, &mnt_id);
> +	do_statmount(mnt_id, mnt_root, source);
> +
> +	if (!(dir_name = strdup(mnt_root)))
> +		err(EXIT_FAILURE, "strdup");
> +	dir_name = dirname(dir_name);
> +	old_name = basename(mnt_root);
> +
> +	mnt = do_mount(source);
> +
> +	do_rename(mnt, dir_name, old_name, new_name, mode);
> +}
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 5/7] tools/vm-set-persist.c: init

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 7809 bytes --]

Demi Marie Obenour <demiobenour@gmail.com> writes:

> On 12/13/25 20:42, Alyssa Ross wrote:
>> diff --git a/tools/vm-set-persist.c b/tools/vm-set-persist.c
>> new file mode 100644
>> index 00000000..ac759504
>> --- /dev/null
>> +++ b/tools/vm-set-persist.c
>> @@ -0,0 +1,179 @@
>> +// SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
>> +// SPDX-License-Identifier: EUPL-1.2+
>> +
>> +#include <err.h>
>> +#include <fcntl.h>
>> +#include <libgen.h>
>> +#include <unistd.h>
>> +#include <sched.h>
>> +#include <stdio.h>
>> +#include <stdlib.h>
>> +#include <stdint.h>
>> +#include <string.h>
>> +
>> +// No <sys/stat.h> until musl declares stx_mnt_id.
>> +#include <sys/syscall.h>
>> +
>> +#include <linux/fs.h>
>> +#include <linux/mount.h>
>> +#include <linux/openat2.h>
>> +#include <linux/stat.h>
>> +#include <linux/unistd.h>
>> +
>> +// Including trailing NUL bytes.
>> +static const int MNT_ROOT_MAX_LEN = 43;
>> +static const int SOURCE_MAX_LEN = 28;
>> +
>> +static void set_mount_namespace(const char vm_id[static 1])
>> +{
>> +	char ns_path[28];
>> +	int r = snprintf(ns_path, sizeof ns_path,
>> +	                 "/run/vm/by-id/%s/ns/mnt", vm_id);
>> +
>> +	if (r == -1)
>> +		err(EXIT_FAILURE, "snprintf");
>> +	if ((size_t)r >= sizeof ns_path)
>> +		errx(EXIT_FAILURE, "VM ID unexpectedly long");
>> +
>> +	if ((r = open(ns_path, O_RDONLY | O_CLOEXEC)) == -1)
>> +		err(EXIT_FAILURE, "open");
>> +	if (setns(r, CLONE_NEWNS) == -1)
>> +		err(EXIT_FAILURE, "setns");
>> +	close(r);
>> +}
>> +
>> +static void do_statx(const char path[static 1],
>> +                     mode_t mode[static 1], uint64_t mnt_id[static 1])
>> +{
>> +	struct statx stx;
>> +
>> +	if (syscall(__NR_statx, AT_FDCWD, path, AT_SYMLINK_NOFOLLOW,
>> +	            STATX_MODE | STATX_MNT_ID_UNIQUE, &stx) == -1)
>> +		err(EXIT_FAILURE, "statx");
>
> Here (and below), I recommend using wrapper functions around syscalls
> instead of raw syscalls.  This makes it much easier to check the
> types used against the manpage.

I doubt this code is going to change much before the next musl release
(which will bring wrappers for most of these) anyway.

> This is also missing casts to long.

If casts to long are necessary every time you call syscall() in C,
fixing that is going to need to start with musl, which doesn't seem to
think that's necessary.

>> +	if (!(stx.stx_attributes & STATX_ATTR_MOUNT_ROOT)) {
>> +		if (stx.stx_attributes_mask & STATX_ATTR_MOUNT_ROOT)
>> +			errx(EXIT_FAILURE,
>> +			     "VM disk-backed directory not mounted");
>> +
>> +		errx(EXIT_FAILURE, "statx didn't return STATX_ATTR_MOUNT_ROOT");
>> +	}
>> +
>> +	if (!(stx.stx_mask & STATX_MNT_ID_UNIQUE))
>> +		errx(EXIT_FAILURE, "statx didn't return STATX_MNT_ID_UNIQUE");
>> +	if (!(stx.stx_mask & STATX_MODE))
>> +		errx(EXIT_FAILURE, "statx didn't return STATX_MODE");
>> +
>> +	*mode = stx.stx_mode;
>> +	*mnt_id = stx.stx_mnt_id;
>> +}
>> +
>> +static int do_mount(const char source[static 1])
>> +{
>> +	int mnt, fs = syscall(__NR_fsopen, "btrfs", FSOPEN_CLOEXEC);
>> +	if (fs == -1)
>> +		err(EXIT_FAILURE, "fsopen");
>> +	if (syscall(__NR_fsconfig, fs, FSCONFIG_SET_STRING,
>> +	            "source", source, 0) == -1)
>
> This might be too paranoid, but if possible I would use openat2()
> with RESOLVE_NO_SYMLINKS to get a file descriptor to the source,
> and then use /proc/thread-self/fd/FD_NUMBER here.

If the kernel gives us a symlink for some reason we probably want to
follow it.  Swapping out the path would require root.

>> +		err(EXIT_FAILURE, "FSCONFIG_SET_STRING source");
>> +	if (syscall(__NR_fsconfig, fs, FSCONFIG_SET_FLAG,
>> +	            "rw", nullptr, 0) == -1)
>> +		err(EXIT_FAILURE, "FSCONFIG_SET_FLAG rw");
>> +	if (syscall(__NR_fsconfig, fs, FSCONFIG_CMD_CREATE,
>> +	            nullptr, nullptr, 0) == -1)
>> +		err(EXIT_FAILURE, "FSCONFIG_CMD_CREATE");
>> +	if ((mnt = syscall(__NR_fsmount, fs, FSMOUNT_CLOEXEC,
>> +	                   MOUNT_ATTR_NOSUID | MOUNT_ATTR_NOSYMFOLLOW |
>> +	                   MOUNT_ATTR_NOEXEC | MOUNT_ATTR_NODEV)) == -1)
>> +		err(EXIT_FAILURE, "fsmount");
>> +	close(fs);
>> +	return mnt;
>> +}
>> +
>> +static void do_statmount(uint64_t mnt_id,
>> +                         char mnt_root[static MNT_ROOT_MAX_LEN],
>> +                         char source[static SOURCE_MAX_LEN])
>> +{
>> +	int r;
>> +	char sm_buf[sizeof(struct statmount) +
>> +	            MNT_ROOT_MAX_LEN + SOURCE_MAX_LEN];
>> +	struct statmount *sm = (struct statmount *)sm_buf;
>> +	struct mnt_id_req req = {
>> +		.size = sizeof req,
>> +		.mnt_id = mnt_id,
>> +		.param = STATMOUNT_MNT_ROOT | STATMOUNT_SB_SOURCE,
>> +	};
>> +
>> +	if (syscall(__NR_statmount, &req, sm, sizeof sm_buf, 0) == -1)
>> +		err(EXIT_FAILURE, "statmount");
>> +
>> +	r = snprintf(mnt_root, MNT_ROOT_MAX_LEN, "%s", sm->str + sm->mnt_root);
>> +	if (r == -1)
>> +		err(EXIT_FAILURE, "snprintf");
>> +	if (r >= MNT_ROOT_MAX_LEN)
>> +		errx(EXIT_FAILURE, "unexpectedly long mnt_root");
>> +
>> +	r = snprintf(source, SOURCE_MAX_LEN, "%s", sm->str + sm->sb_source);
>> +	if (r == -1)
>> +		err(EXIT_FAILURE, "snprintf");
>> +	if (r >= SOURCE_MAX_LEN)
>> +		errx(EXIT_FAILURE, "unexpectedly long sb_source");
>> +}
>
> Here and elsewhere, I suggest a wrapper around snprintf().

There's a readability cost to unfamiliar wrappers that I don't think is
justified here.  Maybe if we were doing this a lot more.

>> +static void do_rename(int mnt, const char dir_name[static 1],
>> +                      const char old_name[static 1],
>> +                      const char new_name[static 1], mode_t mode)
>> +{
>> +	struct open_how how = {
>> +		.flags = O_PATH | O_CLOEXEC | O_DIRECTORY | O_NOFOLLOW,
>> +		.resolve = RESOLVE_NO_MAGICLINKS | RESOLVE_IN_ROOT |
>> +		           RESOLVE_NO_SYMLINKS | RESOLVE_NO_XDEV,
>> +	};
>> +	int dir = syscall(__NR_openat2, mnt, dir_name, &how, sizeof how);
>> +	if (dir == -1)
>> +		err(EXIT_FAILURE, "openat2");
>> +
>> +	if (syscall(__NR_mkdirat, dir, new_name, mode) == -1)
>> +		err(EXIT_FAILURE, "mkdirat");
>> +	if (syscall(__NR_renameat2, dir, old_name, dir, new_name,
>> +	            RENAME_EXCHANGE) == -1)
>> +		err(EXIT_FAILURE, "renameat2");
>> +}
>> +
>> +int main(int argc, char *argv[])
>> +{
>> +	int mnt;
>> +	mode_t mode;
>> +	uint64_t mnt_id;
>> +	char *disk_path, *dir_name, *old_name, *new_name,
>> +	     mnt_root[MNT_ROOT_MAX_LEN], source[SOURCE_MAX_LEN];
>> +
>> +	if (argc != 3) {
>> +		fprintf(stderr, "Usage: vm-set-persist ID INSTANCE\n");
>> +		exit(EXIT_FAILURE);
>> +	}
>> +
>> +	if (strchr(argv[1], '/'))
>> +		errx(EXIT_FAILURE, "invalid VM ID");
>
> I'd check for ".", "..", empty string, and NAME_MAX (255) as well.

Any of those will just mean the program fails.  Same goes for /, of
course, but that's more likely to happen by accident I think.

>> +	if (strchr(argv[2], '/'))
>> +		errx(EXIT_FAILURE, "invalid persistent directory name");
>> +	if (asprintf(&disk_path, "/run/fs/%s/disk", argv[1]) == -1)
>> +		err(EXIT_FAILURE, "asprintf");
>> +	if (asprintf(&new_name, "persist.%s", argv[2]) == -1)
>> +		err(EXIT_FAILURE, "asprintf");
>
> I'd check that this doesn't go beyond NAME_MAX.
>
>> +	set_mount_namespace(argv[1]);
>> +
>> +	do_statx(disk_path, &mode, &mnt_id);
>> +	do_statmount(mnt_id, mnt_root, source);
>> +
>> +	if (!(dir_name = strdup(mnt_root)))
>> +		err(EXIT_FAILURE, "strdup");
>> +	dir_name = dirname(dir_name);
>> +	old_name = basename(mnt_root);
>> +
>> +	mnt = do_mount(source);
>> +
>> +	do_rename(mnt, dir_name, old_name, new_name, mode);
>> +}
> -- 
> Sincerely,
> Demi Marie Obenour (she/her/hers)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: [PATCH v2 1/7] host/rootfs: make fs root directories shared

[Alyssa Ross]

This patch has been committed as b78339a64b5b591877507f3ca33452ca5bc5117f,
which can be viewed online at
https://spectrum-os.org/git/spectrum/commit/?id=b78339a64b5b591877507f3ca33452ca5bc5117f.

This is an automated message.  Send comments/questions/requests to:
Alyssa Ross <hi@alyssa.is>

Re: [PATCH v2 2/7] host/rootfs: give VMs a disk-backed directory

[Alyssa Ross]

This patch has been committed as 851ea0bee54173a624d864db9cac0bc4a98d8098,
which can be viewed online at
https://spectrum-os.org/git/spectrum/commit/?id=851ea0bee54173a624d864db9cac0bc4a98d8098.

This is an automated message.  Send comments/questions/requests to:
Alyssa Ross <hi@alyssa.is>

Re: [PATCH v2 4/7] host/rootfs: clean up obsolete tmp dirs on mount

[Alyssa Ross]

This patch has been committed as d15065400664eacc0937621524923d7b0cee108b,
which can be viewed online at
https://spectrum-os.org/git/spectrum/commit/?id=d15065400664eacc0937621524923d7b0cee108b.

This is an automated message.  Send comments/questions/requests to:
Alyssa Ross <hi@alyssa.is>

Re: [PATCH v2 3/7] host/rootfs: clean up obsolete tmp dirs on VM exit

[Alyssa Ross]

This patch has been committed as 0c01afbc9f2969d92f31d60f23644c045e82f46a,
which can be viewed online at
https://spectrum-os.org/git/spectrum/commit/?id=0c01afbc9f2969d92f31d60f23644c045e82f46a.

This is an automated message.  Send comments/questions/requests to:
Alyssa Ross <hi@alyssa.is>

Re: [PATCH v2 5/7] tools/vm-set-persist.c: init

[Alyssa Ross]

This patch has been committed as 0c9c3907210c59291c2b1407c0488735a677b331,
which can be viewed online at
https://spectrum-os.org/git/spectrum/commit/?id=0c9c3907210c59291c2b1407c0488735a677b331.

This is an automated message.  Send comments/questions/requests to:
Alyssa Ross <hi@alyssa.is>

Re: [PATCH v2 6/7] host/rootfs: run transient VMs with persistence

[Alyssa Ross]

This patch has been committed as 12ebaabe5c8a52cd5659841e79f6749bebcb5b66,
which can be viewed online at
https://spectrum-os.org/git/spectrum/commit/?id=12ebaabe5c8a52cd5659841e79f6749bebcb5b66.

This is an automated message.  Send comments/questions/requests to:
Alyssa Ross <hi@alyssa.is>

Re: [PATCH v2 7/7] Documentation: document persistence

[Alyssa Ross]

This patch has been committed as 1370198ae7830ca1e4783a9c94d30cbb709dfe9b,
which can be viewed online at
https://spectrum-os.org/git/spectrum/commit/?id=1370198ae7830ca1e4783a9c94d30cbb709dfe9b.

This is an automated message.  Send comments/questions/requests to:
Alyssa Ross <hi@alyssa.is>