From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id DC784C866; Sun, 14 Dec 2025 01:45:36 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id B7259C84B; Sun, 14 Dec 2025 01:45:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_MISSING,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=4.0.1 Received: from fhigh-a2-smtp.messagingengine.com (fhigh-a2-smtp.messagingengine.com [103.168.172.153]) by atuin.qyliss.net (Postfix) with ESMTPS id 7D139C7C3 for ; Sun, 14 Dec 2025 01:45:32 +0000 (UTC) Received: from phl-compute-03.internal (phl-compute-03.internal [10.202.2.43]) by mailfhigh.phl.internal (Postfix) with ESMTP id 43A55140015F; Sat, 13 Dec 2025 20:45:30 -0500 (EST) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-03.internal (MEProxy); Sat, 13 Dec 2025 20:45:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alyssa.is; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm3; t=1765676730; x=1765763130; bh=AAIaRlF1X43Wak+77xIKX4N0vu2AgQnCp79EY7y/bL4=; b= LjPN9AnFPwRbLG56OggjhLMebP246s0M5pKQd0m+PjynXVAjy2Yk20dqUapZakzz g1XgOwlbof2q6vemM0bBCOPQmXMP2gGvkODa6Pe+AYjU+alnOWMJVoN9La3T3gyA Rs5gBKl7+0JP7ayetWMRS22BJDWSdsGK7DfBSnuNGipe+Dan5eUiYrLdDXXyLakf KMnEq/kKkjzfQy/+jXcjBmND4RGqOUBU4fyDBbEq5GF+BafI1cA0TsbQHfaxJzfW aiB/ybpUyF6G8D+5oG3j9t2jFSQocgYB8l6z71G/7s2ix540y5RJrBQAC7Wm6Tyt l0AU6CDUzUNbYL3S22CaxQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1765676730; x= 1765763130; bh=AAIaRlF1X43Wak+77xIKX4N0vu2AgQnCp79EY7y/bL4=; b=c Pu8q9Ier7IvWVv+DTMIbAkmBFIvuGMj0AnauTaiFeopmMchFx4cB1aW3Q4km6whX 2t9RtFPYNS473IhR8rL52yFdWWnW3yDdLVPKuuF+MUC0ae3pIdf6R0Zg31R0wP89 yI4PuvQzABSbQtR0MWqkX7wvGmxQ7bakDtp/0GONzlIG8YDlgesdpcWTwLmeUmNP N9P36PM98XfR6D4zfnktqixjps5SBhh0wBcwqc0RO9AobUj0y6bAy2EqwZrR+kh2 F7+FCu/hxZwdVpl0IuN1Vz5yxDUzAnUV77Zw/8lfQfJHmEHwpzA7sve6AubimmAw FgeZqSNBa/rowL+FuVweA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefgedrtddtgdefvdeiiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufffkffojghfgggtgfesthekredtredtjeenucfhrhhomheptehlhihsshgr ucftohhsshcuoehhihesrghlhihsshgrrdhisheqnecuggftrfgrthhtvghrnhephffhue dtfeekhfeiiefhvdejieeiveetteffkeevffelfffggfelkeelffeggfetnecuffhomhgr ihhnpehsphgvtghtrhhumhdqohhsrdhorhhgpdguvghfihhnihhtihhonhhsrdhmvgguih grnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhephhhi segrlhihshhsrgdrihhspdhnsggprhgtphhtthhopedvpdhmohguvgepshhmthhpohhuth dprhgtphhtthhopeguvghmihhosggvnhhouhhrsehgmhgrihhlrdgtohhmpdhrtghpthht ohepuggvvhgvlhesshhpvggtthhruhhmqdhoshdrohhrgh X-ME-Proxy: Feedback-ID: i12284293:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 13 Dec 2025 20:45:29 -0500 (EST) Received: by fw12.qyliss.net (Postfix, from userid 1000) id EA33D7BF71E9; Sun, 14 Dec 2025 02:45:23 +0100 (CET) From: Alyssa Ross To: devel@spectrum-os.org Subject: [PATCH v2 2/7] host/rootfs: give VMs a disk-backed directory Date: Sun, 14 Dec 2025 02:42:26 +0100 Message-ID: <20251214014229.775825-4-hi@alyssa.is> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20251214014229.775825-2-hi@alyssa.is> References: <20251214014229.775825-2-hi@alyssa.is> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Message-ID-Hash: BBTWFWJHFUQQMPNHKH5KBFSOPIW2MJUF X-Message-ID-Hash: BBTWFWJHFUQQMPNHKH5KBFSOPIW2MJUF X-MailFrom: hi@alyssa.is X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Demi Marie Obenour X-Mailman-Version: 3.3.9 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: This is the first step towards persistent VM data. For now, these directories are never reused, and also don't get cleaned up. Both are things to work on in future. Even without persistence, it's good to not have to store everything a VM might write into its home directory in RAM. For AppImage and Flatpak VMs, disk-backed directories are stored on the partition containing the AppImage or Flatpak, in the new Spectrum/ hierarchy. This will enable locating them for later reuse. System VMs, on the other hand, don't have a natural partition to store data on — there may not even be a writable partition at the time they're launched. Since we don't expect persistence from system VMs, we just use a tmpfs for their "disk-backed" directories, which is reset on every boot. Doing it this way avoids the need for multiple routes in img/app — it can always assume it gets a "disk" directory from the host. For manually imported VMs, it's up to the user where these directories should be, or whether they should have the same behavior as system VMs. Signed-off-by: Alyssa Ross --- v2: reset "disk-backed" directories of system VMs between boots v1: https://spectrum-os.org/lists/archives/spectrum-devel/20251213161637.510752-2-hi@alyssa.is/ .../using-spectrum/creating-custom-vms.adoc | 10 ++++-- .../image/usr/bin/create-vm-dependencies | 1 + host/rootfs/image/usr/bin/run-appimage | 22 ++++++++++--- host/rootfs/image/usr/bin/run-flatpak | 15 ++++++++- host/rootfs/image/usr/bin/run-vmm | 32 ++++++++++++++++++- host/rootfs/image/usr/bin/vm-import | 17 +++++++++- img/app/Makefile | 2 +- img/app/image/etc/fstab | 1 - img/app/image/etc/s6-rc/app/run | 7 ++-- img/app/scripts/start-virtiofsd.elb | 2 +- release/checks/integration/lib.c | 3 +- release/checks/integration/networking.c | 2 +- release/checks/integration/portal.c | 2 +- 13 files changed, 97 insertions(+), 19 deletions(-) diff --git a/Documentation/using-spectrum/creating-custom-vms.adoc b/Documentation/using-spectrum/creating-custom-vms.adoc index 229c0140..68213c89 100644 --- a/Documentation/using-spectrum/creating-custom-vms.adoc +++ b/Documentation/using-spectrum/creating-custom-vms.adoc @@ -13,9 +13,13 @@ configurations are directories under a dedicated parent directory, and the name of each configuration directory determines the name of the VM. After mounting the persistent storage partition, the configured VMs can be made available by running `vm-import user -/media/4e43cdc2-82b2-4d94-8a90-b6c6189312d2/vms`, replacing -/media/4e43cdc2-82b2-4d94-8a90-b6c6189312d2/vms with the directory -containing the VM definitions. +/media/4e43cdc2-82b2-4d94-8a90-b6c6189312d2/vms +/media/4e43cdc2-82b2-4d94-8a90-b6c6189312d2/Spectrum/data/spectrum/storage`, +replacing /media/4e43cdc2-82b2-4d94-8a90-b6c6189312d2/vms with the +directory containing the VM definitions, and +/media/4e43cdc2-82b2-4d94-8a90-b6c6189312d2/Spectrum/data/spectrum/storage +with the directory where disk-backed directories for the VMs should be +created. The directory can contain the following files: diff --git a/host/rootfs/image/usr/bin/create-vm-dependencies b/host/rootfs/image/usr/bin/create-vm-dependencies index 6bf12d03..b7545425 100755 --- a/host/rootfs/image/usr/bin/create-vm-dependencies +++ b/host/rootfs/image/usr/bin/create-vm-dependencies @@ -6,6 +6,7 @@ if { mkdir -p /run/doc/${1}/doc /run/fs/${1}/config + /run/fs/${1}/disk /run/fs/${1}/doc /run/vm/by-id/${1}/ns } diff --git a/host/rootfs/image/usr/bin/run-appimage b/host/rootfs/image/usr/bin/run-appimage index b9464f8b..f62844f8 100755 --- a/host/rootfs/image/usr/bin/run-appimage +++ b/host/rootfs/image/usr/bin/run-appimage @@ -25,6 +25,18 @@ if { if { create-vm-dependencies $id } } +backtick diskdir { + s6-setuidgid fs + + backtick -E mountpoint { + importas -Siu 1 + findmnt -no TARGET -T $1 + } + + if { mkdir -p -- ${mountpoint}/Spectrum/data/spectrum/storage } + mktemp -d -- ${mountpoint}/Spectrum/data/spectrum/storage/tmp.XXXXXX +} + if { s6-envuidgid fs s6-applyuidgid -Uzu 0 @@ -32,15 +44,17 @@ if { multisubstitute { importas -Siu id importas -Siu 1 + importas -Siu diskdir } nsenter --preserve-credentials -S0 --mount=/run/vm/by-id/${id}/ns/mnt --user=/run/vm/by-id/${id}/ns/user - cd /run/fs/${id}/config - if { redirfd -w 1 type echo appimage } - if { touch run } - mount --bind $1 run + cd /run/fs/${id} + if { redirfd -w 1 config/type echo appimage } + if { touch config/run } + if { mount --bind $1 config/run } + mount --bind -- $diskdir disk } importas -Siu id diff --git a/host/rootfs/image/usr/bin/run-flatpak b/host/rootfs/image/usr/bin/run-flatpak index 2d3e7ea0..9a7ffa33 100755 --- a/host/rootfs/image/usr/bin/run-flatpak +++ b/host/rootfs/image/usr/bin/run-flatpak @@ -25,21 +25,34 @@ if { if { create-vm-dependencies $id } } +backtick diskdir { + s6-setuidgid fs + + importas -Siu 1 + + if { mkdir -p -- ${1}/Spectrum/data/spectrum/storage } + mktemp -d -- ${1}/Spectrum/data/spectrum/storage/tmp.XXXXXX +} + if { s6-envuidgid fs s6-applyuidgid -Uzu 0 multisubstitute { importas -Siu id + importas -Siu diskdir elgetpositionals } nsenter --preserve-credentials -S0 --mount=/run/vm/by-id/${id}/ns/mnt --user=/run/vm/by-id/${id}/ns/user + cd /run/fs/${id}/config if { redirfd -w 1 type echo flatpak } - mount-flatpak $@ + if { mount-flatpak $@ } + + mount --bind -- $diskdir /run/fs/${id}/disk } importas -Siu id diff --git a/host/rootfs/image/usr/bin/run-vmm b/host/rootfs/image/usr/bin/run-vmm index 7c2b9af5..4661f5f5 100755 --- a/host/rootfs/image/usr/bin/run-vmm +++ b/host/rootfs/image/usr/bin/run-vmm @@ -1,7 +1,37 @@ -#!/bin/execlineb -s0 +#!/bin/execlineb # SPDX-License-Identifier: EUPL-1.2+ # SPDX-FileCopyrightText: 2024-2025 Alyssa Ross +if { + backtick -D "" mnt { + importas -Siu 1 + nsenter --mount=/run/vm/by-id/${1}/ns/mnt + findmnt -no FSTYPE,SOURCE /run/fs/${1}/disk + } + + multisubstitute { + importas -Siu mnt + importas -Siu 1 + } + + case $mnt { + "^$|^tmpfs fallback$" { + s6-envuidgid fs + s6-applyuidgid -Uzu 0 + nsenter --preserve-credentials -S0 + --mount=/run/vm/by-id/${1}/ns/mnt + --user=/run/vm/by-id/${1}/ns/user + foreground { + redirfd -w 2 /dev/null + umount -- /run/fs/${1}/disk + } + mount -t tmpfs -o mode=0700 -- fallback /run/fs/${1}/disk + } + } +} + +elgetpositionals + s6-ipcserver-socketbinder -B /run/vm/by-id/${1}/vmm getpid -E vmm_pid diff --git a/host/rootfs/image/usr/bin/vm-import b/host/rootfs/image/usr/bin/vm-import index 014eab87..e931ddd7 100755 --- a/host/rootfs/image/usr/bin/vm-import +++ b/host/rootfs/image/usr/bin/vm-import @@ -1,4 +1,4 @@ -#!/bin/execlineb -S2 +#!/bin/execlineb -S3 # SPDX-License-Identifier: EUPL-1.2+ # SPDX-FileCopyrightText: 2023-2024 Alyssa Ross @@ -20,4 +20,19 @@ if { ln -s -- /run/service/vmm/instance/${id} /run/vm/by-id/${id}/service } if { create-vm-dependencies $id } +if { + case $# { + 3 { + s6-envuidgid fs + s6-applyuidgid -Uzu 0 + nsenter --preserve-credentials -S0 + --mount=/run/vm/by-id/${id}/ns/mnt + --user=/run/vm/by-id/${id}/ns/user + + if { mkdir -p -- ${3}/${name} } + mount --bind -- ${3}/${name} /run/fs/${id}/disk + } + } +} + s6-instance-create -- /run/service/vmm $id diff --git a/img/app/Makefile b/img/app/Makefile index 7e3d05b2..2e720a91 100644 --- a/img/app/Makefile +++ b/img/app/Makefile @@ -30,7 +30,7 @@ $(imgdir)/appvm/blk/root.img: ../../scripts/make-gpt.sh ../../scripts/sfdisk-fie build/rootfs.erofs:root:5460386f-2203-4911-8694-91400125c604:root mv $@.tmp $@ -DIRS = dev home/user host run mnt proc sys tmp \ +DIRS = dev host run mnt proc sys tmp \ etc/s6-linux-init/run-image/pipewire \ etc/s6-linux-init/run-image/service \ etc/s6-linux-init/run-image/user \ diff --git a/img/app/image/etc/fstab b/img/app/image/etc/fstab index 5f78ab87..f51eace0 100644 --- a/img/app/image/etc/fstab +++ b/img/app/image/etc/fstab @@ -5,4 +5,3 @@ devpts /dev/pts devpts nosuid,noexec,gid=5,mode=620 0 0 tmpfs /dev/shm tmpfs nosuid,nodev 0 0 sysfs /sys sysfs nosuid,nodev,noexec 0 0 tmpfs /tmp tmpfs nosuid,nodev 0 0 -tmpfs /home/user tmpfs nodev,mode=0700,uid=1000,gid=1000 0 0 diff --git a/img/app/image/etc/s6-rc/app/run b/img/app/image/etc/s6-rc/app/run index f91877d4..f36d153c 100755 --- a/img/app/image/etc/s6-rc/app/run +++ b/img/app/image/etc/s6-rc/app/run @@ -4,11 +4,12 @@ export TMPDIR /run -export HOME /home/user -cd /home/user - if { /etc/mdev/wait virtiofs-host } +if { install -do user -g user /host/disk/home } +export HOME /host/disk/home +cd /host/disk/home + foreground { redirfd -r 0 /host/config/type withstdinas -E type diff --git a/img/app/scripts/start-virtiofsd.elb b/img/app/scripts/start-virtiofsd.elb index 9efb436b..d861a22b 100755 --- a/img/app/scripts/start-virtiofsd.elb +++ b/img/app/scripts/start-virtiofsd.elb @@ -7,7 +7,7 @@ background { if { mkdir -p build/fs } unshare -rUm if { mount -t tmpfs -o nosuid,nodev fs build/fs } - if { mkdir build/fs/config } + if { mkdir build/fs/config build/fs/disk } if { importas -Si CONFIG mount --rbind -- ${CONFIG}/fs build/fs/config } unshare --map-user 1000 --map-group 1000 importas -SsD virtiofsd VIRTIOFSD diff --git a/release/checks/integration/lib.c b/release/checks/integration/lib.c index 51f6bae7..3dcce471 100644 --- a/release/checks/integration/lib.c +++ b/release/checks/integration/lib.c @@ -195,6 +195,7 @@ struct vm *start_qemu(struct config c) "-drive", nullptr, "-drive", nullptr, "-smbios", nullptr, + "-snapshot", "-m", "4G", "-nodefaults", "-machine", "virtualization=on", @@ -242,7 +243,7 @@ struct vm *start_qemu(struct config c) if (asprintf(efi_arg, "file=%s,format=raw,if=pflash,readonly=true", c.drives.efi) == -1 || asprintf(img_arg, "file=%s,format=raw,if=virtio,readonly=true", c.drives.img) == -1 || - asprintf(user_data_arg, "file=%s,format=raw,if=virtio,readonly=true", c.drives.user_data) == -1 || + asprintf(user_data_arg, "file=%s,format=raw,if=virtio", c.drives.user_data) == -1 || asprintf(console_arg, "type=11,value=io.systemd.stub.kernel-cmdline-extra=%s%s", c.serial.console ? "console=" : "", c.serial.console ? c.serial.console : "") == -1) { diff --git a/release/checks/integration/networking.c b/release/checks/integration/networking.c index 078e31fc..d581b647 100644 --- a/release/checks/integration/networking.c +++ b/release/checks/integration/networking.c @@ -151,7 +151,7 @@ void test(struct config c) "mkdir /run/mnt && " "mount \"$(findfs UUID=a7834806-2f82-4faf-8ac4-4f8fd8a474ca)\" /run/mnt && " "s6-rc -bu change vmm-env && " - "vm-import user /run/mnt/vms && " + "vm-import user /run/mnt/vms /run/mnt/storage && " "vm-start \"$(basename \"$(readlink /run/vm/by-name/user.nc)\")\" && " "tail -Fc +0 /run/log/current /run/*.log &\n", vm_console_writer(vm)) == EOF) { diff --git a/release/checks/integration/portal.c b/release/checks/integration/portal.c index 6ba5654a..dc459791 100644 --- a/release/checks/integration/portal.c +++ b/release/checks/integration/portal.c @@ -16,7 +16,7 @@ void test(struct config c) "mkdir /run/mnt && " "mount \"$(findfs UUID=a7834806-2f82-4faf-8ac4-4f8fd8a474ca)\" /run/mnt && " "s6-rc -bu change vmm-env && " - "vm-import user /run/mnt/vms && " + "vm-import user /run/mnt/vms /run/mnt/storage && " "(tail -Fc +0 /run/*.log &) && " "s6-svc -O /run/vm/by-name/user.portal/service && " "vm-start \"$(basename \"$(readlink /run/vm/by-name/user.portal)\")\" && " -- 2.51.0