From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id E139D16B93; Fri, 28 Nov 2025 13:47:20 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 7E0E716B6D; Fri, 28 Nov 2025 13:47:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_MISSING,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=4.0.1 Received: from fout-a5-smtp.messagingengine.com (fout-a5-smtp.messagingengine.com [103.168.172.148]) by atuin.qyliss.net (Postfix) with ESMTPS id 2394B16B68 for ; Fri, 28 Nov 2025 13:47:17 +0000 (UTC) Received: from phl-compute-02.internal (phl-compute-02.internal [10.202.2.42]) by mailfout.phl.internal (Postfix) with ESMTP id 1F82DEC03A9; Fri, 28 Nov 2025 08:47:15 -0500 (EST) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-02.internal (MEProxy); Fri, 28 Nov 2025 08:47:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alyssa.is; h=cc :cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1764337635; x=1764424035; bh=a5g2BCO8Hr QK4ofphcdCLLu7/fnaPJx+nC+4WC/UIqg=; b=MyF6U/37VI6HNP3KFFjrxpdaZJ kE6umUKjY+RQA/GEkDo9liVYlKjRpXKRZLEhmho4GNpO/WH7zWg/Al153Lnnh7Le 8oMrcIAkYVY6xSQO2LNBWPw4L+pfQg+Wz7X/fKONqWTlzsigK4z1EMhOXjzlhHpS aVXhCJNPQlJLhwV6WmLUc14LLPdHWjy0W8znCCSJQaJmf5rq2qh0N6XFXOWd34Jx BgLL6WYpjJWJPCD/Tpqc1A0ObmiYQ9N99Vy3l8gQ1MdDpgXZptE5INeMei9o8faM AM97ilXYHRsF9izS8odeWJILfOG+2rRODJTMA1ptJ5H/X+GNTqDeu2HBH3kw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1764337635; x=1764424035; bh=a5g2BCO8HrQK4ofphcdCLLu7/fnaPJx+nC+ 4WC/UIqg=; b=abzheNCdvTKx6CCL7ixwABp4bU0ZXouutyXk4Xk4jLRG97Mz08P UCYrAM3fb65/Cxe9xXAqzyO6rwNghvDUgAro4X09V/Ur0+IniVBSt5KJrpqsr6bH XQTnJHMzPRd8KHYvOQBU8z46L+j6PVLTWGBec+kEytIvOFrkfoobJPi84YJd10tN PDfTi7PXuxQFAr1cVpXdv0Gv8Qhp1z6FW/q5J6iYE1DdFPvMvkxRWzVn1zoC9NTN aQ4NAhSlxkcbrgKyi1LydizBESVKjnlUDlPzjvUFefRckj9lZSjUHPvfus1N1T8B ROH10TyHfWi4xgcznmHulvOXrze0/vI9ewQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggddvhedttdegucetufdoteggodetrf dotffvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfurfetoffkrfgpnffqhgenuceu rghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujf gurhephffvvefujghffffkgggtsehgtderredttdejnecuhfhrohhmpeetlhihshhsrgcu tfhoshhsuceohhhisegrlhihshhsrgdrihhsqeenucggtffrrghtthgvrhhnpeelveeftd dvvdelgefffffftedtleevhfdujeeuiefffefffffgteffhffhfeffueenucffohhmrghi nhepphhrohhvihguvghrshdrnhgvthenucevlhhushhtvghrufhiiigvpedtnecurfgrrh grmhepmhgrihhlfhhrohhmpehhihesrghlhihsshgrrdhishdpnhgspghrtghpthhtohep vddpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepuggvmhhiohgsvghnohhurhesgh hmrghilhdrtghomhdprhgtphhtthhopeguvghvvghlsehsphgvtghtrhhumhdqohhsrdho rhhg X-ME-Proxy: Feedback-ID: i12284293:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 28 Nov 2025 08:47:14 -0500 (EST) Received: by fw12.qyliss.net (Postfix, from userid 1000) id 016DC2BE58E4; Fri, 28 Nov 2025 14:47:02 +0100 (CET) From: Alyssa Ross To: Demi Marie Obenour Subject: Re: [PATCH v5 11/13] Support updates via systemd-sysupdate In-Reply-To: <20251126-updates-v5-11-fd746748febd@gmail.com> References: <20251126-updates-v5-0-fd746748febd@gmail.com> <20251126-updates-v5-11-fd746748febd@gmail.com> Date: Fri, 28 Nov 2025 14:47:01 +0100 Message-ID: <87zf86nuay.fsf@alyssa.is> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Message-ID-Hash: YPXQIWOUC3JE4POFKWHD3QEPH6I2YEAY X-Message-ID-Hash: YPXQIWOUC3JE4POFKWHD3QEPH6I2YEAY X-MailFrom: hi@alyssa.is X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Spectrum OS Development X-Mailman-Version: 3.3.9 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Demi Marie Obenour writes: > diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile > index a6d9f23e9f5277b7c79a53105eb2dfe1bab1451e..74ff64019560aae6387df0e1b= 3409bc174251bdb 100644 > --- a/host/rootfs/Makefile > +++ b/host/rootfs/Makefile > @@ -10,6 +10,7 @@ include file-list.mk > ROOT_FS =3D build >=20=20 > DIRS =3D \ > + boot \ > dev \ > etc/s6-linux-init/env \ > etc/s6-linux-init/run-image/configs \ > @@ -33,13 +34,15 @@ DIRS =3D \ > etc/s6-linux-init/run-image/vm/by-id \ > etc/s6-linux-init/run-image/vm/by-name \ > ext \ > + home \ > proc \ > run \ > - sys > + sys \ > + tmp >=20=20 > FIFOS =3D etc/s6-linux-init/run-image/service/s6-svscan-log/fifo >=20=20 > -BUILD_FILES =3D build/etc/s6-rc > +BUILD_FILES =3D build/etc/s6-rc build/etc/os-release build/etc/update-url >=20=20 > # This rule produces three files but Make only (portably) > # supports one output per rule. Instead of resorting to temporary > @@ -59,12 +62,22 @@ $(ROOT_FS_IMAGE): ../../scripts/make-erofs.sh $(PACKA= GES_FILE) $(FILES) $(BUILD_ > mkdir -p $(ROOT_FS) && \ > { \ > cat $(PACKAGES_FILE) ;\ > + printf '%s\n%s\n' "$$UPDATE_SIGNING_KEY" /etc/systemd/import-pubrin= g.gpg; \ Inconsistent use of shell variable instead of make macro. > for file in $(FILES) $(LINKS); do printf '%s\n%s\n' $$file "$${file= #image/}"; done ;\ > for file in $(BUILD_FILES); do printf '%s\n%s\n' $$file $${file#bui= ld/}; done ;\ > printf 'build/empty\n%s\n' $(DIRS) ;\ > printf 'build/fifo\n%s\n' $(FIFOS) ;\ > } | ../../scripts/make-erofs.sh $@ >=20=20 > +build/etc/update-url: > + mkdir -p build/etc > + # might have metacharacters, so avoid interpolation > + printf %s\\n "$${UPDATE_URL:?'update URL empty or missing'}" > build/et= c/update-url I'm learning so many shell parameter expansions I didn't know from you :) > diff --git a/host/rootfs/image/usr/bin/spectrum-update b/host/rootfs/imag= e/usr/bin/spectrum-update > new file mode 100755 > index 0000000000000000000000000000000000000000..613b43570d0538fce20296ccb= 1de2a6364e0df55 > --- /dev/null > +++ b/host/rootfs/image/usr/bin/spectrum-update > @@ -0,0 +1,92 @@ > +#!/bin/execlineb -WS1 > +# SPDX-License-Identifier: EUPL-1.2+ > +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour > + > +if { mkdir -p -m 0700 /run/updater } > + > +# Take a global lock to avoid races. > +s6-setlock /run/update-lock > + > +foreground { redirfd -w 2 /dev/null rmdir -- $1 } > +if { umask 0077 mkdir -p -- $1 } > +cd $1 > +foreground { > + # If this exists already that is okay. > + foreground { redirfd -w 2 /dev/null btrfs subvolume create -- shared } > + > + # Delete any stale temporary files. Delete any existing signature > + # files. If the VM is still running (it should not be), the VM might > + # have write access to the directory. However, updates-dir-check is > + # safe against that. > + if { updates-dir-check cleanup shared } > + > + if { > + foreground { > + # TODO: suppress only "subvolume does not exist" errors. > + redirfd -w 2 /dev/null > + btrfs subvolume delete snapshot > + } > + rm -f snapshot > + } > + > + backtick -E update_vm_id { > + backtick -E id_path { readlink /run/vm/by-name/sys.appvm-systemd-sys= update } > + basename -- $id_path > + } > + > + # $fsdir is read-only to the guest, but read-write to the host. > + # Directories bind-mounted into it are read-write to the guest. > + # See etc/s6-linux-init/run-image/service/vhost-user-fs/template/run > + # for details. > + This still refers to a non-existent variable. > + # Set up /etc with what the VM needs. The VM will overlay this > + # on its own /etc. > + # > + # In the future, this should use a bind mount instead of copying > + # into a tmpfs. However, this would significantly complicate the > + # cleanup code. Deleting fs/etc would require undoing the bind > + # mounts instead of rm -rf. Once this code is in a separate mount > + # namespace, the copies should be replaced by bind mounts. > + if { > + if { rm -rf -- /run/vm/by-id/${update_vm_id}/fs/etc } > + umask 022 > + if { mkdir -p -- /run/vm/by-id/${update_vm_id}/fs/updates /run/vm/by= -id/${update_vm_id}/fs/etc/systemd } > + if { cp -R -- /etc/vm-sysupdate.d /etc/update-url /run/vm/by-id/${up= date_vm_id}/fs/etc } > + cp -- /etc/systemd/import-pubring.gpg /run/vm/by-id/${update_vm_id}/= fs/etc/systemd > + } > + > + # If the directory is already mounted, unmount it. This prevents a > + # confusing error from mount. > + foreground { redirfd -w 2 /dev/null umount -- /run/vm/by-id/${update_v= m_id}/fs/updates } > + > + # Share the update directory with the VM. > + if { mount --bind -- shared /run/vm/by-id/${update_vm_id}/fs/updates } > + > + # Start the update VM. > + if { vm-start $update_vm_id } > + > + # Wait for the VM to exit. > + # TODO: This is racy. If the update finishes before this code runs, > + # the s6-svwait call will fail. > + if { s6-svwait -D /run/service/vmm/instance/${update_vm_id} } > + > + # Remove the bind mount. > + if { umount -- /run/vm/by-id/${update_vm_id}/fs/updates } > + > + # Ensure that the VM cannot change the directory > + # while systemd-sysupdate is using it. > + if { btrfs subvolume snapshot -- shared snapshot } > + > + # Validate the update directory. Delete any stale temporary files. > + # Check that a signature file was downloaded. > + if { updates-dir-check check snapshot } > + > + unshare --mount > + if { mount --bind -o ro -- snapshot /run/updater } > + > + /usr/lib/systemd/systemd-sysupdate update Why not just make a readonly snapshot? (btrfs subvolume snapshot -r) > diff --git a/vm/app/systemd-sysupdate/default.nix b/vm/app/systemd-sysupd= ate/default.nix > new file mode 100644 > index 0000000000000000000000000000000000000000..69be0bab500ea2ea6cb3b6d71= edbf1a3e7bddbba > --- /dev/null > +++ b/vm/app/systemd-sysupdate/default.nix > @@ -0,0 +1,26 @@ > +# SPDX-License-Identifier: MIT > +# SPDX-FileCopyrightText: 2023 Alyssa Ross > +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour > + > +import ../../../lib/call-package.nix ( > +{ callSpectrumPackage, curl, lib, src > +, runCommand, systemd, writeScript > +}: > + > +let > + downloadUpdate =3D builtins.path { > + name =3D "download-update"; > + path =3D ./download-update; > + }; builtins.path is overkill here surely, as opposed to just writing ${./download-update} below? > +in > + > +callSpectrumPackage ../../make-vm.nix {} { > + providers.net =3D [ "sys.netvm" ]; > + type =3D "nix"; > + run =3D writeScript "run-script" '' > +#!/usr/bin/env -S execlineb -WS0 #!/bin/execlineb -WS0 would be fine =E2=80=94 we know that'll exist in the = VM. > diff --git a/vm/app/systemd-sysupdate/download-update b/vm/app/systemd-sy= supdate/download-update > new file mode 100755 > index 0000000000000000000000000000000000000000..eada41c6c8ad5edcedd9f4d76= b76492e0b8be826 > --- /dev/null > +++ b/vm/app/systemd-sysupdate/download-update > @@ -0,0 +1,68 @@ > +#!/usr/bin/env -S execlineb -WS0 > +# SPDX-License-Identifier: EUPL-1.2+ > +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour > +export LC_ALL C > +export LANGUAGE C > +if { mount -toverlay -olowerdir=3D/run/virtiofs/virtiofs0/etc:/etc -- ov= erlay /etc } > +backtick tmpdir { mktemp -d /tmp/sysupdate-XXXXXX } > +# Not a useless use of cat: if there are NUL bytes in the URL > +# busybox's awk might misbehave. > +backtick update_url { cat /etc/update-url } > +if { > + backtick sed_rhs { > + # Use awk to both validate the URL and to escape sed metacharacters. > + # Reject URLs with control characters, query parameters, or fragment= s. > + # They *cannot* work and so are rejected to produce better error mes= sages. > + # > + # curl rejects control characters with "Malformed input to a URL fun= ction". > + # Fragment specifiers ("#") and query parameters ("?") break concate= nating > + # /SHA256SUMS and /SHA256SUMS.sha256.asc onto the update URL. Also,= it is > + # simpler to reject update URLs that contain whitespace than to try = to > + # escape them. > + # > + # Backslash needs to be escaped once for systemd-sysupdate and again= for sed. > + # Ampersand needs to be escaped once for sed. > + awk "BEGIN { > + update_url =3D ENVIRON[\"update_url\"]; > + if (update_url ~ /^[^\\001-\\040?#\\x7F]+$/) { > + # Use & to avoid extra escaping (16 or 32 backslashes!) > + # and a divergence between POSIX and GNU awk. > + gsub(/\\\\/, \"&&&&\", update_url); > + gsub(/&/, \"\\\\\\\\&\", update_url); > + print update_url; > + exit 0; > + } else { > + print ARGV[2] > \"/dev/stderr\"; > + exit 100; > + } > + }" -- $3 > + "Bad update URL from host: control characters, whitespace, query par= ameters, and fragment specifiers not allowed" > + } > + elglob -w -0 transfer_file_ /etc/vm-sysupdate.d/*.transfer > + forx -E transfer_file { $transfer_file_ } > + backtick target_basename { > + basename -- $transfer_file > + } > + multisubstitute { > + importas -iuS sed_rhs > + importas -iuS target_basename > + importas -iuS tmpdir > + define sed_input $transfer_file > + } You could avoid some serial substitution here if you wanted, by not passing -E to forx: forx transfer_file { $transfer_file_ } backtick target_basename { importas -iuS transfer_file basename -- $transfer_file } multisubstitute { =E2=80=A6 } > + redirfd -w 1 ${tmpdir}/${target_basename} > + sed -E -- "s#@UPDATE_URL@#${sed_rhs}#g" $sed_input Using awk to escape stuff for sed seems a bit Rube Goldberg. Would it make more sense to just do the replacement in the awk program? Actually a lot of this might be nicer in awk than execline? Feel free to tell me to leave it this way for now, though. > +} > +multisubstitute { > + importas -iuS update_url > + importas -iuS CURL_PATH > + importas -iuS SYSTEMD_SYSUPDATE_PATH > + importas -iuS tmpdir > +} > +if { $SYSTEMD_SYSUPDATE_PATH --definitions=3D${tmpdir} update } > +# [ and ] are allowed in update URLs so that IPv6 addresses work, but > +# they cause globbing in the curl command-line tool by default. Use --g= loboff > +# to disable this feature. > +if { $CURL_PATH -L --proto-redir =3Dhttp,https --globoff > + -o /run/virtiofs/virtiofs0/updates/SHA256SUMS -- ${update_url}/SHA2= 56SUMS } > +$CURL_PATH -L --proto-redir =3Dhttp,https --globoff > + -o /run/virtiofs/virtiofs0/updates/SHA256SUMS.sha256.asc -- ${updat= e_url}/SHA256SUMS.sha256.asc Much easier to understand now. Thanks! --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQQGoGac7QfI+H5ZtFCZddwkt31pFQUCaSmn1QAKCRCZddwkt31p FS/xAP4kTgewzoDTv2YVPwWsxb35Wuzn6Qw7AZK811KNSNKDNQD9EQwqf4e7QFnx XZfo8yTqgCQz2BKSSp2yZSCM2TYE1gM= =6+MM -----END PGP SIGNATURE----- --=-=-=--