* [PATCH] Set up control groups for most services
@ 2026-06-20 14:23 Demi Marie Obenour
2026-06-20 17:27 ` [PATCH v2] " Demi Marie Obenour
0 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-06-20 14:23 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Alyssa Ross, Demi Marie Obenour
The cgroups are handled by a Rust tool. The name of the cgroup is
autogenerated from the service name.
Using cgroups for process control is not implemented yet.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
.codespellrc | 2 +-
host/rootfs/default.nix | 6 +-
host/rootfs/file-list.mk | 12 +
host/rootfs/image/etc/fstab | 1 +
.../s6-linux-init/run-image/service/getty-tty2/run | 1 +
.../s6-linux-init/run-image/service/getty-tty3/run | 1 +
.../s6-linux-init/run-image/service/getty-tty4/run | 1 +
.../run-image/service/root-terminal/run | 1 +
.../run-image/service/s6-linux-init-shutdownd/run | 1 +
.../run-image/service/s6-svscan-log/run | 1 +
.../service/serial-getty-generator/finish | 5 +
.../run-image/service/serial-getty-generator/run | 3 +-
.../run-image/service/serial-getty/finish | 5 +
.../run-image/service/serial-getty/run | 3 +-
.../run-image/service/serial-getty/template/run | 1 +
.../run-image/service/vm-services/finish | 5 +
.../run-image/service/vm-services/run | 3 +-
.../vm-services/template/data/service/dbus/finish | 5 +
.../vm-services/template/data/service/dbus/run | 13 +-
.../template/data/service/spectrum-router/finish | 5 +
.../template/data/service/spectrum-router/run | 3 +-
.../template/data/service/vhost-user-fs/finish | 5 +
.../template/data/service/vhost-user-fs/run | 1 +
.../template/data/service/vhost-user-gpu/finish | 5 +
.../template/data/service/vhost-user-gpu/run | 4 +-
.../xdg-desktop-portal-spectrum-host/finish | 5 +
.../service/xdg-desktop-portal-spectrum-host/run | 2 +
.../run-image/service/vm-services/template/finish | 5 +
.../run-image/service/vm-services/template/run | 1 +
.../etc/s6-linux-init/run-image/service/vmm/finish | 5 +
.../etc/s6-linux-init/run-image/service/vmm/run | 3 +-
host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +
host/rootfs/image/etc/s6-rc/systemd-udevd/run | 2 +-
host/rootfs/image/etc/s6-rc/weston/finish | 5 +
host/rootfs/image/etc/s6-rc/weston/run | 2 +-
pkgs/default.nix | 1 +
tools/cgroup-setup/Cargo.lock | 67 ++++++
tools/cgroup-setup/Cargo.lock.license | 2 +
tools/cgroup-setup/Cargo.toml | 17 ++
tools/cgroup-setup/default.nix | 18 ++
tools/cgroup-setup/src/cgroup.rs | 217 ++++++++++++++++++
tools/cgroup-setup/src/main.rs | 253 +++++++++++++++++++++
42 files changed, 687 insertions(+), 16 deletions(-)
diff --git a/.codespellrc b/.codespellrc
index d8023afc64ec44e98a88c5e397c8d5c681dde063..ae20da8309530759ac729689825a4afc683afa09 100644
--- a/.codespellrc
+++ b/.codespellrc
@@ -2,4 +2,4 @@
# SPDX-License-Identifier: CC0-1.0
[codespell]
-ignore-words-list = crate,passt,rouge,ser
+ignore-words-list = crate,passt,rouge,ser,WRONLY
diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix
index 6bfeefbe0a5f76c1538ccb40e5eb8f291f5d3592..ccf626e2ec0f4bf96573dc5edf058c9375bb65f6 100644
--- a/host/rootfs/default.nix
+++ b/host/rootfs/default.nix
@@ -8,7 +8,7 @@ import ../../lib/call-package.nix (
}:
pkgsMusl.callPackage (
-{ spectrum-host-tools, spectrum-router
+{ spectrum-host-tools, spectrum-router, spectrum-cgroup-setup
, lib, stdenvNoCC, nixos, runCommand, writeClosure, erofs-utils, s6-rc
, btrfs-progs, bubblewrap, busybox, cloud-hypervisor, cosmic-files
, crosvm, cryptsetup, dejavu_fonts, dbus, execline, foot, fuse3
@@ -27,8 +27,8 @@ let
packages = [
btrfs-progs bubblewrap cloud-hypervisor cosmic-files crosvm cryptsetup dbus
execline fuse3 inotify-tools iproute2 jq kmod mdevd mount-flatpak s6
- s6-linux-init s6-rc shadow socat spectrum-host-tools spectrum-router
- virtiofsd xdg-desktop-portal-spectrum-host
+ s6-linux-init s6-rc shadow socat spectrum-cgroup-setup spectrum-host-tools
+ spectrum-router virtiofsd xdg-desktop-portal-spectrum-host
(foot.override { allowPgo = false; })
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 3899d620717fc97f42e669e5313c4100dcf5b1cd..acf5534e7b966811c545daf2d3aa4f26ca59e15e 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -21,25 +21,35 @@ FILES = \
image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run \
image/etc/s6-linux-init/run-image/service/s6-svscan-log/notification-fd \
image/etc/s6-linux-init/run-image/service/s6-svscan-log/run \
+ image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish \
image/etc/s6-linux-init/run-image/service/serial-getty-generator/run \
+ image/etc/s6-linux-init/run-image/service/serial-getty/finish \
image/etc/s6-linux-init/run-image/service/serial-getty/notification-fd \
image/etc/s6-linux-init/run-image/service/serial-getty/run \
image/etc/s6-linux-init/run-image/service/serial-getty/template/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/finish \
image/etc/s6-linux-init/run-image/service/vm-services/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/run \
+ image/etc/s6-linux-init/run-image/service/vmm/finish \
image/etc/s6-linux-init/run-image/service/vmm/notification-fd \
image/etc/s6-linux-init/run-image/service/vmm/run \
image/etc/s6-linux-init/run-image/service/vmm/template/notification-fd \
@@ -96,6 +106,7 @@ S6_RC_FILES = \
image/etc/s6-rc/systemd-udevd-coldplug/dependencies.d/systemd-udevd \
image/etc/s6-rc/systemd-udevd-coldplug/type \
image/etc/s6-rc/systemd-udevd-coldplug/up \
+ image/etc/s6-rc/systemd-udevd/finish \
image/etc/s6-rc/systemd-udevd/notification-fd \
image/etc/s6-rc/systemd-udevd/run \
image/etc/s6-rc/systemd-udevd/type \
@@ -108,6 +119,7 @@ S6_RC_FILES = \
image/etc/s6-rc/vmm-env/contents.d/systemd-udevd-coldplug \
image/etc/s6-rc/vmm-env/type \
image/etc/s6-rc/weston/dependencies.d/systemd-udevd-coldplug \
+ image/etc/s6-rc/weston/finish \
image/etc/s6-rc/weston/notification-fd \
image/etc/s6-rc/weston/run \
image/etc/s6-rc/weston/type
diff --git a/host/rootfs/image/etc/fstab b/host/rootfs/image/etc/fstab
index 18bb5e45abafeaf43871306ce8233239e5c07f76..d92364d83f4d26f766fcd5b494614c06a630d2fe 100644
--- a/host/rootfs/image/etc/fstab
+++ b/host/rootfs/image/etc/fstab
@@ -6,3 +6,4 @@ tmpfs /dev/shm tmpfs nosuid,nodev 0 0
tmpfs /media tmpfs nosuid,nodev,noexec,nosymfollow,mode=755 0 0
sysfs /sys sysfs nosuid,nodev,noexec 0 0
tmpfs /tmp tmpfs nosuid,nodev 0 0
+cgroup2 /sys/fs/cgroup cgroup2 nosuid,nodev,noexec,nosymfollow 0 0
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
index 4d2cd7172e887e90d9ce87489cdacf41e51f77e8..5b2421a3dedf7c282ea818f4064535d0fc29d511 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty2 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
index 06abf5422365ff4edc9379e5cbcab716e6830399..3ca382f76368174ee1f99ea880827958c57b602c 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty3 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
index 3dd235fc93d403fc8971e923ec62cb025684b67a..983bd82927e468bab6ae20f61411680d173e4924 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty4 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
index 86b9a1ef70300de52f33a84cbfac22a77e00df36..fb55327dff5f189f543fb3629702caad9ff247e2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
@@ -2,6 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# cgroups make no sense here
s6-ipcserver-socketbinder -a 0700 /run/root-terminal
if { chown wayland /run/root-terminal }
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
index 327f4ccc5056e7ef87c089e666090307845cae3a..a6733604e91692f8440fd68efeb1a6b7378be167 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# cgroups make no sense for shutdownd
s6-linux-init-shutdownd -Bc /etc/s6-linux-init
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
index 8f7bd889e50d76a49b98d8ba9e2c6172f287277c..cf3f5c7b499d997fd426062651eca64b3accc146 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
@@ -2,5 +2,6 @@
# SPDX-License-Identifier: ISC
# SPDX-FileCopyrightText: Copyright (c) 2015-2024 Laurent Bercot <ska-skaware@skarnet.org>
+# cgroups make no sense for a logger
redirfd -rnb 0 fifo
s6-log -bpd3 -- T /run/log
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
new file mode 100755
index 0000000000000000000000000000000000000000..6ee682c37219e504e3d9d0e9081ec67e85ad9f5f
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+
+cgroup-setup -- serial-getty-generator
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
index bf66ab0878fc6d8e77ca71a4a89e50c139a6bb11..5c9988c994ed9b942329055673b332d0aa918957 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
@@ -1,7 +1,8 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
piperw 3 4
background {
fdclose 3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
index 78f794202bf174f3c036f3e20755ac087a988277..ed0808b9e45b077023f237a0f9c0e5c830015ac2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
@@ -1,5 +1,6 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
s6-svscan -d3 instance
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
index f3ed9a7f08e60206afd222f4fc02c5fb19527561..775b828c72b9d73bef8a3355fb3ae0536975f474 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021, 2024 Alyssa Ross <hi@alyssa.is>
+# Don't put this in a cgroup. We want nohup to work.
getty -i -n -l /etc/login 0,115200,57600,38400,9600 $1 dumb
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/finish
new file mode 100755
index 0000000000000000000000000000000000000000..ed324e5a0871e6d019bc925a91f8aa8ab1aa6a18
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+
+cgroup-setup -- vm-services-template
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
index 78f794202bf174f3c036f3e20755ac087a988277..ed0808b9e45b077023f237a0f9c0e5c830015ac2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
@@ -1,5 +1,6 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
s6-svscan -d3 instance
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0299fd3705aaf5950e2ac0f686d3d211a74c9fc2
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+
+cgroup-setup -- dbus-daemon
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
index f0507dc92aadf25c092f0719eb767542ac0c4451..1a9b10892805c0350ce12de93e156b2a338ffd24 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
@@ -1,7 +1,8 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
importas -i VM VM
if {
@@ -16,10 +17,14 @@ redirfd -r 0 /dev/null
s6-envuidgid fs
s6-applyuidgid -Uzu 0
-getcwd -E dir
+getcwd dir
+multisubstitute {
+ define VM_ ${VM}
+ importas -iS dir
+}
nsenter --preserve-credentials -S0
- --mount=/run/vm/by-id/${VM}/ns/mnt
- --user=/run/vm/by-id/${VM}/ns/user
+ --mount=/run/vm/by-id/${VM_}/ns/mnt
+ --user=/run/vm/by-id/${VM_}/ns/user
unshare --cgroup --ipc --net --uts
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish
new file mode 100755
index 0000000000000000000000000000000000000000..78d1229247ea2a457833e323a9428e04f9a1ed3c
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- spectrum-router
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
index 3de58c27facc78c48176d8f9a6dd1827ac926f1a..21e37ab8a3dad1ab701a0989fb408fed100657cf 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
@@ -1,8 +1,9 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
importas -i VM VM
s6-ipcserver-socketbinder -a 0770 /run/vm/by-id/${VM}/router-driver.sock
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish
new file mode 100755
index 0000000000000000000000000000000000000000..4a6ca0452c437e9312964164b75843e28734ae18
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- vhost-user-fs
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
index cb059aebc8dfe2c8a5b059d5bfea884de53bf922..d4321263269ae46dd6a1b72e39c57c1c07cb06ba 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
@@ -2,6 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2021-2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
s6-ipcserver-socketbinder -a 0700 -B env/virtiofsd.sock
importas -i VM VM
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish
new file mode 100755
index 0000000000000000000000000000000000000000..c1cbc523415b573c4205d5e77d7d5cc29ff51481
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- crosvm
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
index c3bfafe02ec6d2ee418fe56b033bbc4c7a73b186..ad705da8460a86667b47d40e61fee167d240a9c6 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
@@ -1,8 +1,10 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
+cgroup-setup -- $1
+
s6-ipcserver-socketbinder -a 0700 -b 1 env/crosvm.sock
multisubstitute {
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish
new file mode 100755
index 0000000000000000000000000000000000000000..d0cd4081fee3df4d898ef1e4803c6c8fa3c3ed45
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- xdg-desktop-portal-spectrum-host
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
index 909957b817136149dd01c3b98847238ecda8032d..54860d67f284e7cabbfc955f44ef819ff3f593bc 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
@@ -2,6 +2,8 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
+
importas -i VM VM
export DBUS_SESSION_BUS_ADDRESS unix:path=/run/portal-bus/${VM}
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
new file mode 100755
index 0000000000000000000000000000000000000000..da3fc34f057ce5788ddf3ff172f122cf53f8b705
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- vm-services@${3}
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
index bf90e99eef4106a9ad17dfba310b6edb3eaa64a3..13f2cec03a087be4602c14ed78749cce1f6a26a1 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
@@ -2,6 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
export VM $1
s6-svscan -d3 data/service
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/finish
new file mode 100755
index 0000000000000000000000000000000000000000..51d7d2a7f92b98761f7b836bf9ee4ebdfab41cc7
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+
+cgroup-setup -- vmm
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/run
index 78f794202bf174f3c036f3e20755ac087a988277..ed0808b9e45b077023f237a0f9c0e5c830015ac2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/run
@@ -1,5 +1,6 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
s6-svscan -d3 instance
diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/finish b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
new file mode 100755
index 0000000000000000000000000000000000000000..73e4c0315d345a443f844244c84b8546f076771c
--- /dev/null
+++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- systemd-udevd
diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/run b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
old mode 100644
new mode 100755
index aec6444e951503eae988e666b77fda8f2ae33d72..69c67dd9620e557391076d5505fcdc8bdbddac67
--- a/host/rootfs/image/etc/s6-rc/systemd-udevd/run
+++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
@@ -1,7 +1,7 @@
#!/bin/execlineb -WP
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
-s6-setlock /run/sd-notify-wrapper/systemd-udevd.lock
+cgroup-setup -- systemd-udevd
s6-ipcserver-socketbinder -b0 -m -a 0600 /run/sd-notify-wrapper/systemd-udevd.sock
background -d {
fdmove 1 3
diff --git a/host/rootfs/image/etc/s6-rc/weston/finish b/host/rootfs/image/etc/s6-rc/weston/finish
new file mode 100755
index 0000000000000000000000000000000000000000..a37813bddc0c599eb44ab9901e1685be45cba207
--- /dev/null
+++ b/host/rootfs/image/etc/s6-rc/weston/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- weston
diff --git a/host/rootfs/image/etc/s6-rc/weston/run b/host/rootfs/image/etc/s6-rc/weston/run
old mode 100644
new mode 100755
index fd59586c719391deb546c29578341e16a61ed4ce..5a6e64d41d6e4f2714949f13bad3019479560e04
--- a/host/rootfs/image/etc/s6-rc/weston/run
+++ b/host/rootfs/image/etc/s6-rc/weston/run
@@ -2,6 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2021, 2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- weston
importas -Siu WAYLAND_DISPLAY
piperw 4 3
@@ -40,7 +41,6 @@ redirfd -r 0 /dev/tty1
importas -i home HOME
cd $home
-if { udevadm wait /dev/dri/card0 }
s6-setuidgid wayland
bwrap
# no --unshare-net, breaks udev hotplug
diff --git a/pkgs/default.nix b/pkgs/default.nix
index 4dbdfee8ba330f5ba5c9fedee2d1bb8a44af5722..22c0d782bf4b5467fe77d5a2d71fb8df3af3e756 100644
--- a/pkgs/default.nix
+++ b/pkgs/default.nix
@@ -51,6 +51,7 @@ let
driverSupport = true;
};
spectrum-router = self.callSpectrumPackage ../tools/router {};
+ spectrum-cgroup-setup = self.callSpectrumPackage ../tools/cgroup-setup {};
xdg-desktop-portal-spectrum-host =
self.callSpectrumPackage ../tools/xdg-desktop-portal-spectrum-host {};
diff --git a/tools/cgroup-setup/Cargo.lock b/tools/cgroup-setup/Cargo.lock
new file mode 100644
index 0000000000000000000000000000000000000000..fe967b3aa02c296c87b6b36ac59253dbe0a32de9
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.lock
@@ -0,0 +1,67 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 4
+
+[[package]]
+name = "bitflags"
+version = "2.11.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3"
+
+[[package]]
+name = "cgroup-setup"
+version = "0.0.0"
+dependencies = [
+ "libc",
+ "rustix",
+]
+
+[[package]]
+name = "errno"
+version = "0.3.14"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
+dependencies = [
+ "libc",
+ "windows-sys",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.186"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
+
+[[package]]
+name = "linux-raw-sys"
+version = "0.12.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
+
+[[package]]
+name = "rustix"
+version = "1.1.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
+dependencies = [
+ "bitflags",
+ "errno",
+ "libc",
+ "linux-raw-sys",
+ "windows-sys",
+]
+
+[[package]]
+name = "windows-link"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
+
+[[package]]
+name = "windows-sys"
+version = "0.61.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
+dependencies = [
+ "windows-link",
+]
diff --git a/tools/cgroup-setup/Cargo.lock.license b/tools/cgroup-setup/Cargo.lock.license
new file mode 100644
index 0000000000000000000000000000000000000000..fd7246cd0bc2e9a980898eae42531fef601f524b
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.lock.license
@@ -0,0 +1,2 @@
+// SPDX-License-Identifier: EUPL-1.2+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
diff --git a/tools/cgroup-setup/Cargo.toml b/tools/cgroup-setup/Cargo.toml
new file mode 100644
index 0000000000000000000000000000000000000000..9cce0be706151dfd5264a81afb3ee9a6c851367c
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.toml
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: CC0-1.0
+# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+[package]
+name = "cgroup-setup"
+edition = "2024"
+
+[dependencies]
+libc = "0.2.177"
+rustix = { version = "1.1.2", features = ["fs"] }
+
+[profile.release]
+split-debuginfo = "symbols"
+strip = "symbols"
+lto = true
+panic = "abort"
diff --git a/tools/cgroup-setup/default.nix b/tools/cgroup-setup/default.nix
new file mode 100644
index 0000000000000000000000000000000000000000..9e716b8f270828b733f39961e5ca37a290a872b4
--- /dev/null
+++ b/tools/cgroup-setup/default.nix
@@ -0,0 +1,18 @@
+# SPDX-FileCopyrightText: 2024 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
+# SPDX-License-Identifier: MIT
+
+import ../../lib/call-package.nix (
+{ src, lib, rustPlatform }:
+
+rustPlatform.buildRustPackage {
+ name = "spectrum-cgroup-setup";
+
+ src = lib.fileset.toSource {
+ root = ../..;
+ fileset = lib.fileset.intersection src ./.;
+ };
+ sourceRoot = "source/tools/cgroup-setup";
+
+ cargoLock.lockFile = ./Cargo.lock;
+}) (_: {})
diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs
new file mode 100644
index 0000000000000000000000000000000000000000..3cc6627e01f9bf01118dcfda849d431c128f5017
--- /dev/null
+++ b/tools/cgroup-setup/src/cgroup.rs
@@ -0,0 +1,217 @@
+// SPDX-License-Identifier: EUPL-1.2+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+use std::fs::File;
+use std::io::{Read, Seek};
+use std::os::fd::{AsFd, OwnedFd};
+use std::os::fd::{AsRawFd, BorrowedFd};
+
+use std::path::{Path, PathBuf};
+
+use rustix::fs::AtFlags;
+use rustix::path::Arg;
+use rustix::{
+ fs::{Mode, OFlags, ResolveFlags},
+ io::Errno,
+};
+
+pub(crate) struct LeafCgroup {
+ path: PathBuf,
+ fd: OwnedFd,
+}
+
+mod epoll {
+ use std::os::fd::{AsRawFd, FromRawFd, OwnedFd};
+
+ #[expect(dead_code)]
+ pub(super) struct Epoll(OwnedFd, OwnedFd);
+
+ impl Epoll {
+ #[expect(dead_code)]
+ pub(super) fn new(other_fd: OwnedFd) -> Self {
+ // SAFETY: FFI call with valid arguments
+ let fd = unsafe { libc::epoll_create1(libc::EPOLL_CLOEXEC) };
+ if fd == -1 {
+ panic!("epoll_create1 failed: {}", std::io::Error::last_os_error());
+ }
+
+ let mut event = libc::epoll_event {
+ events: (libc::EPOLLIN | libc::EPOLLPRI | libc::EPOLLRDHUP) as _,
+ r#u64: 0,
+ };
+
+ let res = unsafe {
+ libc::epoll_ctl(
+ fd.as_raw_fd(),
+ libc::EPOLL_CTL_ADD,
+ other_fd.as_raw_fd(),
+ &raw mut event,
+ )
+ };
+ assert_eq!(res, 0, "kernel out of memory?");
+
+ // SAFETY: epoll_create1 returns valid FD or -1 and we checked the error case
+ Self(unsafe { OwnedFd::from_raw_fd(fd) }, other_fd)
+ }
+ }
+}
+
+enum Access {
+ Read,
+ Write,
+}
+
+impl LeafCgroup {
+ fn open_subtree(&self, path: &std::path::Path, access: Access) -> Result<OwnedFd, Errno> {
+ rustix::fs::openat2(
+ self.fd.as_fd(),
+ path,
+ OFlags::NOATIME
+ | OFlags::CLOEXEC
+ | OFlags::NOFOLLOW
+ | match access {
+ Access::Write => OFlags::WRONLY,
+ Access::Read => OFlags::RDONLY,
+ },
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )
+ }
+
+ pub fn kill_processes(&self) -> std::io::Result<()> {
+ let kill_fd = self.open_subtree(std::path::Path::new("cgroup.kill"), Access::Write)?;
+ let wait_file = self.open_subtree(std::path::Path::new("cgroup.events"), Access::Read)?;
+ let poll_fd = wait_file.as_raw_fd();
+ let mut wait_fd = File::from(wait_file);
+ assert_eq!(rustix::io::write(kill_fd.as_fd(), b"1")?, 1, "kernel bug");
+ let mut fds = libc::pollfd {
+ fd: poll_fd,
+ events: libc::POLLIN | libc::POLLPRI | libc::POLLRDHUP,
+ revents: 0,
+ };
+ // SAFETY: FFI call, valid arguments, fds contains 1 element
+ let mut v = vec![];
+ 'a: loop {
+ v.clear();
+ if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
+ panic!("poll failed");
+ }
+ wait_fd
+ .seek(std::io::SeekFrom::Start(0))
+ .expect("Seek on control group file should succeed");
+ wait_fd
+ .read_to_end(&mut v)
+ .expect("reading from control group should work");
+ for substr in v.split(|&c| c == b'\n') {
+ if substr == b"populated 0" {
+ break 'a;
+ }
+ }
+ }
+ Ok(())
+ }
+}
+
+impl AsFd for LeafCgroup {
+ fn as_fd(&self) -> std::os::fd::BorrowedFd<'_> {
+ self.fd.as_fd()
+ }
+}
+
+impl LeafCgroup {
+ pub(crate) fn open_cgroup_at(&self, p: &Path) -> Result<Self, Errno> {
+ let fd = rustix::fs::openat2(
+ self.fd.as_fd(),
+ p,
+ OFlags::NOATIME
+ | OFlags::CLOEXEC
+ | OFlags::NOFOLLOW
+ | OFlags::RDONLY
+ | OFlags::DIRECTORY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )?;
+ Ok(Self {
+ fd,
+ path: self.path.join(p),
+ })
+ }
+ pub(crate) fn open_cgroup(fd: BorrowedFd, p: &Path) -> Result<Self, Errno> {
+ let fd = rustix::fs::openat2(
+ fd,
+ p,
+ OFlags::NOATIME
+ | OFlags::CLOEXEC
+ | OFlags::NOFOLLOW
+ | OFlags::RDONLY
+ | OFlags::DIRECTORY,
+ Mode::empty(),
+ //ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ ResolveFlags::NO_SYMLINKS,
+ )?;
+ Ok(Self {
+ fd,
+ path: p.to_owned(),
+ })
+ }
+ pub(crate) fn make_child(&self, p: &Path) -> Result<(), Errno> {
+ rustix::fs::mkdirat(
+ self.fd.as_fd(),
+ p,
+ Mode::RUSR
+ | Mode::WUSR
+ | Mode::XUSR
+ | Mode::RGRP
+ | Mode::XGRP
+ | Mode::ROTH
+ | Mode::XOTH,
+ )
+ }
+
+ pub(crate) fn delete_child(&self, path: &Path) -> Result<(), Errno> {
+ remove_all(100, self.as_fd(), &path.as_cow_c_str().unwrap())
+ }
+}
+
+fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> {
+ if remaining_depth < 1 {
+ panic!("control groups too deeply nested");
+ }
+ let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating");
+ while let Some(element) = d.next() {
+ let element = element.expect("Iterating through a cgroup directory failed?");
+ if element.file_type() != rustix::fs::FileType::Directory {
+ continue;
+ }
+
+ let remaining_depth = remaining_depth - 1;
+ let d: &rustix::fs::Dir = &d;
+ let dirfd = d.fd().unwrap();
+ let path = element.file_name();
+ remove_all(remaining_depth, dirfd, path)?;
+ }
+ Ok(())
+}
+
+fn remove_all(
+ remaining_depth: usize,
+ dirfd: BorrowedFd<'_>,
+ path: &std::ffi::CStr,
+) -> Result<(), Errno> {
+ if path == c"." || path == c".." {
+ return Ok(());
+ }
+ if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() {
+ return Ok(());
+ }
+ let fd = rustix::fs::openat2(
+ dirfd,
+ path,
+ OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )?;
+ remove_recursively(fd, remaining_depth)?;
+ rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?;
+ Ok(())
+}
diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
new file mode 100644
index 0000000000000000000000000000000000000000..f10eadb013b987d33c741f327e5c180164824801
--- /dev/null
+++ b/tools/cgroup-setup/src/main.rs
@@ -0,0 +1,253 @@
+// SPDX-License-Identifier: EUPL-1.2+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+use std::{
+ ffi::{OsStr, OsString},
+ fs::File,
+ io::Write as _,
+ os::unix::prelude::*,
+ path::{Component, Path, PathBuf},
+};
+
+use rustix::{
+ fs::{FlockOperation, Mode, OFlags, ResolveFlags},
+ io::Errno,
+};
+
+mod cgroup;
+
+macro_rules! fail {
+ ($($arg:tt)*) => {{
+ eprintln!($($arg)*);
+ std::process::exit(1)}
+ };
+}
+
+fn parse_string(original: &str, msg: &str, num: usize) -> u64 {
+ let (to_parse, scale, unit): (_, u64, &'static str) = match original.as_bytes() {
+ [rest @ .., b'K', b'B'] => (rest, 1_000, "KB"),
+ [rest @ .., b'M', b'B'] => (rest, 1_000_000, "MB"),
+ [rest @ .., b'G', b'B'] => (rest, 1_000_000_000, "GB"),
+ [rest @ .., b'T', b'B'] => (rest, 1_000_000_000_000, "TB"),
+ [rest @ .., b'P', b'B'] => (rest, 1_000_000_000_000_000, "PB"),
+ [rest @ .., b'E', b'B'] => (rest, 1_000_000_000_000_000_000, "EB"),
+ [rest @ .., b'K', b'i', b'B'] => (rest, 1 << 10, "KiB"),
+ [rest @ .., b'M', b'i', b'B'] => (rest, 1 << 20, "MiB"),
+ [rest @ .., b'G', b'i', b'B'] => (rest, 1 << 30, "GiB"),
+ [rest @ .., b'T', b'i', b'B'] => (rest, 1 << 40, "TiB"),
+ [rest @ .., b'P', b'i', b'B'] => (rest, 1 << 50, "PiB"),
+ [rest @ .., b'E', b'i', b'B'] => (rest, 1 << 60, "EiB"),
+ [rest @ .., b'B'] if rest.last().map(u8::is_ascii_digit).unwrap_or(false) => (rest, 1, "B"),
+ _ => fail!("{msg}: {original:?} is missing a unit suffix"),
+ };
+ let (to_parse, radix) = match to_parse {
+ [b'0', b'x' | b'X', s @ ..] => (s, 16),
+ _ => (to_parse, 10),
+ };
+ // SAFETY: lopping ASCII bytes off start
+ // and end of UTF-8 string leaves valid UTF_8
+ let result: u64 =
+ match u64::from_str_radix(unsafe { str::from_utf8_unchecked(to_parse) }, radix) {
+ Ok(e) => e,
+ Err(e) => {
+ fail!("{msg}: argument {num}: Cannot parse as u64: {e}")
+ }
+ };
+ if to_parse[0] == b'0' || to_parse[0] == b'+' {
+ fail!(
+ "{msg}: argument {num}: Leading {} in {original} not allowed",
+ to_parse[0] as char
+ )
+ }
+ result.checked_mul(scale).unwrap_or_else(|| {
+ fail!(
+ "{msg}: {original} is too large for unit {unit} ({original} × {scale} = {} > {})",
+ u128::from(result) * u128::from(scale),
+ u64::MAX,
+ )
+ })
+}
+
+fn main() {
+ let mut args = std::env::args_os();
+ if args.next().is_none() {
+ fail!("No command line arguments (argv[0] is NULL)");
+ }
+ let mut args = args.enumerate().map(|(num, arg)| {
+ match str::from_utf8(<OsStr as OsStrExt>::as_bytes(&arg)) {
+ Ok(v) => (num, v.to_owned()),
+ Err(e) => {
+ fail!(
+ "Argument {num} bytes {} through {} are invalid UTF-8",
+ e.valid_up_to(),
+ e.error_len().unwrap_or_else(|| arg.len())
+ )
+ }
+ }
+ });
+ let mut finished = false;
+ let mut positional_arguments;
+ let mut values = [
+ ("pids.max", None),
+ ("memory.high", None),
+ ("memory.max", None),
+ ];
+
+ let last = loop {
+ let Some((num, arg_str)) = args.next() else {
+ finished = true;
+ break None;
+ };
+ if arg_str.as_bytes().get(0) != Some(&b'-') {
+ break Some(arg_str.to_owned());
+ }
+ let Some(suffix) = arg_str.strip_prefix("--") else {
+ fail!("Short options (with a single '-', as in {arg_str:?}) are not supported")
+ };
+
+ if suffix.is_empty() {
+ break None;
+ }
+
+ for value in &mut values {
+ if let Some(suffix) = suffix.strip_prefix(value.0)
+ && let Some(suffix) = suffix.strip_prefix("=")
+ {
+ value.1 = Some(parse_string(suffix, &value.0, num));
+ }
+ }
+
+ fail!("Unknown option {arg_str:?}")
+ };
+
+ if finished {
+ positional_arguments = vec![]
+ } else {
+ positional_arguments = last.into_iter().chain(args.map(|a| a.1)).collect()
+ }
+
+ if positional_arguments.is_empty() {
+ fail!(
+ "Have {} arguments, expect at least 1",
+ positional_arguments.len()
+ )
+ }
+
+ // slow but we do not care
+ let mut child_path = positional_arguments.remove(0);
+ if child_path.len() > 247 {
+ fail!("Cgroup name {child_path:?} too long");
+ }
+ if child_path.as_bytes().contains(&b'/') {
+ fail!("Cgroup name {child_path:?} contains /");
+ }
+ child_path += ".service";
+ let child_path = Path::new(&child_path);
+
+ let local_cgroup =
+ std::fs::read("/proc/thread-self/cgroup").expect("Should be able to read from procfs");
+ if !local_cgroup.starts_with(b"0::/")
+ || !local_cgroup.ends_with(b"\n")
+ || local_cgroup.contains(&b'\0')
+ {
+ fail!("Kernel bug: Bad contents of /proc/thread-self/cgroup");
+ }
+
+ let mut local_cgroup = PathBuf::from(<OsString as OsStringExt>::from_vec(
+ local_cgroup[4..local_cgroup.len() - 1].to_owned(),
+ ));
+
+ match local_cgroup.components().next_back() {
+ Some(Component::Prefix(_)) => unreachable!("does not occur on Linux"),
+ Some(Component::RootDir) | None => {}
+ Some(Component::CurDir) => unreachable!("not produced by kernel"),
+ Some(Component::ParentDir) => unreachable!("not produced by kernel"),
+ Some(Component::Normal(os_str)) => {
+ if os_str.as_bytes() == b"@inner.service" {
+ let _ = local_cgroup.pop();
+ }
+ }
+ }
+
+ if local_cgroup.as_os_str().is_empty() {
+ local_cgroup = PathBuf::from(".");
+ }
+
+ let cgroup_root = rustix::fs::openat2(
+ rustix::fs::CWD,
+ Path::new("/sys/fs/cgroup"),
+ OFlags::DIRECTORY | OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW,
+ Mode::empty(),
+ ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_SYMLINKS,
+ )
+ .expect("Cannot get cgroup root");
+
+ let cgroup = match cgroup::LeafCgroup::open_cgroup(cgroup_root.as_fd(), &local_cgroup) {
+ Ok(e) => e,
+ Err(e) => fail!(
+ "Bad cgroup operation {e:?} opening {}",
+ local_cgroup.display()
+ ),
+ };
+
+ rustix::fs::flock(cgroup.as_fd(), FlockOperation::LockExclusive).expect("oops");
+ purge_cgroup(&cgroup, child_path);
+ if positional_arguments.is_empty() {
+ return;
+ }
+ if let Err(e) = cgroup.make_child(child_path) {
+ fail!("Cannot create child cgroup: {e}")
+ }
+ let child = cgroup
+ .open_cgroup_at(Path::new(&child_path))
+ .expect("cannot open child cgroup");
+ child
+ .make_child(Path::new("@inner.service"))
+ .expect("cannot make child cgroup");
+ let child = child
+ .open_cgroup_at(Path::new("@inner.service"))
+ .expect("cannot open child cgroup");
+
+ for (name, value) in values {
+ if let Some(value) = value {
+ write_cgroup_value(&child, name, &value.to_string());
+ }
+ }
+
+ // SAFETY: safe FFI call
+ let pid = unsafe { libc::getpid() };
+ write_cgroup_value(&child, "cgroup.procs", &pid.to_string());
+
+ let program = positional_arguments.remove(0);
+ let e = std::process::Command::new(program.clone())
+ .args(positional_arguments)
+ .exec();
+ fail!("Cannot spawn child {:?}: {}", program, e);
+}
+
+fn write_cgroup_value(child: &cgroup::LeafCgroup, name: &str, value: &str) {
+ let path = Path::new(name);
+ let fd = rustix::fs::openat2(
+ child.as_fd(),
+ path,
+ OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::WRONLY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )
+ .unwrap_or_else(|e| fail!("Cannot open {:?}: {}", path, e));
+ let () = File::from(fd)
+ .write_all(value.as_bytes())
+ .unwrap_or_else(|e| fail!("Cannot write {:?} to {:?}: {}", name, path, e));
+}
+
+fn purge_cgroup(cgroup: &cgroup::LeafCgroup, child_path: &Path) {
+ let child = match cgroup.open_cgroup_at(child_path) {
+ Ok(child_cgroup) => child_cgroup,
+ Err(Errno::NOENT) => return,
+ Err(other) => fail!("Cannot open child cgroup {child_path:?}: {other}"),
+ };
+ child.kill_processes().expect("cannot kill children");
+ cgroup
+ .delete_child(child_path)
+ .expect("cannot delete child");
+}
---
base-commit: 36ea9d85403a29f1e700f534020cc4592d78ee50
change-id: 20260528-cgroups-d609e270e649
--
Sincerely,
Demi Marie Obenour (she/her/hers)
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v2] Set up control groups for most services
2026-06-20 14:23 [PATCH] Set up control groups for most services Demi Marie Obenour
@ 2026-06-20 17:27 ` Demi Marie Obenour
2026-06-24 12:13 ` Alyssa Ross
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
0 siblings, 2 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-06-20 17:27 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Alyssa Ross, Demi Marie Obenour
The cgroups are handled by a Rust tool. The name of the cgroup is
autogenerated from the service name.
Using cgroups for process control is not implemented yet.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Changes in v2:
- Omit resource control support. It was completely broken, and the way
Spectrum run scripts work means that it is better to setup resource
controls just before the execve() into the final service process.
This will be done by a separate tool.
- Link to v1: https://spectrum-os.org/lists/archives/spectrum-devel/20260620-cgroups-v1-1-0e5abf35101b@gmail.com
---
.codespellrc | 2 +-
host/rootfs/default.nix | 6 +-
host/rootfs/file-list.mk | 12 ++
host/rootfs/image/etc/fstab | 1 +
.../s6-linux-init/run-image/service/getty-tty2/run | 1 +
.../s6-linux-init/run-image/service/getty-tty3/run | 1 +
.../s6-linux-init/run-image/service/getty-tty4/run | 1 +
.../run-image/service/root-terminal/run | 1 +
.../run-image/service/s6-linux-init-shutdownd/run | 1 +
.../run-image/service/s6-svscan-log/run | 1 +
.../service/serial-getty-generator/finish | 5 +
.../run-image/service/serial-getty-generator/run | 3 +-
.../run-image/service/serial-getty/finish | 5 +
.../run-image/service/serial-getty/run | 3 +-
.../run-image/service/serial-getty/template/run | 1 +
.../run-image/service/vm-services/finish | 5 +
.../run-image/service/vm-services/run | 3 +-
.../vm-services/template/data/service/dbus/finish | 5 +
.../vm-services/template/data/service/dbus/run | 3 +-
.../template/data/service/spectrum-router/finish | 5 +
.../template/data/service/spectrum-router/run | 3 +-
.../template/data/service/vhost-user-fs/finish | 5 +
.../template/data/service/vhost-user-fs/run | 1 +
.../template/data/service/vhost-user-gpu/finish | 5 +
.../template/data/service/vhost-user-gpu/run | 4 +-
.../xdg-desktop-portal-spectrum-host/finish | 5 +
.../service/xdg-desktop-portal-spectrum-host/run | 2 +
.../run-image/service/vm-services/template/finish | 5 +
.../run-image/service/vm-services/template/run | 1 +
.../etc/s6-linux-init/run-image/service/vmm/finish | 5 +
.../etc/s6-linux-init/run-image/service/vmm/run | 3 +-
host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +
host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +-
host/rootfs/image/etc/s6-rc/weston/finish | 5 +
host/rootfs/image/etc/s6-rc/weston/run | 3 +-
pkgs/default.nix | 1 +
tools/cgroup-setup/Cargo.lock | 67 +++++++
tools/cgroup-setup/Cargo.lock.license | 2 +
tools/cgroup-setup/Cargo.toml | 17 ++
tools/cgroup-setup/default.nix | 18 ++
tools/cgroup-setup/src/cgroup.rs | 181 +++++++++++++++++++
tools/cgroup-setup/src/main.rs | 196 +++++++++++++++++++++
42 files changed, 590 insertions(+), 14 deletions(-)
diff --git a/.codespellrc b/.codespellrc
index d8023afc64ec44e98a88c5e397c8d5c681dde063..ae20da8309530759ac729689825a4afc683afa09 100644
--- a/.codespellrc
+++ b/.codespellrc
@@ -2,4 +2,4 @@
# SPDX-License-Identifier: CC0-1.0
[codespell]
-ignore-words-list = crate,passt,rouge,ser
+ignore-words-list = crate,passt,rouge,ser,WRONLY
diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix
index 6bfeefbe0a5f76c1538ccb40e5eb8f291f5d3592..ccf626e2ec0f4bf96573dc5edf058c9375bb65f6 100644
--- a/host/rootfs/default.nix
+++ b/host/rootfs/default.nix
@@ -8,7 +8,7 @@ import ../../lib/call-package.nix (
}:
pkgsMusl.callPackage (
-{ spectrum-host-tools, spectrum-router
+{ spectrum-host-tools, spectrum-router, spectrum-cgroup-setup
, lib, stdenvNoCC, nixos, runCommand, writeClosure, erofs-utils, s6-rc
, btrfs-progs, bubblewrap, busybox, cloud-hypervisor, cosmic-files
, crosvm, cryptsetup, dejavu_fonts, dbus, execline, foot, fuse3
@@ -27,8 +27,8 @@ let
packages = [
btrfs-progs bubblewrap cloud-hypervisor cosmic-files crosvm cryptsetup dbus
execline fuse3 inotify-tools iproute2 jq kmod mdevd mount-flatpak s6
- s6-linux-init s6-rc shadow socat spectrum-host-tools spectrum-router
- virtiofsd xdg-desktop-portal-spectrum-host
+ s6-linux-init s6-rc shadow socat spectrum-cgroup-setup spectrum-host-tools
+ spectrum-router virtiofsd xdg-desktop-portal-spectrum-host
(foot.override { allowPgo = false; })
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 3899d620717fc97f42e669e5313c4100dcf5b1cd..acf5534e7b966811c545daf2d3aa4f26ca59e15e 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -21,25 +21,35 @@ FILES = \
image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run \
image/etc/s6-linux-init/run-image/service/s6-svscan-log/notification-fd \
image/etc/s6-linux-init/run-image/service/s6-svscan-log/run \
+ image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish \
image/etc/s6-linux-init/run-image/service/serial-getty-generator/run \
+ image/etc/s6-linux-init/run-image/service/serial-getty/finish \
image/etc/s6-linux-init/run-image/service/serial-getty/notification-fd \
image/etc/s6-linux-init/run-image/service/serial-getty/run \
image/etc/s6-linux-init/run-image/service/serial-getty/template/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/finish \
image/etc/s6-linux-init/run-image/service/vm-services/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/run \
+ image/etc/s6-linux-init/run-image/service/vmm/finish \
image/etc/s6-linux-init/run-image/service/vmm/notification-fd \
image/etc/s6-linux-init/run-image/service/vmm/run \
image/etc/s6-linux-init/run-image/service/vmm/template/notification-fd \
@@ -96,6 +106,7 @@ S6_RC_FILES = \
image/etc/s6-rc/systemd-udevd-coldplug/dependencies.d/systemd-udevd \
image/etc/s6-rc/systemd-udevd-coldplug/type \
image/etc/s6-rc/systemd-udevd-coldplug/up \
+ image/etc/s6-rc/systemd-udevd/finish \
image/etc/s6-rc/systemd-udevd/notification-fd \
image/etc/s6-rc/systemd-udevd/run \
image/etc/s6-rc/systemd-udevd/type \
@@ -108,6 +119,7 @@ S6_RC_FILES = \
image/etc/s6-rc/vmm-env/contents.d/systemd-udevd-coldplug \
image/etc/s6-rc/vmm-env/type \
image/etc/s6-rc/weston/dependencies.d/systemd-udevd-coldplug \
+ image/etc/s6-rc/weston/finish \
image/etc/s6-rc/weston/notification-fd \
image/etc/s6-rc/weston/run \
image/etc/s6-rc/weston/type
diff --git a/host/rootfs/image/etc/fstab b/host/rootfs/image/etc/fstab
index 18bb5e45abafeaf43871306ce8233239e5c07f76..d92364d83f4d26f766fcd5b494614c06a630d2fe 100644
--- a/host/rootfs/image/etc/fstab
+++ b/host/rootfs/image/etc/fstab
@@ -6,3 +6,4 @@ tmpfs /dev/shm tmpfs nosuid,nodev 0 0
tmpfs /media tmpfs nosuid,nodev,noexec,nosymfollow,mode=755 0 0
sysfs /sys sysfs nosuid,nodev,noexec 0 0
tmpfs /tmp tmpfs nosuid,nodev 0 0
+cgroup2 /sys/fs/cgroup cgroup2 nosuid,nodev,noexec,nosymfollow 0 0
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
index 4d2cd7172e887e90d9ce87489cdacf41e51f77e8..5b2421a3dedf7c282ea818f4064535d0fc29d511 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty2 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
index 06abf5422365ff4edc9379e5cbcab716e6830399..3ca382f76368174ee1f99ea880827958c57b602c 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty3 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
index 3dd235fc93d403fc8971e923ec62cb025684b67a..983bd82927e468bab6ae20f61411680d173e4924 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty4 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
index 86b9a1ef70300de52f33a84cbfac22a77e00df36..fb55327dff5f189f543fb3629702caad9ff247e2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
@@ -2,6 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# cgroups make no sense here
s6-ipcserver-socketbinder -a 0700 /run/root-terminal
if { chown wayland /run/root-terminal }
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
index 327f4ccc5056e7ef87c089e666090307845cae3a..a6733604e91692f8440fd68efeb1a6b7378be167 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# cgroups make no sense for shutdownd
s6-linux-init-shutdownd -Bc /etc/s6-linux-init
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
index 8f7bd889e50d76a49b98d8ba9e2c6172f287277c..cf3f5c7b499d997fd426062651eca64b3accc146 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
@@ -2,5 +2,6 @@
# SPDX-License-Identifier: ISC
# SPDX-FileCopyrightText: Copyright (c) 2015-2024 Laurent Bercot <ska-skaware@skarnet.org>
+# cgroups make no sense for a logger
redirfd -rnb 0 fifo
s6-log -bpd3 -- T /run/log
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
index bf66ab0878fc6d8e77ca71a4a89e50c139a6bb11..5c9988c994ed9b942329055673b332d0aa918957 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
@@ -1,7 +1,8 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
piperw 3 4
background {
fdclose 3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
index 78f794202bf174f3c036f3e20755ac087a988277..ed0808b9e45b077023f237a0f9c0e5c830015ac2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
@@ -1,5 +1,6 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
s6-svscan -d3 instance
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
index f3ed9a7f08e60206afd222f4fc02c5fb19527561..775b828c72b9d73bef8a3355fb3ae0536975f474 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021, 2024 Alyssa Ross <hi@alyssa.is>
+# Don't put this in a cgroup. We want nohup to work.
getty -i -n -l /etc/login 0,115200,57600,38400,9600 $1 dumb
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
index 78f794202bf174f3c036f3e20755ac087a988277..ed0808b9e45b077023f237a0f9c0e5c830015ac2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
@@ -1,5 +1,6 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
s6-svscan -d3 instance
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
index f0507dc92aadf25c092f0719eb767542ac0c4451..89676ac91d3e22db7726fc869a065137b031d0be 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
@@ -1,7 +1,8 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
importas -i VM VM
if {
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
index 3de58c27facc78c48176d8f9a6dd1827ac926f1a..21e37ab8a3dad1ab701a0989fb408fed100657cf 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
@@ -1,8 +1,9 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
importas -i VM VM
s6-ipcserver-socketbinder -a 0770 /run/vm/by-id/${VM}/router-driver.sock
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
index cb059aebc8dfe2c8a5b059d5bfea884de53bf922..d4321263269ae46dd6a1b72e39c57c1c07cb06ba 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
@@ -2,6 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2021-2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
s6-ipcserver-socketbinder -a 0700 -B env/virtiofsd.sock
importas -i VM VM
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
index c3bfafe02ec6d2ee418fe56b033bbc4c7a73b186..ad705da8460a86667b47d40e61fee167d240a9c6 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
@@ -1,8 +1,10 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
+cgroup-setup -- $1
+
s6-ipcserver-socketbinder -a 0700 -b 1 env/crosvm.sock
multisubstitute {
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
index 909957b817136149dd01c3b98847238ecda8032d..54860d67f284e7cabbfc955f44ef819ff3f593bc 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
@@ -2,6 +2,8 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
+
importas -i VM VM
export DBUS_SESSION_BUS_ADDRESS unix:path=/run/portal-bus/${VM}
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
index bf90e99eef4106a9ad17dfba310b6edb3eaa64a3..13f2cec03a087be4602c14ed78749cce1f6a26a1 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
@@ -2,6 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
export VM $1
s6-svscan -d3 data/service
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/run
index 78f794202bf174f3c036f3e20755ac087a988277..ed0808b9e45b077023f237a0f9c0e5c830015ac2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/run
@@ -1,5 +1,6 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
s6-svscan -d3 instance
diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/finish b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/run b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
old mode 100644
new mode 100755
index aec6444e951503eae988e666b77fda8f2ae33d72..c37909af23fa611a75442108100b4fd9db1e22f8
--- a/host/rootfs/image/etc/s6-rc/systemd-udevd/run
+++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
@@ -1,7 +1,8 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
-s6-setlock /run/sd-notify-wrapper/systemd-udevd.lock
+
+cgroup-setup -- $1
s6-ipcserver-socketbinder -b0 -m -a 0600 /run/sd-notify-wrapper/systemd-udevd.sock
background -d {
fdmove 1 3
diff --git a/host/rootfs/image/etc/s6-rc/weston/finish b/host/rootfs/image/etc/s6-rc/weston/finish
new file mode 100755
index 0000000000000000000000000000000000000000..0c16cc8060762eac3a91a49b54048407dfbe25c5
--- /dev/null
+++ b/host/rootfs/image/etc/s6-rc/weston/finish
@@ -0,0 +1,5 @@
+#!/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-setup -- $3
diff --git a/host/rootfs/image/etc/s6-rc/weston/run b/host/rootfs/image/etc/s6-rc/weston/run
old mode 100644
new mode 100755
index fd59586c719391deb546c29578341e16a61ed4ce..6794a3ed729c9b99336cb64df95bbb3eba644582
--- a/host/rootfs/image/etc/s6-rc/weston/run
+++ b/host/rootfs/image/etc/s6-rc/weston/run
@@ -1,7 +1,8 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2021, 2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup -- $1
importas -Siu WAYLAND_DISPLAY
piperw 4 3
diff --git a/pkgs/default.nix b/pkgs/default.nix
index 4dbdfee8ba330f5ba5c9fedee2d1bb8a44af5722..22c0d782bf4b5467fe77d5a2d71fb8df3af3e756 100644
--- a/pkgs/default.nix
+++ b/pkgs/default.nix
@@ -51,6 +51,7 @@ let
driverSupport = true;
};
spectrum-router = self.callSpectrumPackage ../tools/router {};
+ spectrum-cgroup-setup = self.callSpectrumPackage ../tools/cgroup-setup {};
xdg-desktop-portal-spectrum-host =
self.callSpectrumPackage ../tools/xdg-desktop-portal-spectrum-host {};
diff --git a/tools/cgroup-setup/Cargo.lock b/tools/cgroup-setup/Cargo.lock
new file mode 100644
index 0000000000000000000000000000000000000000..fe967b3aa02c296c87b6b36ac59253dbe0a32de9
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.lock
@@ -0,0 +1,67 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 4
+
+[[package]]
+name = "bitflags"
+version = "2.11.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3"
+
+[[package]]
+name = "cgroup-setup"
+version = "0.0.0"
+dependencies = [
+ "libc",
+ "rustix",
+]
+
+[[package]]
+name = "errno"
+version = "0.3.14"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
+dependencies = [
+ "libc",
+ "windows-sys",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.186"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
+
+[[package]]
+name = "linux-raw-sys"
+version = "0.12.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
+
+[[package]]
+name = "rustix"
+version = "1.1.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
+dependencies = [
+ "bitflags",
+ "errno",
+ "libc",
+ "linux-raw-sys",
+ "windows-sys",
+]
+
+[[package]]
+name = "windows-link"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
+
+[[package]]
+name = "windows-sys"
+version = "0.61.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
+dependencies = [
+ "windows-link",
+]
diff --git a/tools/cgroup-setup/Cargo.lock.license b/tools/cgroup-setup/Cargo.lock.license
new file mode 100644
index 0000000000000000000000000000000000000000..aa108acd23886d8302eaf7babff90d1b08ae19fb
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.lock.license
@@ -0,0 +1,2 @@
+SPDX-License-Identifier: EUPL-1.2+
+SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
diff --git a/tools/cgroup-setup/Cargo.toml b/tools/cgroup-setup/Cargo.toml
new file mode 100644
index 0000000000000000000000000000000000000000..9cce0be706151dfd5264a81afb3ee9a6c851367c
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.toml
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: CC0-1.0
+# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+[package]
+name = "cgroup-setup"
+edition = "2024"
+
+[dependencies]
+libc = "0.2.177"
+rustix = { version = "1.1.2", features = ["fs"] }
+
+[profile.release]
+split-debuginfo = "symbols"
+strip = "symbols"
+lto = true
+panic = "abort"
diff --git a/tools/cgroup-setup/default.nix b/tools/cgroup-setup/default.nix
new file mode 100644
index 0000000000000000000000000000000000000000..9e716b8f270828b733f39961e5ca37a290a872b4
--- /dev/null
+++ b/tools/cgroup-setup/default.nix
@@ -0,0 +1,18 @@
+# SPDX-FileCopyrightText: 2024 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
+# SPDX-License-Identifier: MIT
+
+import ../../lib/call-package.nix (
+{ src, lib, rustPlatform }:
+
+rustPlatform.buildRustPackage {
+ name = "spectrum-cgroup-setup";
+
+ src = lib.fileset.toSource {
+ root = ../..;
+ fileset = lib.fileset.intersection src ./.;
+ };
+ sourceRoot = "source/tools/cgroup-setup";
+
+ cargoLock.lockFile = ./Cargo.lock;
+}) (_: {})
diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs
new file mode 100644
index 0000000000000000000000000000000000000000..cd77becc0b40bfdcd983e53b63b7c21e4308d5fc
--- /dev/null
+++ b/tools/cgroup-setup/src/cgroup.rs
@@ -0,0 +1,181 @@
+// SPDX-License-Identifier: EUPL-1.2+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+use std::fs::File;
+use std::io::{Read, Seek};
+use std::os::fd::{AsFd, OwnedFd};
+use std::os::fd::{AsRawFd, BorrowedFd};
+
+use std::path::{Path, PathBuf};
+
+use rustix::fs::AtFlags;
+use rustix::path::Arg;
+use rustix::{
+ fs::{Mode, OFlags, ResolveFlags},
+ io::Errno,
+};
+
+pub(crate) struct LeafCgroup {
+ path: PathBuf,
+ fd: OwnedFd,
+}
+
+enum Access {
+ Read,
+ Write,
+}
+
+impl LeafCgroup {
+ fn open_subtree(&self, path: &std::path::Path, access: Access) -> Result<OwnedFd, Errno> {
+ rustix::fs::openat2(
+ self.fd.as_fd(),
+ path,
+ OFlags::NOATIME
+ | OFlags::CLOEXEC
+ | OFlags::NOFOLLOW
+ | match access {
+ Access::Write => OFlags::WRONLY,
+ Access::Read => OFlags::RDONLY,
+ },
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )
+ }
+
+ pub fn kill_processes(&self) -> std::io::Result<()> {
+ let kill_fd = self.open_subtree(std::path::Path::new("cgroup.kill"), Access::Write)?;
+ let wait_file = self.open_subtree(std::path::Path::new("cgroup.events"), Access::Read)?;
+ let poll_fd = wait_file.as_raw_fd();
+ let mut wait_fd = File::from(wait_file);
+ assert_eq!(rustix::io::write(kill_fd.as_fd(), b"1")?, 1, "kernel bug");
+ let mut fds = libc::pollfd {
+ fd: poll_fd,
+ events: libc::POLLIN | libc::POLLPRI | libc::POLLRDHUP,
+ revents: 0,
+ };
+ // SAFETY: FFI call, valid arguments, fds contains 1 element
+ let mut v = vec![];
+ 'a: loop {
+ v.clear();
+ if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
+ panic!("poll failed");
+ }
+ wait_fd
+ .seek(std::io::SeekFrom::Start(0))
+ .expect("Seek on control group file should succeed");
+ wait_fd
+ .read_to_end(&mut v)
+ .expect("reading from control group should work");
+ for substr in v.split(|&c| c == b'\n') {
+ if substr == b"populated 0" {
+ break 'a;
+ }
+ }
+ }
+ Ok(())
+ }
+}
+
+impl AsFd for LeafCgroup {
+ fn as_fd(&self) -> std::os::fd::BorrowedFd<'_> {
+ self.fd.as_fd()
+ }
+}
+
+impl LeafCgroup {
+ pub(crate) fn open_cgroup_at(&self, p: &Path) -> Result<Self, Errno> {
+ let fd = rustix::fs::openat2(
+ self.fd.as_fd(),
+ p,
+ OFlags::NOATIME
+ | OFlags::CLOEXEC
+ | OFlags::NOFOLLOW
+ | OFlags::RDONLY
+ | OFlags::DIRECTORY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )?;
+ Ok(Self {
+ fd,
+ path: self.path.join(p),
+ })
+ }
+ pub(crate) fn open_cgroup(fd: BorrowedFd, p: &Path) -> Result<Self, Errno> {
+ let fd = rustix::fs::openat2(
+ fd,
+ p,
+ OFlags::NOATIME
+ | OFlags::CLOEXEC
+ | OFlags::NOFOLLOW
+ | OFlags::RDONLY
+ | OFlags::DIRECTORY,
+ Mode::empty(),
+ //ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ ResolveFlags::NO_SYMLINKS,
+ )?;
+ Ok(Self {
+ fd,
+ path: p.to_owned(),
+ })
+ }
+ pub(crate) fn make_child(&self, p: &Path) -> Result<(), Errno> {
+ rustix::fs::mkdirat(
+ self.fd.as_fd(),
+ p,
+ Mode::RUSR
+ | Mode::WUSR
+ | Mode::XUSR
+ | Mode::RGRP
+ | Mode::XGRP
+ | Mode::ROTH
+ | Mode::XOTH,
+ )
+ }
+
+ pub(crate) fn delete_child(&self, path: &Path) -> Result<(), Errno> {
+ remove_all(100, self.as_fd(), &path.as_cow_c_str().unwrap())
+ }
+}
+
+fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> {
+ if remaining_depth < 1 {
+ panic!("control groups too deeply nested");
+ }
+ let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating");
+ while let Some(element) = d.next() {
+ let element = element.expect("Iterating through a cgroup directory failed?");
+ if element.file_type() != rustix::fs::FileType::Directory {
+ continue;
+ }
+
+ let remaining_depth = remaining_depth - 1;
+ let d: &rustix::fs::Dir = &d;
+ let dirfd = d.fd().unwrap();
+ let path = element.file_name();
+ remove_all(remaining_depth, dirfd, path)?;
+ }
+ Ok(())
+}
+
+fn remove_all(
+ remaining_depth: usize,
+ dirfd: BorrowedFd<'_>,
+ path: &std::ffi::CStr,
+) -> Result<(), Errno> {
+ if path == c"." || path == c".." {
+ return Ok(());
+ }
+ if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() {
+ return Ok(());
+ }
+ let fd = rustix::fs::openat2(
+ dirfd,
+ path,
+ OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )?;
+ remove_recursively(fd, remaining_depth)?;
+ rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?;
+ Ok(())
+}
diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
new file mode 100644
index 0000000000000000000000000000000000000000..358703a73fcbdc38794312fe3987e733f37c2df0
--- /dev/null
+++ b/tools/cgroup-setup/src/main.rs
@@ -0,0 +1,196 @@
+// SPDX-License-Identifier: EUPL-1.2+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+use std::{
+ ffi::{OsStr, OsString},
+ fs::File,
+ io::Write as _,
+ os::unix::prelude::*,
+ path::{Component, Path, PathBuf},
+};
+
+use rustix::{
+ fs::{FlockOperation, Mode, OFlags, ResolveFlags},
+ io::Errno,
+};
+
+mod cgroup;
+
+macro_rules! fail {
+ ($($arg:tt)*) => {{
+ eprintln!($($arg)*);
+ std::process::exit(1)}
+ };
+}
+
+fn main() {
+ let mut args = std::env::args_os();
+ let Some(prog_name) = args.next() else {
+ fail!("No command line arguments (argv[0] is NULL)");
+ };
+ let mut cgroup_relative_path = args.next();
+ if cgroup_relative_path.as_deref() == Some(OsStr::from_bytes(b"--")) {
+ cgroup_relative_path = args.next();
+ }
+
+ let Some(cgroup_relative_path) = cgroup_relative_path else {
+ fail!("{prog_name:?}: Have no positional arguments, expect at least 1")
+ };
+ let mut cgroup_relative_path = cgroup_relative_path.into_vec();
+
+ // slow but we do not care
+ if cgroup_relative_path.len() > 247 {
+ fail!("{prog_name:?}: Cgroup name {cgroup_relative_path:?} too long");
+ }
+ if cgroup_relative_path.is_empty() {
+ fail!("{prog_name:?}: Cgroup name {cgroup_relative_path:?} empty");
+ }
+ if cgroup_relative_path[0] == b'-' {
+ fail!("{prog_name:?}: Cgroup name {cgroup_relative_path:?} starts with '-'");
+ }
+ for &i in &cgroup_relative_path {
+ match i {
+ b'/' => fail!(
+ "{prog_name:?}: Cgroup name {cgroup_relative_path:?} \
+contains /"
+ ),
+ b'$' => fail!(
+ "{prog_name:?}: Cgroup name {cgroup_relative_path:?} \
+contains $: did you forget an execline substitution?"
+ ),
+ b'!'..=b'~' => {}
+ _ => fail!(
+ "{prog_name:?}: Cgroup name {cgroup_relative_path:?} \
+contains space, non-ASCII character, or control character (byte {i})"
+ ),
+ }
+ }
+ cgroup_relative_path.extend_from_slice(b".service");
+ let cgroup_relative_path = Path::new(OsStr::from_bytes(&cgroup_relative_path));
+
+ let local_cgroup = std::fs::read("/proc/thread-self/cgroup")
+ .unwrap_or_else(|e| fail!("{prog_name:?}: cannot read /proc/thread-self/cgroup: {e}"));
+
+ if !local_cgroup.starts_with(b"0::/")
+ || !local_cgroup.ends_with(b"\n")
+ || local_cgroup.contains(&b'\0')
+ {
+ fail!("{prog_name:?}: Kernel bug: Bad contents of /proc/thread-self/cgroup");
+ }
+
+ let mut local_cgroup = PathBuf::from(<OsString as OsStringExt>::from_vec(
+ local_cgroup[4..local_cgroup.len() - 1].to_owned(),
+ ));
+
+ match local_cgroup.components().next_back() {
+ Some(Component::Prefix(_)) => unreachable!("does not occur on Linux"),
+ Some(Component::RootDir) | None => {}
+ Some(Component::CurDir) => unreachable!("not produced by kernel"),
+ Some(Component::ParentDir) => unreachable!("not produced by kernel"),
+ Some(Component::Normal(os_str)) => {
+ if os_str.as_bytes() == b"@inner.service" {
+ let _ = local_cgroup.pop();
+ }
+ }
+ }
+
+ if local_cgroup.as_os_str().is_empty() {
+ local_cgroup = PathBuf::from(".");
+ }
+
+ let cgroup_root = rustix::fs::openat2(
+ rustix::fs::CWD,
+ Path::new("/sys/fs/cgroup"),
+ OFlags::DIRECTORY | OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW,
+ Mode::empty(),
+ ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_SYMLINKS,
+ )
+ .unwrap_or_else(|e| fail!("{prog_name:?}: cannot open /sys/fs/cgroup: {e}"));
+
+ let cgroup = match cgroup::LeafCgroup::open_cgroup(cgroup_root.as_fd(), &local_cgroup) {
+ Ok(e) => e,
+ Err(e) => fail!(
+ "{prog_name:?}: Failed to open {}: {e:?}",
+ local_cgroup.display()
+ ),
+ };
+
+ match rustix::fs::flock(cgroup.as_fd(), FlockOperation::LockExclusive) {
+ Ok(()) => {}
+ Err(e) => fail!("{prog_name:?}: cannot lock cgroup: {e}"),
+ }
+ purge_cgroup(&prog_name, &cgroup, cgroup_relative_path);
+ let Some(program_name) = args.next() else {
+ return;
+ };
+ if let Err(e) = cgroup.make_child(cgroup_relative_path) {
+ fail!("{prog_name:?}: cannot create child cgroup: {e}")
+ }
+ let child = cgroup
+ .open_cgroup_at(cgroup_relative_path)
+ .unwrap_or_else(|e| {
+ fail!("{prog_name:?}: cannot create child cgroup {cgroup_relative_path:?}: {e}")
+ });
+ child
+ .make_child(Path::new("@inner.service"))
+ .unwrap_or_else(|e| {
+ fail!("{prog_name:?}: cannot create child cgroup {cgroup_relative_path:?}/@inner.service: {e}")
+ });
+ let child = child
+ .open_cgroup_at(Path::new("@inner.service"))
+ .unwrap_or_else(|e| {
+ fail!("{prog_name:?}: cannot open child cgroup {cgroup_relative_path:?}/@inner.service: {e}")
+ });
+
+ // SAFETY: safe FFI call
+ let pid = unsafe { libc::getpid() };
+ write_cgroup_value(&prog_name, &child, "cgroup.procs", &pid.to_string());
+
+ let e = std::process::Command::new(&program_name).args(args).exec();
+ fail!(
+ "{prog_name:?}: cannot spawn child {:?}: {}",
+ program_name,
+ e
+ );
+}
+
+fn write_cgroup_value(prog_name: &OsStr, child: &cgroup::LeafCgroup, name: &str, value: &str) {
+ let path = Path::new(name);
+ let fd = rustix::fs::openat2(
+ child.as_fd(),
+ path,
+ OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::WRONLY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )
+ .unwrap_or_else(|e| fail!("{prog_name:?}: cannot open {:?}: {}", path, e));
+ let () = File::from(fd)
+ .write_all(value.as_bytes())
+ .unwrap_or_else(|e| {
+ fail!(
+ "{prog_name:?}: Cannot write {:?} to {:?}: {}",
+ name,
+ path,
+ e
+ )
+ });
+}
+
+fn purge_cgroup(prog_name: &OsStr, cgroup: &cgroup::LeafCgroup, cgroup_relative_path: &Path) {
+ let child = match cgroup.open_cgroup_at(cgroup_relative_path) {
+ Ok(child_cgroup) => child_cgroup,
+ Err(Errno::NOENT) => return,
+ Err(other) => {
+ fail!("{prog_name:?}: cannot open child cgroup {cgroup_relative_path:?}: {other}")
+ }
+ };
+ child.kill_processes().unwrap_or_else(|e| {
+ fail!("{prog_name:?}: cannot kill programs in {cgroup_relative_path:?}: {e}")
+ });
+
+ cgroup
+ .delete_child(cgroup_relative_path)
+ .unwrap_or_else(|e| {
+ fail!("{prog_name:?}: delete child cgroup {cgroup_relative_path:?}: {e}")
+ });
+}
---
base-commit: 36ea9d85403a29f1e700f534020cc4592d78ee50
change-id: 20260528-cgroups-d609e270e649
--
Sincerely,
Demi Marie Obenour (she/her/hers)
^ permalink raw reply related [flat|nested] 132+ messages in thread
* Re: [PATCH v2] Set up control groups for most services
2026-06-20 17:27 ` [PATCH v2] " Demi Marie Obenour
@ 2026-06-24 12:13 ` Alyssa Ross
2026-06-24 12:36 ` Alyssa Ross
2026-06-25 2:03 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
1 sibling, 2 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-06-24 12:13 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 21991 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> The cgroups are handled by a Rust tool. The name of the cgroup is
> autogenerated from the service name.
>
> Using cgroups for process control is not implemented yet.
>
Perhaps you could include some more details of how the cgroup hierarchy
is supposed to be set up and why, either in the patch body or in
documentation?
As I understood it, the point of using cgroups was that we could bundle
all services for a VM, including the VMM, into a single cgroup, but I
don't see that here, just a pure translation of the service hierarchy
(which the VMM might not even be part of). Are per-VM cgroups coming
later?
> diff --git a/host/rootfs/image/etc/fstab b/host/rootfs/image/etc/fstab
> index 18bb5e45abafeaf43871306ce8233239e5c07f76..d92364d83f4d26f766fcd5b494614c06a630d2fe 100644
> --- a/host/rootfs/image/etc/fstab
> +++ b/host/rootfs/image/etc/fstab
> @@ -6,3 +6,4 @@ tmpfs /dev/shm tmpfs nosuid,nodev 0 0
> tmpfs /media tmpfs nosuid,nodev,noexec,nosymfollow,mode=755 0 0
> sysfs /sys sysfs nosuid,nodev,noexec 0 0
> tmpfs /tmp tmpfs nosuid,nodev 0 0
> +cgroup2 /sys/fs/cgroup cgroup2 nosuid,nodev,noexec,nosymfollow 0 0
Alignment is off here. The rest of the file uses tabs at 8 characters.
(I know, I know, but it's traditional.)
> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
> index bf66ab0878fc6d8e77ca71a4a89e50c139a6bb11..5c9988c994ed9b942329055673b332d0aa918957 100755
> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
> @@ -1,7 +1,8 @@
> -#!/bin/execlineb -WP
> +#!/bin/execlineb -WS1
> # SPDX-License-Identifier: EUPL-1.2+
> # SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
>
> +cgroup-setup -- $1
> piperw 3 4
> background {
> fdclose 3
What benefit does this cgroup provide?
> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
> index 78f794202bf174f3c036f3e20755ac087a988277..ed0808b9e45b077023f237a0f9c0e5c830015ac2 100755
> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
> @@ -1,5 +1,6 @@
> -#!/bin/execlineb -WP
> +#!/bin/execlineb -WS1
> # SPDX-License-Identifier: EUPL-1.2+
> # SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
>
> +cgroup-setup -- $1
> s6-svscan -d3 instance
We're putting supervisors of instances services in a cgroup? Can you
explain to me why that's useful?
> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
> index f0507dc92aadf25c092f0719eb767542ac0c4451..89676ac91d3e22db7726fc869a065137b031d0be 100755
> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
> @@ -1,7 +1,8 @@
> -#!/bin/execlineb -WP
> +#!/bin/execlineb -WS1
> # SPDX-License-Identifier: EUPL-1.2+
> # SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
>
> +cgroup-setup -- $1
> importas -i VM VM
>
> if {
This seems to get stuck in udevadm wait?
> diff --git a/tools/cgroup-setup/Cargo.toml b/tools/cgroup-setup/Cargo.toml
> new file mode 100644
> index 0000000000000000000000000000000000000000..9cce0be706151dfd5264a81afb3ee9a6c851367c
> --- /dev/null
> +++ b/tools/cgroup-setup/Cargo.toml
> @@ -0,0 +1,17 @@
> +# SPDX-License-Identifier: CC0-1.0
> +# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +
> +[package]
> +name = "cgroup-setup"
> +edition = "2024"
> +
> +[dependencies]
> +libc = "0.2.177"
> +rustix = { version = "1.1.2", features = ["fs"] }
> +
> +[profile.release]
> +split-debuginfo = "symbols"
> +strip = "symbols"
> +lto = true
> +panic = "abort"
What's with all this stuff? Why do we set it only for this program, and
not for any others? Please don't introduce divergence in build settings
between programs without a good reason.
> diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs
> new file mode 100644
> index 0000000000000000000000000000000000000000..cd77becc0b40bfdcd983e53b63b7c21e4308d5fc
> --- /dev/null
> +++ b/tools/cgroup-setup/src/cgroup.rs
> @@ -0,0 +1,181 @@
> +// SPDX-License-Identifier: EUPL-1.2+
> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +
> +use std::fs::File;
> +use std::io::{Read, Seek};
> +use std::os::fd::{AsFd, OwnedFd};
> +use std::os::fd::{AsRawFd, BorrowedFd};
> +
> +use std::path::{Path, PathBuf};
> +
> +use rustix::fs::AtFlags;
> +use rustix::path::Arg;
> +use rustix::{
> + fs::{Mode, OFlags, ResolveFlags},
> + io::Errno,
> +};
> +
> +pub(crate) struct LeafCgroup {
> + path: PathBuf,
> + fd: OwnedFd,
root_fd would be a clearer name, if I'm understanding correctly.
> +}
> +
> +enum Access {
> + Read,
> + Write,
> +}
Given this isn't public, it doesn't seem to add any value over passing
around OFlags::RDONLY and OFlags::WRONLY directly.
> +
> +impl LeafCgroup {
> + fn open_subtree(&self, path: &std::path::Path, access: Access) -> Result<OwnedFd, Errno> {
> + rustix::fs::openat2(
> + self.fd.as_fd(),
> + path,
> + OFlags::NOATIME
> + | OFlags::CLOEXEC
> + | OFlags::NOFOLLOW
> + | match access {
> + Access::Write => OFlags::WRONLY,
> + Access::Read => OFlags::RDONLY,
> + },
> + Mode::empty(),
> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
> + )
> + }
> +
> + pub fn kill_processes(&self) -> std::io::Result<()> {
> + let kill_fd = self.open_subtree(std::path::Path::new("cgroup.kill"), Access::Write)?;
> + let wait_file = self.open_subtree(std::path::Path::new("cgroup.events"), Access::Read)?;
> + let poll_fd = wait_file.as_raw_fd();
> + let mut wait_fd = File::from(wait_file);
> + assert_eq!(rustix::io::write(kill_fd.as_fd(), b"1")?, 1, "kernel bug");
> + let mut fds = libc::pollfd {
> + fd: poll_fd,
> + events: libc::POLLIN | libc::POLLPRI | libc::POLLRDHUP,
> + revents: 0,
> + };
> + // SAFETY: FFI call, valid arguments, fds contains 1 element
> + let mut v = vec![];
> + 'a: loop {
> + v.clear();
> + if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
> + panic!("poll failed");
> + }
Shouldn't we start polling before writing? Otherwise we might miss the
notification, if it comes in between the write and the poll.
> + wait_fd
> + .seek(std::io::SeekFrom::Start(0))
> + .expect("Seek on control group file should succeed");
> + wait_fd
> + .read_to_end(&mut v)
> + .expect("reading from control group should work");
> + for substr in v.split(|&c| c == b'\n') {
> + if substr == b"populated 0" {
Could do this in one line, and avoid the labelled break:
if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
(I think "line" is a clearer name than "substr".)
> + break 'a;
> + }
> + }
> + }
> + Ok(())
> + }
> +}
> +
> +impl AsFd for LeafCgroup {
> + fn as_fd(&self) -> std::os::fd::BorrowedFd<'_> {
> + self.fd.as_fd()
> + }
> +}
> +
> +impl LeafCgroup {
> + pub(crate) fn open_cgroup_at(&self, p: &Path) -> Result<Self, Errno> {
> + let fd = rustix::fs::openat2(
> + self.fd.as_fd(),
> + p,
> + OFlags::NOATIME
> + | OFlags::CLOEXEC
> + | OFlags::NOFOLLOW
> + | OFlags::RDONLY
> + | OFlags::DIRECTORY,
> + Mode::empty(),
> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
> + )?;
> + Ok(Self {
> + fd,
> + path: self.path.join(p),
> + })
> + }
> + pub(crate) fn open_cgroup(fd: BorrowedFd, p: &Path) -> Result<Self, Errno> {
> + let fd = rustix::fs::openat2(
> + fd,
> + p,
> + OFlags::NOATIME
> + | OFlags::CLOEXEC
> + | OFlags::NOFOLLOW
> + | OFlags::RDONLY
> + | OFlags::DIRECTORY,
> + Mode::empty(),
> + //ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
Stray comment.
> + ResolveFlags::NO_SYMLINKS,
> + )?;
> + Ok(Self {
> + fd,
> + path: p.to_owned(),
> + })
> + }
> + pub(crate) fn make_child(&self, p: &Path) -> Result<(), Errno> {
> + rustix::fs::mkdirat(
> + self.fd.as_fd(),
> + p,
> + Mode::RUSR
> + | Mode::WUSR
> + | Mode::XUSR
> + | Mode::RGRP
> + | Mode::XGRP
> + | Mode::ROTH
> + | Mode::XOTH,
> + )
> + }
> +
> + pub(crate) fn delete_child(&self, path: &Path) -> Result<(), Errno> {
> + remove_all(100, self.as_fd(), &path.as_cow_c_str().unwrap())
> + }
> +}
> +
> +fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> {
> + if remaining_depth < 1 {
> + panic!("control groups too deeply nested");
> + }
> + let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating");
> + while let Some(element) = d.next() {
> + let element = element.expect("Iterating through a cgroup directory failed?");
> + if element.file_type() != rustix::fs::FileType::Directory {
> + continue;
> + }
> +
> + let remaining_depth = remaining_depth - 1;
> + let d: &rustix::fs::Dir = &d;
> + let dirfd = d.fd().unwrap();
> + let path = element.file_name();
> + remove_all(remaining_depth, dirfd, path)?;
> + }
> + Ok(())
> +}
> +
> +fn remove_all(
> + remaining_depth: usize,
> + dirfd: BorrowedFd<'_>,
> + path: &std::ffi::CStr,
> +) -> Result<(), Errno> {
> + if path == c"." || path == c".." {
> + return Ok(());
> + }
> + if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() {
> + return Ok(());
> + }
> + let fd = rustix::fs::openat2(
> + dirfd,
> + path,
> + OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY,
> + Mode::empty(),
> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
> + )?;
> + remove_recursively(fd, remaining_depth)?;
> + rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?;
> + Ok(())
> +}
> diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
> new file mode 100644
> index 0000000000000000000000000000000000000000..358703a73fcbdc38794312fe3987e733f37c2df0
> --- /dev/null
> +++ b/tools/cgroup-setup/src/main.rs
> @@ -0,0 +1,196 @@
> +// SPDX-License-Identifier: EUPL-1.2+
> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +
> +use std::{
> + ffi::{OsStr, OsString},
> + fs::File,
> + io::Write as _,
> + os::unix::prelude::*,
> + path::{Component, Path, PathBuf},
> +};
> +
> +use rustix::{
> + fs::{FlockOperation, Mode, OFlags, ResolveFlags},
> + io::Errno,
> +};
> +
> +mod cgroup;
> +
> +macro_rules! fail {
> + ($($arg:tt)*) => {{
> + eprintln!($($arg)*);
> + std::process::exit(1)}
> + };
> +}
I'd find the error handling Rustier if we did error returns with the ?
operator, like how mount-flatpak does it. The error type can just be
String.
> +
> +fn main() {
> + let mut args = std::env::args_os();
> + let Some(prog_name) = args.next() else {
> + fail!("No command line arguments (argv[0] is NULL)");
> + };
> + let mut cgroup_relative_path = args.next();
> + if cgroup_relative_path.as_deref() == Some(OsStr::from_bytes(b"--")) {
> + cgroup_relative_path = args.next();
> + }
We could just not support -- and simplify invocation, right?
> +
> + let Some(cgroup_relative_path) = cgroup_relative_path else {
> + fail!("{prog_name:?}: Have no positional arguments, expect at least 1")
> + };
> + let mut cgroup_relative_path = cgroup_relative_path.into_vec();
> +
> + // slow but we do not care
Slow?
> + if cgroup_relative_path.len() > 247 {
This magic number could perhaps use a name.
> + fail!("{prog_name:?}: Cgroup name {cgroup_relative_path:?} too long");
> + }
> + if cgroup_relative_path.is_empty() {
> + fail!("{prog_name:?}: Cgroup name {cgroup_relative_path:?} empty");
> + }
> + if cgroup_relative_path[0] == b'-' {
> + fail!("{prog_name:?}: Cgroup name {cgroup_relative_path:?} starts with '-'");
> + }
Is that a problem?
> + for &i in &cgroup_relative_path {
> + match i {
> + b'/' => fail!(
> + "{prog_name:?}: Cgroup name {cgroup_relative_path:?} \
> +contains /"
> + ),
> + b'$' => fail!(
> + "{prog_name:?}: Cgroup name {cgroup_relative_path:?} \
> +contains $: did you forget an execline substitution?"
> + ),
> + b'!'..=b'~' => {}
> + _ => fail!(
> + "{prog_name:?}: Cgroup name {cgroup_relative_path:?} \
> +contains space, non-ASCII character, or control character (byte {i})"
> + ),
Surely every kernel interface we're going to use is 8-bit clean. If
this is about log viewing again, I really don't like it. Even if we're
careful about it in first-party code (which itself is a big ask), we
can't expect it of every other package that might log something. People
viewing logs have to use robust tools to do so regardless, or we have to
mitigate this somewhere centrally, e.g. in s6 log. Doing this piecemeal
does not meaningfully mitigate the problem, and arguably creates a false
sense of security.
> + }
> + }
> + cgroup_relative_path.extend_from_slice(b".service");
What do we need the suffix for?
> + let cgroup_relative_path = Path::new(OsStr::from_bytes(&cgroup_relative_path));
> +
> + let local_cgroup = std::fs::read("/proc/thread-self/cgroup")
Ooc, why /proc/thread-self instead of /proc/self?
> + .unwrap_or_else(|e| fail!("{prog_name:?}: cannot read /proc/thread-self/cgroup: {e}"));
> +
> + if !local_cgroup.starts_with(b"0::/")
> + || !local_cgroup.ends_with(b"\n")
> + || local_cgroup.contains(&b'\0')
> + {
> + fail!("{prog_name:?}: Kernel bug: Bad contents of /proc/thread-self/cgroup");
> + }
We are not a kernel fuzzer. We can't reasonably write programs that
have to anticipate kernel contract violations.
> +
> + let mut local_cgroup = PathBuf::from(<OsString as OsStringExt>::from_vec(
> + local_cgroup[4..local_cgroup.len() - 1].to_owned(),
… but maybe we could make this clearer, and still satisfy your instincts
to validate, by using slice::strip_prefix and slice::strip_suffix,
unwrapping the results?
> + ));
> +
> + match local_cgroup.components().next_back() {
> + Some(Component::Prefix(_)) => unreachable!("does not occur on Linux"),
> + Some(Component::RootDir) | None => {}
> + Some(Component::CurDir) => unreachable!("not produced by kernel"),
> + Some(Component::ParentDir) => unreachable!("not produced by kernel"),
> + Some(Component::Normal(os_str)) => {
> + if os_str.as_bytes() == b"@inner.service" {
> + let _ = local_cgroup.pop();
> + }
> + }
> + }
Could we not simplify this with local_cgroup.file_name()? If it's None,
it's the root directory; otherwise it's the equivalent of normal.
> +
> + if local_cgroup.as_os_str().is_empty() {
> + local_cgroup = PathBuf::from(".");
> + }
Path::is_empty is stable in 1.98. Could we mention that in a TODO/FIXME
comment so we can use it when available?
> +
> + let cgroup_root = rustix::fs::openat2(
> + rustix::fs::CWD,
> + Path::new("/sys/fs/cgroup"),
> + OFlags::DIRECTORY | OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW,
> + Mode::empty(),
> + ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_SYMLINKS,
> + )
> + .unwrap_or_else(|e| fail!("{prog_name:?}: cannot open /sys/fs/cgroup: {e}"));
Why do we need to use openat2 for this, but not for reading
/proc/thread-self/cgroup? I'm a bit surprised to see openat2 here at
all, since only root could manipulate these paths, in which case they
have no need to use this program as a confused deputy, right?
If openat2 is to stay, isn't OFlags::NOFOLLOW redundant with
ResolveFlags::NO_SYMLINKS?
> +
> + let cgroup = match cgroup::LeafCgroup::open_cgroup(cgroup_root.as_fd(), &local_cgroup) {
> + Ok(e) => e,
> + Err(e) => fail!(
> + "{prog_name:?}: Failed to open {}: {e:?}",
> + local_cgroup.display()
> + ),
> + };
> +
> + match rustix::fs::flock(cgroup.as_fd(), FlockOperation::LockExclusive) {
> + Ok(()) => {}
> + Err(e) => fail!("{prog_name:?}: cannot lock cgroup: {e}"),
> + }
Would std::fs::File::lock not be a bit Rustier?
> + purge_cgroup(&prog_name, &cgroup, cgroup_relative_path);
I find it a little weird that we run this program in both service
startup and finish, when there are really two completely separable parts
to it, right? Why is purging not a separate program that only runs in
finish?
> + let Some(program_name) = args.next() else {
> + return;
> + };
This could be moved closer to where it's used.
> + if let Err(e) = cgroup.make_child(cgroup_relative_path) {
> + fail!("{prog_name:?}: cannot create child cgroup: {e}")
> + }
> + let child = cgroup
> + .open_cgroup_at(cgroup_relative_path)
> + .unwrap_or_else(|e| {
> + fail!("{prog_name:?}: cannot create child cgroup {cgroup_relative_path:?}: {e}")
> + });
> + child
> + .make_child(Path::new("@inner.service"))
> + .unwrap_or_else(|e| {
> + fail!("{prog_name:?}: cannot create child cgroup {cgroup_relative_path:?}/@inner.service: {e}")
> + });
> + let child = child
> + .open_cgroup_at(Path::new("@inner.service"))
> + .unwrap_or_else(|e| {
> + fail!("{prog_name:?}: cannot open child cgroup {cgroup_relative_path:?}/@inner.service: {e}")
> + });
This would really benefit from having been written up, so I don't have
to guess what it's for. We only really need it for supervisors, right?
> +
> + // SAFETY: safe FFI call
> + let pid = unsafe { libc::getpid() };
Not std::process::id()?
> + write_cgroup_value(&prog_name, &child, "cgroup.procs", &pid.to_string());
> +
> + let e = std::process::Command::new(&program_name).args(args).exec();
> + fail!(
> + "{prog_name:?}: cannot spawn child {:?}: {}",
> + program_name,
> + e
> + );
> +}
> +
> +fn write_cgroup_value(prog_name: &OsStr, child: &cgroup::LeafCgroup, name: &str, value: &str) {
Perhaps it would be nicer for this to be a method on LeafCgroup?
> + let path = Path::new(name);
> + let fd = rustix::fs::openat2(
> + child.as_fd(),
> + path,
> + OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::WRONLY,
> + Mode::empty(),
> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
> + )
> + .unwrap_or_else(|e| fail!("{prog_name:?}: cannot open {:?}: {}", path, e));
> + let () = File::from(fd)
> + .write_all(value.as_bytes())
> + .unwrap_or_else(|e| {
> + fail!(
> + "{prog_name:?}: Cannot write {:?} to {:?}: {}",
> + name,
> + path,
> + e
> + )
> + });
Similarly to above, if we don't need openat2, this could be done more
nicely as std::fs::write.
> +}
> +
> +fn purge_cgroup(prog_name: &OsStr, cgroup: &cgroup::LeafCgroup, cgroup_relative_path: &Path) {
> + let child = match cgroup.open_cgroup_at(cgroup_relative_path) {
> + Ok(child_cgroup) => child_cgroup,
> + Err(Errno::NOENT) => return,
> + Err(other) => {
> + fail!("{prog_name:?}: cannot open child cgroup {cgroup_relative_path:?}: {other}")
> + }
> + };
> + child.kill_processes().unwrap_or_else(|e| {
> + fail!("{prog_name:?}: cannot kill programs in {cgroup_relative_path:?}: {e}")
> + });
> +
> + cgroup
> + .delete_child(cgroup_relative_path)
> + .unwrap_or_else(|e| {
> + fail!("{prog_name:?}: delete child cgroup {cgroup_relative_path:?}: {e}")
> + });
> +}
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v2] Set up control groups for most services
2026-06-24 12:13 ` Alyssa Ross
@ 2026-06-24 12:36 ` Alyssa Ross
2026-06-25 2:03 ` Demi Marie Obenour
1 sibling, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-06-24 12:36 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 996 bytes --]
Alyssa Ross <hi@alyssa.is> writes:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
>> index f0507dc92aadf25c092f0719eb767542ac0c4451..89676ac91d3e22db7726fc869a065137b031d0be 100755
>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
>> @@ -1,7 +1,8 @@
>> -#!/bin/execlineb -WP
>> +#!/bin/execlineb -WS1
>> # SPDX-License-Identifier: EUPL-1.2+
>> # SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
>>
>> +cgroup-setup -- $1
>> importas -i VM VM
>>
>> if {
>
> This seems to get stuck in udevadm wait?
Actually I see the same without this patch applied, so ignore this one.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v2] Set up control groups for most services
2026-06-24 12:13 ` Alyssa Ross
2026-06-24 12:36 ` Alyssa Ross
@ 2026-06-25 2:03 ` Demi Marie Obenour
2026-06-25 3:03 ` Demi Marie Obenour
2026-06-25 9:49 ` Alyssa Ross
1 sibling, 2 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-06-25 2:03 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1.1: Type: text/plain, Size: 25750 bytes --]
On 6/24/26 08:13, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> The cgroups are handled by a Rust tool. The name of the cgroup is
>> autogenerated from the service name.
>>
>> Using cgroups for process control is not implemented yet.
>>
>
> Perhaps you could include some more details of how the cgroup hierarchy
> is supposed to be set up and why, either in the patch body or in
> documentation?
Will fix in v3.
> As I understood it, the point of using cgroups was that we could bundle
> all services for a VM, including the VMM, into a single cgroup, but I
> don't see that here, just a pure translation of the service hierarchy
> (which the VMM might not even be part of). Are per-VM cgroups coming
> later?
I think it would be significantly simpler to have the VMM be
just another VM service. This would naturally put it under the
vm-services cgroup, solving this problem. Since this would mostly
involve changes to execline scripting, I think it would be better if
you wrote this code.
>> diff --git a/host/rootfs/image/etc/fstab b/host/rootfs/image/etc/fstab
>> index 18bb5e45abafeaf43871306ce8233239e5c07f76..d92364d83f4d26f766fcd5b494614c06a630d2fe 100644
>> --- a/host/rootfs/image/etc/fstab
>> +++ b/host/rootfs/image/etc/fstab
>> @@ -6,3 +6,4 @@ tmpfs /dev/shm tmpfs nosuid,nodev 0 0
>> tmpfs /media tmpfs nosuid,nodev,noexec,nosymfollow,mode=755 0 0
>> sysfs /sys sysfs nosuid,nodev,noexec 0 0
>> tmpfs /tmp tmpfs nosuid,nodev 0 0
>> +cgroup2 /sys/fs/cgroup cgroup2 nosuid,nodev,noexec,nosymfollow 0 0
>
> Alignment is off here. The rest of the file uses tabs at 8 characters.
> (I know, I know, but it's traditional.)
>
>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
>> index bf66ab0878fc6d8e77ca71a4a89e50c139a6bb11..5c9988c994ed9b942329055673b332d0aa918957 100755
>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
>> @@ -1,7 +1,8 @@
>> -#!/bin/execlineb -WP
>> +#!/bin/execlineb -WS1
>> # SPDX-License-Identifier: EUPL-1.2+
>> # SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
>>
>> +cgroup-setup -- $1
>> piperw 3 4
>> background {
>> fdclose 3
>
> What benefit does this cgroup provide?
None, I'll remove it.
The only requirement of the current code is that if a service is under
a cgroup, its parent should *also* be under a cgroup. Otherwise you
can get conflicting cgroup paths.
>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
>> index 78f794202bf174f3c036f3e20755ac087a988277..ed0808b9e45b077023f237a0f9c0e5c830015ac2 100755
>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
>> @@ -1,5 +1,6 @@
>> -#!/bin/execlineb -WP
>> +#!/bin/execlineb -WS1
>> # SPDX-License-Identifier: EUPL-1.2+
>> # SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
>>
>> +cgroup-setup -- $1
>> s6-svscan -d3 instance
>
> We're putting supervisors of instances services in a cgroup? Can you
> explain to me why that's useful?
This allows configuring limits that apply to the whole instance, and
means that the whole instance can be reliably killed. Services spawned
by the instance will be in their own sub-cgroups, so one can apply
separate limits to them so long as those limits do not exceed those
of the containing cgroup.
The hierarchy works like this:
/sys/fs/cgroup
service1.service
@inner.service # processes go here
service2.service
@inner.service # and here
service3.service
@inner.service # and here
You can apply resource limits at any level of the hierarchy. Limits in
nested cgroups can be smaller but not larger. Terminating processes
in a cgroup with cgroup.kill also terminates all processes in child
cgroups. After terminating the processes in a cgroup, cgroup-setup
will delete everything in the hierarchy recursively.
>> diff --git a/tools/cgroup-setup/Cargo.toml b/tools/cgroup-setup/Cargo.toml
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..9cce0be706151dfd5264a81afb3ee9a6c851367c
>> --- /dev/null
>> +++ b/tools/cgroup-setup/Cargo.toml
>> @@ -0,0 +1,17 @@
>> +# SPDX-License-Identifier: CC0-1.0
>> +# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
>> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>> +
>> +[package]
>> +name = "cgroup-setup"
>> +edition = "2024"
>> +
>> +[dependencies]
>> +libc = "0.2.177"
>> +rustix = { version = "1.1.2", features = ["fs"] }
>> +
>> +[profile.release]
>> +split-debuginfo = "symbols"
>> +strip = "symbols"
>> +lto = true
>> +panic = "abort"
>
> What's with all this stuff? Why do we set it only for this program, and
> not for any others? Please don't introduce divergence in build settings
> between programs without a good reason.
I was trying to reduce the binary size as it seemed rather bloated.
I'll leave them out
>> diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..cd77becc0b40bfdcd983e53b63b7c21e4308d5fc
>> --- /dev/null
>> +++ b/tools/cgroup-setup/src/cgroup.rs
>> @@ -0,0 +1,181 @@
>> +// SPDX-License-Identifier: EUPL-1.2+
>> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>> +
>> +use std::fs::File;
>> +use std::io::{Read, Seek};
>> +use std::os::fd::{AsFd, OwnedFd};
>> +use std::os::fd::{AsRawFd, BorrowedFd};
>> +
>> +use std::path::{Path, PathBuf};
>> +
>> +use rustix::fs::AtFlags;
>> +use rustix::path::Arg;
>> +use rustix::{
>> + fs::{Mode, OFlags, ResolveFlags},
>> + io::Errno,
>> +};
>> +
>> +pub(crate) struct LeafCgroup {
>> + path: PathBuf,
>> + fd: OwnedFd,
>
> root_fd would be a clearer name, if I'm understanding correctly.
It's the FD for the cgroup directory, but not for the root of the
cgroup tree.
>> +}
>> +
>> +enum Access {
>> + Read,
>> + Write,
>> +}
>
> Given this isn't public, it doesn't seem to add any value over passing
> around OFlags::RDONLY and OFlags::WRONLY directly.
Will change in v3.
>> +
>> +impl LeafCgroup {
>> + fn open_subtree(&self, path: &std::path::Path, access: Access) -> Result<OwnedFd, Errno> {
>> + rustix::fs::openat2(
>> + self.fd.as_fd(),
>> + path,
>> + OFlags::NOATIME
>> + | OFlags::CLOEXEC
>> + | OFlags::NOFOLLOW
>> + | match access {
>> + Access::Write => OFlags::WRONLY,
>> + Access::Read => OFlags::RDONLY,
>> + },
>> + Mode::empty(),
>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>> + )
>> + }
>> +
>> + pub fn kill_processes(&self) -> std::io::Result<()> {
>> + let kill_fd = self.open_subtree(std::path::Path::new("cgroup.kill"), Access::Write)?;
>> + let wait_file = self.open_subtree(std::path::Path::new("cgroup.events"), Access::Read)?;
>> + let poll_fd = wait_file.as_raw_fd();
>> + let mut wait_fd = File::from(wait_file);
>> + assert_eq!(rustix::io::write(kill_fd.as_fd(), b"1")?, 1, "kernel bug");
>> + let mut fds = libc::pollfd {
>> + fd: poll_fd,
>> + events: libc::POLLIN | libc::POLLPRI | libc::POLLRDHUP,
>> + revents: 0,
>> + };
>> + // SAFETY: FFI call, valid arguments, fds contains 1 element
>> + let mut v = vec![];
>> + 'a: loop {
>> + v.clear();
>> + if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
>> + panic!("poll failed");
>> + }
>
> Shouldn't we start polling before writing? Otherwise we might miss the
> notification, if it comes in between the write and the poll.
I'll test this, but I think it shouldn't matter. If it does matter, I'll
need to switch to epoll (or io_uring, but that's overkill here).
>> + wait_fd
>> + .seek(std::io::SeekFrom::Start(0))
>> + .expect("Seek on control group file should succeed");
>> + wait_fd
>> + .read_to_end(&mut v)
>> + .expect("reading from control group should work");
>> + for substr in v.split(|&c| c == b'\n') {
>> + if substr == b"populated 0" {
>
> Could do this in one line, and avoid the labelled break:
>
> if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
>
> (I think "line" is a clearer name than "substr".)
Will fix in v3.
>> + break 'a;
>> + }
>> + }
>> + }
>> + Ok(())
>> + }
>> +}
>> +
>> +impl AsFd for LeafCgroup {
>> + fn as_fd(&self) -> std::os::fd::BorrowedFd<'_> {
>> + self.fd.as_fd()
>> + }
>> +}
>> +
>> +impl LeafCgroup {
>> + pub(crate) fn open_cgroup_at(&self, p: &Path) -> Result<Self, Errno> {
>> + let fd = rustix::fs::openat2(
>> + self.fd.as_fd(),
>> + p,
>> + OFlags::NOATIME
>> + | OFlags::CLOEXEC
>> + | OFlags::NOFOLLOW
>> + | OFlags::RDONLY
>> + | OFlags::DIRECTORY,
>> + Mode::empty(),
>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>> + )?;
>> + Ok(Self {
>> + fd,
>> + path: self.path.join(p),
>> + })
>> + }
>> + pub(crate) fn open_cgroup(fd: BorrowedFd, p: &Path) -> Result<Self, Errno> {
>> + let fd = rustix::fs::openat2(
>> + fd,
>> + p,
>> + OFlags::NOATIME
>> + | OFlags::CLOEXEC
>> + | OFlags::NOFOLLOW
>> + | OFlags::RDONLY
>> + | OFlags::DIRECTORY,
>> + Mode::empty(),
>> + //ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>
> Stray comment.
Will fix in v3.
>> + ResolveFlags::NO_SYMLINKS,
>> + )?;
>> + Ok(Self {
>> + fd,
>> + path: p.to_owned(),
>> + })
>> + }
>> + pub(crate) fn make_child(&self, p: &Path) -> Result<(), Errno> {
>> + rustix::fs::mkdirat(
>> + self.fd.as_fd(),
>> + p,
>> + Mode::RUSR
>> + | Mode::WUSR
>> + | Mode::XUSR
>> + | Mode::RGRP
>> + | Mode::XGRP
>> + | Mode::ROTH
>> + | Mode::XOTH,
>> + )
>> + }
>> +
>> + pub(crate) fn delete_child(&self, path: &Path) -> Result<(), Errno> {
>> + remove_all(100, self.as_fd(), &path.as_cow_c_str().unwrap())
>> + }
>> +}
>> +
>> +fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> {
>> + if remaining_depth < 1 {
>> + panic!("control groups too deeply nested");
>> + }
>> + let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating");
>> + while let Some(element) = d.next() {
>> + let element = element.expect("Iterating through a cgroup directory failed?");
>> + if element.file_type() != rustix::fs::FileType::Directory {
>> + continue;
>> + }
>> +
>> + let remaining_depth = remaining_depth - 1;
>> + let d: &rustix::fs::Dir = &d;
>> + let dirfd = d.fd().unwrap();
>> + let path = element.file_name();
>> + remove_all(remaining_depth, dirfd, path)?;
>> + }
>> + Ok(())
>> +}
>> +
>> +fn remove_all(
>> + remaining_depth: usize,
>> + dirfd: BorrowedFd<'_>,
>> + path: &std::ffi::CStr,
>> +) -> Result<(), Errno> {
>> + if path == c"." || path == c".." {
>> + return Ok(());
>> + }
>> + if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() {
>> + return Ok(());
>> + }
>> + let fd = rustix::fs::openat2(
>> + dirfd,
>> + path,
>> + OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY,
>> + Mode::empty(),
>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>> + )?;
>> + remove_recursively(fd, remaining_depth)?;
>> + rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?;
>> + Ok(())
>> +}
>> diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..358703a73fcbdc38794312fe3987e733f37c2df0
>> --- /dev/null
>> +++ b/tools/cgroup-setup/src/main.rs
>> @@ -0,0 +1,196 @@
>> +// SPDX-License-Identifier: EUPL-1.2+
>> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>> +
>> +use std::{
>> + ffi::{OsStr, OsString},
>> + fs::File,
>> + io::Write as _,
>> + os::unix::prelude::*,
>> + path::{Component, Path, PathBuf},
>> +};
>> +
>> +use rustix::{
>> + fs::{FlockOperation, Mode, OFlags, ResolveFlags},
>> + io::Errno,
>> +};
>> +
>> +mod cgroup;
>> +
>> +macro_rules! fail {
>> + ($($arg:tt)*) => {{
>> + eprintln!($($arg)*);
>> + std::process::exit(1)}
>> + };
>> +}
>
> I'd find the error handling Rustier if we did error returns with the ?
> operator, like how mount-flatpak does it. The error type can just be
> String.
Sure!
>> +
>> +fn main() {
>> + let mut args = std::env::args_os();
>> + let Some(prog_name) = args.next() else {
>> + fail!("No command line arguments (argv[0] is NULL)");
>> + };
>> + let mut cgroup_relative_path = args.next();
>> + if cgroup_relative_path.as_deref() == Some(OsStr::from_bytes(b"--")) {
>> + cgroup_relative_path = args.next();
>> + }
>
> We could just not support -- and simplify invocation, right?
Only if we never want to support any options in the future.
>> +
>> + let Some(cgroup_relative_path) = cgroup_relative_path else {
>> + fail!("{prog_name:?}: Have no positional arguments, expect at least 1")
>> + };
>> + let mut cgroup_relative_path = cgroup_relative_path.into_vec();
>> +
>> + // slow but we do not care
>
> Slow?
O(n) for a size-n Vec. Not something you want in a hot path, but fine here.
>> + if cgroup_relative_path.len() > 247 {
>
> This magic number could perhaps use a name.
Sure.
>> + fail!("{prog_name:?}: Cgroup name {cgroup_relative_path:?} too long");
>> + }
>> + if cgroup_relative_path.is_empty() {
>> + fail!("{prog_name:?}: Cgroup name {cgroup_relative_path:?} empty");
>> + }
>> + if cgroup_relative_path[0] == b'-' {
>> + fail!("{prog_name:?}: Cgroup name {cgroup_relative_path:?} starts with '-'");
>> + }
>
> Is that a problem?
In theory, no, but it's probably a bug in the caller.
>> + for &i in &cgroup_relative_path {
>> + match i {
>> + b'/' => fail!(
>> + "{prog_name:?}: Cgroup name {cgroup_relative_path:?} \
>> +contains /"
>> + ),
>> + b'$' => fail!(
>> + "{prog_name:?}: Cgroup name {cgroup_relative_path:?} \
>> +contains $: did you forget an execline substitution?"
>> + ),
>> + b'!'..=b'~' => {}
>> + _ => fail!(
>> + "{prog_name:?}: Cgroup name {cgroup_relative_path:?} \
>> +contains space, non-ASCII character, or control character (byte {i})"
>> + ),
>
> Surely every kernel interface we're going to use is 8-bit clean. If
> this is about log viewing again, I really don't like it. Even if we're
> careful about it in first-party code (which itself is a big ask), we
> can't expect it of every other package that might log something. People
> viewing logs have to use robust tools to do so regardless, or we have to
> mitigate this somewhere centrally, e.g. in s6 log. Doing this piecemeal
> does not meaningfully mitigate the problem, and arguably creates a false
> sense of security.
This was inspired by systemd, which has restrictions on what characters
are allowed in service names. Also, restricting allowed characters
makes it easier to find bugs, such as "${a} ${b}" instead of "${a}"
"${b}" in an execline script.
Forbidding '$' is very much intentional, and is inspired by me making
the exact mistake described by the error message. It also means that
certain names can be reserved for internal use.
>> + }
>> + }
>> + cgroup_relative_path.extend_from_slice(b".service");
>
> What do we need the suffix for?
The kernel may add new control files at any time. However, it will
never use the .service suffix to avoid breaking systemd. Therefore, I
chose to use that suffix to avoid breaking in future kernel releases.
>> + let cgroup_relative_path = Path::new(OsStr::from_bytes(&cgroup_relative_path));
>> +
>> + let local_cgroup = std::fs::read("/proc/thread-self/cgroup")
>
> Ooc, why /proc/thread-self instead of /proc/self?
Why not? The two are aliases in this case, but in general /proc/thread-self is likely the better choice.
>> + .unwrap_or_else(|e| fail!("{prog_name:?}: cannot read /proc/thread-self/cgroup: {e}"));
>> +
>> + if !local_cgroup.starts_with(b"0::/")
>> + || !local_cgroup.ends_with(b"\n")
>> + || local_cgroup.contains(&b'\0')
>> + {
>> + fail!("{prog_name:?}: Kernel bug: Bad contents of /proc/thread-self/cgroup");
>> + }
>
> We are not a kernel fuzzer. We can't reasonably write programs that
> have to anticipate kernel contract violations.
Fair!
>> +
>> + let mut local_cgroup = PathBuf::from(<OsString as OsStringExt>::from_vec(
>> + local_cgroup[4..local_cgroup.len() - 1].to_owned(),
>
> … but maybe we could make this clearer, and still satisfy your instincts
> to validate, by using slice::strip_prefix and slice::strip_suffix,
> unwrapping the results?
That's even better, because it's much easier to read than magic numbers.
>> + ));
>> +
>> + match local_cgroup.components().next_back() {
>> + Some(Component::Prefix(_)) => unreachable!("does not occur on Linux"),
>> + Some(Component::RootDir) | None => {}
>> + Some(Component::CurDir) => unreachable!("not produced by kernel"),
>> + Some(Component::ParentDir) => unreachable!("not produced by kernel"),
>> + Some(Component::Normal(os_str)) => {
>> + if os_str.as_bytes() == b"@inner.service" {
>> + let _ = local_cgroup.pop();
>> + }
>> + }
>> + }
>
> Could we not simplify this with local_cgroup.file_name()? If it's None,
> it's the root directory; otherwise it's the equivalent of normal.
Yup!
>> +
>> + if local_cgroup.as_os_str().is_empty() {
>> + local_cgroup = PathBuf::from(".");
>> + }
>
> Path::is_empty is stable in 1.98. Could we mention that in a TODO/FIXME
> comment so we can use it when available?
Will do.
>> + let cgroup_root = rustix::fs::openat2(
>> + rustix::fs::CWD,
>> + Path::new("/sys/fs/cgroup"),
>> + OFlags::DIRECTORY | OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW,
>> + Mode::empty(),
>> + ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_SYMLINKS,
>> + )
>> + .unwrap_or_else(|e| fail!("{prog_name:?}: cannot open /sys/fs/cgroup: {e}"));
>
> Why do we need to use openat2 for this, but not for reading
> /proc/thread-self/cgroup? I'm a bit surprised to see openat2 here at
> all, since only root could manipulate these paths, in which case they
> have no need to use this program as a confused deputy, right?
>
> If openat2 is to stay, isn't OFlags::NOFOLLOW redundant with
> ResolveFlags::NO_SYMLINKS?
openat2() is indeed pointless here.
>> +
>> + let cgroup = match cgroup::LeafCgroup::open_cgroup(cgroup_root.as_fd(), &local_cgroup) {
>> + Ok(e) => e,
>> + Err(e) => fail!(
>> + "{prog_name:?}: Failed to open {}: {e:?}",
>> + local_cgroup.display()
>> + ),
>> + };
>> +
>> + match rustix::fs::flock(cgroup.as_fd(), FlockOperation::LockExclusive) {
>> + Ok(()) => {}
>> + Err(e) => fail!("{prog_name:?}: cannot lock cgroup: {e}"),
>> + }
>
> Would std::fs::File::lock not be a bit Rustier?
Yes :). I know how to
>> + purge_cgroup(&prog_name, &cgroup, cgroup_relative_path);
>
> I find it a little weird that we run this program in both service
> startup and finish, when there are really two completely separable parts
> to it, right? Why is purging not a separate program that only runs in
> finish?
The finish script might fail, time out, etc. Purging at startup
ensures that any previous cgroup state (like attached BPF programs)
is removed, and it ensures that the program this program invokes will
not run concurrently. It's just more robust overall.
Purging is idempotent, and purging an already-purged cgroup is very
cheap. Also, this allows using cgroup-setup without a finish script,
while still ensuring that if the service restarts there are no stale
processes left behind.
>> + let Some(program_name) = args.next() else {
>> + return;
>> + };
>
> This could be moved closer to where it's used.
Will fix in v3.
>> + if let Err(e) = cgroup.make_child(cgroup_relative_path) {
>> + fail!("{prog_name:?}: cannot create child cgroup: {e}")
>> + }
>> + let child = cgroup
>> + .open_cgroup_at(cgroup_relative_path)
>> + .unwrap_or_else(|e| {
>> + fail!("{prog_name:?}: cannot create child cgroup {cgroup_relative_path:?}: {e}")
>> + });
>> + child
>> + .make_child(Path::new("@inner.service"))
>> + .unwrap_or_else(|e| {
>> + fail!("{prog_name:?}: cannot create child cgroup {cgroup_relative_path:?}/@inner.service: {e}")
>> + });
>> + let child = child
>> + .open_cgroup_at(Path::new("@inner.service"))
>> + .unwrap_or_else(|e| {
>> + fail!("{prog_name:?}: cannot open child cgroup {cgroup_relative_path:?}/@inner.service: {e}")
>> + });
>
> This would really benefit from having been written up, so I don't have
> to guess what it's for. We only really need it for supervisors, right?
It's only *needed* when the program might be called recursively,
but it's easier to just use it everywhere.
>> +
>> + // SAFETY: safe FFI call
>> + let pid = unsafe { libc::getpid() };
>
> Not std::process::id()?
Only because I was not aware of it.
>> + write_cgroup_value(&prog_name, &child, "cgroup.procs", &pid.to_string());
>> +
>> + let e = std::process::Command::new(&program_name).args(args).exec();
>> + fail!(
>> + "{prog_name:?}: cannot spawn child {:?}: {}",
>> + program_name,
>> + e
>> + );
>> +}
>> +
>> +fn write_cgroup_value(prog_name: &OsStr, child: &cgroup::LeafCgroup, name: &str, value: &str) {
>
> Perhaps it would be nicer for this to be a method on LeafCgroup?
Sure!
>> + let path = Path::new(name);
>> + let fd = rustix::fs::openat2(
>> + child.as_fd(),
>> + path,
>> + OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::WRONLY,
>> + Mode::empty(),
>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>> + )
>> + .unwrap_or_else(|e| fail!("{prog_name:?}: cannot open {:?}: {}", path, e));
>> + let () = File::from(fd)
>> + .write_all(value.as_bytes())
>> + .unwrap_or_else(|e| {
>> + fail!(
>> + "{prog_name:?}: Cannot write {:?} to {:?}: {}",
>> + name,
>> + path,
>> + e
>> + )
>> + });
>
> Similarly to above, if we don't need openat2, this could be done more
> nicely as std::fs::write.
std::fs doesn't even expose openat(), and I'm much more comfortable
using a directory FD here. This is a fairly severe limitation in
the stdlib. Fixing it requires using the Native API on Windows,
but that's easy enough.
>> +}
>> +
>> +fn purge_cgroup(prog_name: &OsStr, cgroup: &cgroup::LeafCgroup, cgroup_relative_path: &Path) {
>> + let child = match cgroup.open_cgroup_at(cgroup_relative_path) {
>> + Ok(child_cgroup) => child_cgroup,
>> + Err(Errno::NOENT) => return,
>> + Err(other) => {
>> + fail!("{prog_name:?}: cannot open child cgroup {cgroup_relative_path:?}: {other}")
>> + }
>> + };
>> + child.kill_processes().unwrap_or_else(|e| {
>> + fail!("{prog_name:?}: cannot kill programs in {cgroup_relative_path:?}: {e}")
>> + });
>> +
>> + cgroup
>> + .delete_child(cgroup_relative_path)
>> + .unwrap_or_else(|e| {
>> + fail!("{prog_name:?}: delete child cgroup {cgroup_relative_path:?}: {e}")
>> + });
>> +}
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v2] Set up control groups for most services
2026-06-25 2:03 ` Demi Marie Obenour
@ 2026-06-25 3:03 ` Demi Marie Obenour
2026-06-25 9:55 ` Alyssa Ross
2026-06-25 9:49 ` Alyssa Ross
1 sibling, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-06-25 3:03 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1.1: Type: text/plain, Size: 6712 bytes --]
On 6/24/26 22:03, Demi Marie Obenour wrote:
> On 6/24/26 08:13, Alyssa Ross wrote:
>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>
>>> The cgroups are handled by a Rust tool. The name of the cgroup is
>>> autogenerated from the service name.
>>>
>>> Using cgroups for process control is not implemented yet.
>>>
>>
>> Perhaps you could include some more details of how the cgroup hierarchy
>> is supposed to be set up and why, either in the patch body or in
>> documentation?
>
> Will fix in v3.
>
>> As I understood it, the point of using cgroups was that we could bundle
>> all services for a VM, including the VMM, into a single cgroup, but I
>> don't see that here, just a pure translation of the service hierarchy
>> (which the VMM might not even be part of). Are per-VM cgroups coming
>> later?
>
> I think it would be significantly simpler to have the VMM be
> just another VM service. This would naturally put it under the
> vm-services cgroup, solving this problem. Since this would mostly
> involve changes to execline scripting, I think it would be better if
> you wrote this code.
>
>>> diff --git a/host/rootfs/image/etc/fstab b/host/rootfs/image/etc/fstab
>>> index 18bb5e45abafeaf43871306ce8233239e5c07f76..d92364d83f4d26f766fcd5b494614c06a630d2fe 100644
>>> --- a/host/rootfs/image/etc/fstab
>>> +++ b/host/rootfs/image/etc/fstab
>>> @@ -6,3 +6,4 @@ tmpfs /dev/shm tmpfs nosuid,nodev 0 0
>>> tmpfs /media tmpfs nosuid,nodev,noexec,nosymfollow,mode=755 0 0
>>> sysfs /sys sysfs nosuid,nodev,noexec 0 0
>>> tmpfs /tmp tmpfs nosuid,nodev 0 0
>>> +cgroup2 /sys/fs/cgroup cgroup2 nosuid,nodev,noexec,nosymfollow 0 0
>>
>> Alignment is off here. The rest of the file uses tabs at 8 characters.
>> (I know, I know, but it's traditional.)
>>
>>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
>>> index bf66ab0878fc6d8e77ca71a4a89e50c139a6bb11..5c9988c994ed9b942329055673b332d0aa918957 100755
>>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
>>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
>>> @@ -1,7 +1,8 @@
>>> -#!/bin/execlineb -WP
>>> +#!/bin/execlineb -WS1
>>> # SPDX-License-Identifier: EUPL-1.2+
>>> # SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
>>>
>>> +cgroup-setup -- $1
>>> piperw 3 4
>>> background {
>>> fdclose 3
>>
>> What benefit does this cgroup provide?
>
> None, I'll remove it.
>
> The only requirement of the current code is that if a service is under
> a cgroup, its parent should *also* be under a cgroup. Otherwise you
> can get conflicting cgroup paths.
>
>>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
>>> index 78f794202bf174f3c036f3e20755ac087a988277..ed0808b9e45b077023f237a0f9c0e5c830015ac2 100755
>>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
>>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
>>> @@ -1,5 +1,6 @@
>>> -#!/bin/execlineb -WP
>>> +#!/bin/execlineb -WS1
>>> # SPDX-License-Identifier: EUPL-1.2+
>>> # SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
>>>
>>> +cgroup-setup -- $1
>>> s6-svscan -d3 instance
>>
>> We're putting supervisors of instances services in a cgroup? Can you
>> explain to me why that's useful?
>
> This allows configuring limits that apply to the whole instance, and
> means that the whole instance can be reliably killed. Services spawned
> by the instance will be in their own sub-cgroups, so one can apply
> separate limits to them so long as those limits do not exceed those
> of the containing cgroup.
>
> The hierarchy works like this:
>
> /sys/fs/cgroup
> service1.service
> @inner.service # processes go here
> service2.service
> @inner.service # and here
> service3.service
> @inner.service # and here
>
> You can apply resource limits at any level of the hierarchy. Limits in
> nested cgroups can be smaller but not larger. Terminating processes
> in a cgroup with cgroup.kill also terminates all processes in child
> cgroups. After terminating the processes in a cgroup, cgroup-setup
> will delete everything in the hierarchy recursively.
More generally:
The design I implemented is based on a model where cgroups and services
are very tightly coupled: most services have an associated cgroup and
every cgroup is attached to a service. This is like systemd service
units, except that it can be nested many layers deep.
Since a VM may have many processes that serve it, this model requires
that all of these processes be under a single service. In this case,
that service would be the vm-services service.
That said, I'm not sure which of these service, if any, should be
automatically restarted if they crash. Automatic restart of services
is actually very useful for attackers, as it lets them retry an exploit
over and over until they get the memory layout they need for success.
If you restart the VMM, you might as well restart every other
service as well. Unless Cloud Hypervisor automatically reconnects to
vhost-user devices (and I don't think it does), there's no point in
restarting virtiofsd or crosvm without either re-adding the device
or restarting the VMM. Even if you did re-add the device, I don't
know if the guest would be able to recover.
Without automatic restart, there's no real reason to use s6 to manage
the per-VM services. s6 assumes you do want to restart services
that crash.
An alternative approach is to have cgroups be freestanding objects
that exist independently of whether there are processes in them.
This corresponds to systemd scope units. It's not hard to implement
this: just disable automatic purging and pass the full path to the
cgroup. But it requires each VM service to have identical resource
limits, whereas one typically wants per-process limits. For instance,
none of the services should be able to create child processes, and
per-service cgroups make this easy: just set the maximum amount of
processes in the cgroup to 1.
Incidentally, this works very well with PID namespaces. One can have
the "main" process of the service run in a PID namespace, and have
all the others enter that namespace. Then if the main process dies,
so do all of the others.
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v2] Set up control groups for most services
2026-06-25 2:03 ` Demi Marie Obenour
2026-06-25 3:03 ` Demi Marie Obenour
@ 2026-06-25 9:49 ` Alyssa Ross
1 sibling, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-06-25 9:49 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 25260 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> On 6/24/26 08:13, Alyssa Ross wrote:
>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>
>>> The cgroups are handled by a Rust tool. The name of the cgroup is
>>> autogenerated from the service name.
>>>
>>> Using cgroups for process control is not implemented yet.
>>>
>>
>> Perhaps you could include some more details of how the cgroup hierarchy
>> is supposed to be set up and why, either in the patch body or in
>> documentation?
>
> Will fix in v3.
>
>> As I understood it, the point of using cgroups was that we could bundle
>> all services for a VM, including the VMM, into a single cgroup, but I
>> don't see that here, just a pure translation of the service hierarchy
>> (which the VMM might not even be part of). Are per-VM cgroups coming
>> later?
>
> I think it would be significantly simpler to have the VMM be
> just another VM service. This would naturally put it under the
> vm-services cgroup, solving this problem. Since this would mostly
> involve changes to execline scripting, I think it would be better if
> you wrote this code.
No, I don't think it can work that way, because the VMM is in
increasingly many cases not a service at all, since it's run-once.
(Think about e.g. an appimage VM.) This is why it's not under
vm-services today. (I thought we'd discussed this before, but it would
have been months ago given how long ago we planned this…)
>>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
>>> index 78f794202bf174f3c036f3e20755ac087a988277..ed0808b9e45b077023f237a0f9c0e5c830015ac2 100755
>>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
>>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
>>> @@ -1,5 +1,6 @@
>>> -#!/bin/execlineb -WP
>>> +#!/bin/execlineb -WS1
>>> # SPDX-License-Identifier: EUPL-1.2+
>>> # SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
>>>
>>> +cgroup-setup -- $1
>>> s6-svscan -d3 instance
>>
>> We're putting supervisors of instances services in a cgroup? Can you
>> explain to me why that's useful?
>
> This allows configuring limits that apply to the whole instance, and
> means that the whole instance can be reliably killed. Services spawned
> by the instance will be in their own sub-cgroups, so one can apply
> separate limits to them so long as those limits do not exceed those
> of the containing cgroup.
>
> The hierarchy works like this:
>
> /sys/fs/cgroup
> service1.service
> @inner.service # processes go here
> service2.service
> @inner.service # and here
> service3.service
> @inner.service # and here
>
> You can apply resource limits at any level of the hierarchy. Limits in
> nested cgroups can be smaller but not larger. Terminating processes
> in a cgroup with cgroup.kill also terminates all processes in child
> cgroups. After terminating the processes in a cgroup, cgroup-setup
> will delete everything in the hierarchy recursively.
I just don't really understand when we'd want that. Why would we want
to limit all instances of vm-services, or all instances of serial-getty?
>>> diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs
>>> new file mode 100644
>>> index 0000000000000000000000000000000000000000..cd77becc0b40bfdcd983e53b63b7c21e4308d5fc
>>> --- /dev/null
>>> +++ b/tools/cgroup-setup/src/cgroup.rs
>>> @@ -0,0 +1,181 @@
>>> +// SPDX-License-Identifier: EUPL-1.2+
>>> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>>> +
>>> +use std::fs::File;
>>> +use std::io::{Read, Seek};
>>> +use std::os::fd::{AsFd, OwnedFd};
>>> +use std::os::fd::{AsRawFd, BorrowedFd};
>>> +
>>> +use std::path::{Path, PathBuf};
>>> +
>>> +use rustix::fs::AtFlags;
>>> +use rustix::path::Arg;
>>> +use rustix::{
>>> + fs::{Mode, OFlags, ResolveFlags},
>>> + io::Errno,
>>> +};
>>> +
>>> +pub(crate) struct LeafCgroup {
>>> + path: PathBuf,
>>> + fd: OwnedFd,
>>
>> root_fd would be a clearer name, if I'm understanding correctly.
>
> It's the FD for the cgroup directory, but not for the root of the
> cgroup tree.
But if you open a child cgroup, it's the same FD, right? So it's a root
of something.
>>> +}
>>> +
>>> +enum Access {
>>> + Read,
>>> + Write,
>>> +}
>>
>> Given this isn't public, it doesn't seem to add any value over passing
>> around OFlags::RDONLY and OFlags::WRONLY directly.
>
> Will change in v3.
>
>>> +
>>> +impl LeafCgroup {
>>> + fn open_subtree(&self, path: &std::path::Path, access: Access) -> Result<OwnedFd, Errno> {
>>> + rustix::fs::openat2(
>>> + self.fd.as_fd(),
>>> + path,
>>> + OFlags::NOATIME
>>> + | OFlags::CLOEXEC
>>> + | OFlags::NOFOLLOW
>>> + | match access {
>>> + Access::Write => OFlags::WRONLY,
>>> + Access::Read => OFlags::RDONLY,
>>> + },
>>> + Mode::empty(),
>>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>>> + )
>>> + }
>>> +
>>> + pub fn kill_processes(&self) -> std::io::Result<()> {
>>> + let kill_fd = self.open_subtree(std::path::Path::new("cgroup.kill"), Access::Write)?;
>>> + let wait_file = self.open_subtree(std::path::Path::new("cgroup.events"), Access::Read)?;
>>> + let poll_fd = wait_file.as_raw_fd();
>>> + let mut wait_fd = File::from(wait_file);
>>> + assert_eq!(rustix::io::write(kill_fd.as_fd(), b"1")?, 1, "kernel bug");
>>> + let mut fds = libc::pollfd {
>>> + fd: poll_fd,
>>> + events: libc::POLLIN | libc::POLLPRI | libc::POLLRDHUP,
>>> + revents: 0,
>>> + };
>>> + // SAFETY: FFI call, valid arguments, fds contains 1 element
>>> + let mut v = vec![];
>>> + 'a: loop {
>>> + v.clear();
>>> + if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
>>> + panic!("poll failed");
>>> + }
>>
>> Shouldn't we start polling before writing? Otherwise we might miss the
>> notification, if it comes in between the write and the poll.
>
> I'll test this, but I think it shouldn't matter. If it does matter, I'll
> need to switch to epoll (or io_uring, but that's overkill here).
Not sure it's something you'll be able to reproduce in a test. We'd
need to understand the reason it could never happen.
>>> + wait_fd
>>> + .seek(std::io::SeekFrom::Start(0))
>>> + .expect("Seek on control group file should succeed");
>>> + wait_fd
>>> + .read_to_end(&mut v)
>>> + .expect("reading from control group should work");
>>> + for substr in v.split(|&c| c == b'\n') {
>>> + if substr == b"populated 0" {
>>
>> Could do this in one line, and avoid the labelled break:
>>
>> if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
>>
>> (I think "line" is a clearer name than "substr".)
>
> Will fix in v3.
>
>>> + break 'a;
>>> + }
>>> + }
>>> + }
>>> + Ok(())
>>> + }
>>> +}
>>> +
>>> +impl AsFd for LeafCgroup {
>>> + fn as_fd(&self) -> std::os::fd::BorrowedFd<'_> {
>>> + self.fd.as_fd()
>>> + }
>>> +}
>>> +
>>> +impl LeafCgroup {
>>> + pub(crate) fn open_cgroup_at(&self, p: &Path) -> Result<Self, Errno> {
>>> + let fd = rustix::fs::openat2(
>>> + self.fd.as_fd(),
>>> + p,
>>> + OFlags::NOATIME
>>> + | OFlags::CLOEXEC
>>> + | OFlags::NOFOLLOW
>>> + | OFlags::RDONLY
>>> + | OFlags::DIRECTORY,
>>> + Mode::empty(),
>>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>>> + )?;
>>> + Ok(Self {
>>> + fd,
>>> + path: self.path.join(p),
>>> + })
>>> + }
>>> + pub(crate) fn open_cgroup(fd: BorrowedFd, p: &Path) -> Result<Self, Errno> {
>>> + let fd = rustix::fs::openat2(
>>> + fd,
>>> + p,
>>> + OFlags::NOATIME
>>> + | OFlags::CLOEXEC
>>> + | OFlags::NOFOLLOW
>>> + | OFlags::RDONLY
>>> + | OFlags::DIRECTORY,
>>> + Mode::empty(),
>>> + //ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>>
>> Stray comment.
>
> Will fix in v3.
>
>>> + ResolveFlags::NO_SYMLINKS,
>>> + )?;
>>> + Ok(Self {
>>> + fd,
>>> + path: p.to_owned(),
>>> + })
>>> + }
>>> + pub(crate) fn make_child(&self, p: &Path) -> Result<(), Errno> {
>>> + rustix::fs::mkdirat(
>>> + self.fd.as_fd(),
>>> + p,
>>> + Mode::RUSR
>>> + | Mode::WUSR
>>> + | Mode::XUSR
>>> + | Mode::RGRP
>>> + | Mode::XGRP
>>> + | Mode::ROTH
>>> + | Mode::XOTH,
>>> + )
>>> + }
>>> +
>>> + pub(crate) fn delete_child(&self, path: &Path) -> Result<(), Errno> {
>>> + remove_all(100, self.as_fd(), &path.as_cow_c_str().unwrap())
>>> + }
>>> +}
>>> +
>>> +fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> {
>>> + if remaining_depth < 1 {
>>> + panic!("control groups too deeply nested");
>>> + }
>>> + let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating");
>>> + while let Some(element) = d.next() {
>>> + let element = element.expect("Iterating through a cgroup directory failed?");
>>> + if element.file_type() != rustix::fs::FileType::Directory {
>>> + continue;
>>> + }
>>> +
>>> + let remaining_depth = remaining_depth - 1;
>>> + let d: &rustix::fs::Dir = &d;
>>> + let dirfd = d.fd().unwrap();
>>> + let path = element.file_name();
>>> + remove_all(remaining_depth, dirfd, path)?;
>>> + }
>>> + Ok(())
>>> +}
>>> +
>>> +fn remove_all(
>>> + remaining_depth: usize,
>>> + dirfd: BorrowedFd<'_>,
>>> + path: &std::ffi::CStr,
>>> +) -> Result<(), Errno> {
>>> + if path == c"." || path == c".." {
>>> + return Ok(());
>>> + }
>>> + if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() {
>>> + return Ok(());
>>> + }
>>> + let fd = rustix::fs::openat2(
>>> + dirfd,
>>> + path,
>>> + OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY,
>>> + Mode::empty(),
>>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>>> + )?;
>>> + remove_recursively(fd, remaining_depth)?;
>>> + rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?;
>>> + Ok(())
>>> +}
>>> diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
>>> new file mode 100644
>>> index 0000000000000000000000000000000000000000..358703a73fcbdc38794312fe3987e733f37c2df0
>>> --- /dev/null
>>> +++ b/tools/cgroup-setup/src/main.rs
>>> @@ -0,0 +1,196 @@
>>> +// SPDX-License-Identifier: EUPL-1.2+
>>> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>>> +
>>> +use std::{
>>> + ffi::{OsStr, OsString},
>>> + fs::File,
>>> + io::Write as _,
>>> + os::unix::prelude::*,
>>> + path::{Component, Path, PathBuf},
>>> +};
>>> +
>>> +use rustix::{
>>> + fs::{FlockOperation, Mode, OFlags, ResolveFlags},
>>> + io::Errno,
>>> +};
>>> +
>>> +mod cgroup;
>>> +
>>> +macro_rules! fail {
>>> + ($($arg:tt)*) => {{
>>> + eprintln!($($arg)*);
>>> + std::process::exit(1)}
>>> + };
>>> +}
>>
>> I'd find the error handling Rustier if we did error returns with the ?
>> operator, like how mount-flatpak does it. The error type can just be
>> String.
>
> Sure!
>
>>> +
>>> +fn main() {
>>> + let mut args = std::env::args_os();
>>> + let Some(prog_name) = args.next() else {
>>> + fail!("No command line arguments (argv[0] is NULL)");
>>> + };
>>> + let mut cgroup_relative_path = args.next();
>>> + if cgroup_relative_path.as_deref() == Some(OsStr::from_bytes(b"--")) {
>>> + cgroup_relative_path = args.next();
>>> + }
>>
>> We could just not support -- and simplify invocation, right?
>
> Only if we never want to support any options in the future.
I guess it's nice for out-of-tree users, and it's a trivial amount of
code.
>>> +
>>> + let Some(cgroup_relative_path) = cgroup_relative_path else {
>>> + fail!("{prog_name:?}: Have no positional arguments, expect at least 1")
>>> + };
>>> + let mut cgroup_relative_path = cgroup_relative_path.into_vec();
>>> +
>>> + // slow but we do not care
>>
>> Slow?
>
> O(n) for a size-n Vec. Not something you want in a hot path, but fine here.
How is getting the length of a Vec O(n)? It stores it.
>>> + if cgroup_relative_path.len() > 247 {
>>
>> This magic number could perhaps use a name.
>
> Sure.
>
>>> + fail!("{prog_name:?}: Cgroup name {cgroup_relative_path:?} too long");
>>> + }
>>> + if cgroup_relative_path.is_empty() {
>>> + fail!("{prog_name:?}: Cgroup name {cgroup_relative_path:?} empty");
>>> + }
>>> + if cgroup_relative_path[0] == b'-' {
>>> + fail!("{prog_name:?}: Cgroup name {cgroup_relative_path:?} starts with '-'");
>>> + }
>>
>> Is that a problem?
>
> In theory, no, but it's probably a bug in the caller.
>
>>> + for &i in &cgroup_relative_path {
>>> + match i {
>>> + b'/' => fail!(
>>> + "{prog_name:?}: Cgroup name {cgroup_relative_path:?} \
>>> +contains /"
>>> + ),
>>> + b'$' => fail!(
>>> + "{prog_name:?}: Cgroup name {cgroup_relative_path:?} \
>>> +contains $: did you forget an execline substitution?"
>>> + ),
>>> + b'!'..=b'~' => {}
>>> + _ => fail!(
>>> + "{prog_name:?}: Cgroup name {cgroup_relative_path:?} \
>>> +contains space, non-ASCII character, or control character (byte {i})"
>>> + ),
>>
>> Surely every kernel interface we're going to use is 8-bit clean. If
>> this is about log viewing again, I really don't like it. Even if we're
>> careful about it in first-party code (which itself is a big ask), we
>> can't expect it of every other package that might log something. People
>> viewing logs have to use robust tools to do so regardless, or we have to
>> mitigate this somewhere centrally, e.g. in s6 log. Doing this piecemeal
>> does not meaningfully mitigate the problem, and arguably creates a false
>> sense of security.
>
> This was inspired by systemd, which has restrictions on what characters
> are allowed in service names. Also, restricting allowed characters
> makes it easier to find bugs, such as "${a} ${b}" instead of "${a}"
> "${b}" in an execline script.
>
> Forbidding '$' is very much intentional, and is inspired by me making
> the exact mistake described by the error message. It also means that
> certain names can be reserved for internal use.
In my opinion, one component should not be responsible for ad-hoc bug
checks in another, but I won't push back too much. The other character
limitations seem completely arbitrary to me though.
>>> + }
>>> + }
>>> + cgroup_relative_path.extend_from_slice(b".service");
>>
>> What do we need the suffix for?
>
> The kernel may add new control files at any time. However, it will
> never use the .service suffix to avoid breaking systemd. Therefore, I
> chose to use that suffix to avoid breaking in future kernel releases.
Makes sense.
>>> + let cgroup_relative_path = Path::new(OsStr::from_bytes(&cgroup_relative_path));
>>> +
>>> + let local_cgroup = std::fs::read("/proc/thread-self/cgroup")
>>
>> Ooc, why /proc/thread-self instead of /proc/self?
>
> Why not? The two are aliases in this case, but in general /proc/thread-self is likely the better choice.
I'd just never seen it before.
>>> + .unwrap_or_else(|e| fail!("{prog_name:?}: cannot read /proc/thread-self/cgroup: {e}"));
>>> +
>>> + if !local_cgroup.starts_with(b"0::/")
>>> + || !local_cgroup.ends_with(b"\n")
>>> + || local_cgroup.contains(&b'\0')
>>> + {
>>> + fail!("{prog_name:?}: Kernel bug: Bad contents of /proc/thread-self/cgroup");
>>> + }
>>
>> We are not a kernel fuzzer. We can't reasonably write programs that
>> have to anticipate kernel contract violations.
>
> Fair!
>
>>> +
>>> + let mut local_cgroup = PathBuf::from(<OsString as OsStringExt>::from_vec(
>>> + local_cgroup[4..local_cgroup.len() - 1].to_owned(),
>>
>> … but maybe we could make this clearer, and still satisfy your instincts
>> to validate, by using slice::strip_prefix and slice::strip_suffix,
>> unwrapping the results?
>
> That's even better, because it's much easier to read than magic numbers.
>
>>> + ));
>>> +
>>> + match local_cgroup.components().next_back() {
>>> + Some(Component::Prefix(_)) => unreachable!("does not occur on Linux"),
>>> + Some(Component::RootDir) | None => {}
>>> + Some(Component::CurDir) => unreachable!("not produced by kernel"),
>>> + Some(Component::ParentDir) => unreachable!("not produced by kernel"),
>>> + Some(Component::Normal(os_str)) => {
>>> + if os_str.as_bytes() == b"@inner.service" {
>>> + let _ = local_cgroup.pop();
>>> + }
>>> + }
>>> + }
>>
>> Could we not simplify this with local_cgroup.file_name()? If it's None,
>> it's the root directory; otherwise it's the equivalent of normal.
>
> Yup!
>
>>> +
>>> + if local_cgroup.as_os_str().is_empty() {
>>> + local_cgroup = PathBuf::from(".");
>>> + }
>>
>> Path::is_empty is stable in 1.98. Could we mention that in a TODO/FIXME
>> comment so we can use it when available?
>
> Will do.
>
>>> + let cgroup_root = rustix::fs::openat2(
>>> + rustix::fs::CWD,
>>> + Path::new("/sys/fs/cgroup"),
>>> + OFlags::DIRECTORY | OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW,
>>> + Mode::empty(),
>>> + ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_SYMLINKS,
>>> + )
>>> + .unwrap_or_else(|e| fail!("{prog_name:?}: cannot open /sys/fs/cgroup: {e}"));
>>
>> Why do we need to use openat2 for this, but not for reading
>> /proc/thread-self/cgroup? I'm a bit surprised to see openat2 here at
>> all, since only root could manipulate these paths, in which case they
>> have no need to use this program as a confused deputy, right?
>>
>> If openat2 is to stay, isn't OFlags::NOFOLLOW redundant with
>> ResolveFlags::NO_SYMLINKS?
>
> openat2() is indeed pointless here.
Great — I think we can really simplify the code if we don't use it where
unnecessary, because we can use a lot more of the standard library.
Possibly Rustix isn't even needed?
>>> +
>>> + let cgroup = match cgroup::LeafCgroup::open_cgroup(cgroup_root.as_fd(), &local_cgroup) {
>>> + Ok(e) => e,
>>> + Err(e) => fail!(
>>> + "{prog_name:?}: Failed to open {}: {e:?}",
>>> + local_cgroup.display()
>>> + ),
>>> + };
>>> +
>>> + match rustix::fs::flock(cgroup.as_fd(), FlockOperation::LockExclusive) {
>>> + Ok(()) => {}
>>> + Err(e) => fail!("{prog_name:?}: cannot lock cgroup: {e}"),
>>> + }
>>
>> Would std::fs::File::lock not be a bit Rustier?
>
> Yes :). I know how to
>
>>> + purge_cgroup(&prog_name, &cgroup, cgroup_relative_path);
>>
>> I find it a little weird that we run this program in both service
>> startup and finish, when there are really two completely separable parts
>> to it, right? Why is purging not a separate program that only runs in
>> finish?
>
> The finish script might fail, time out, etc. Purging at startup
> ensures that any previous cgroup state (like attached BPF programs)
> is removed, and it ensures that the program this program invokes will
> not run concurrently. It's just more robust overall.
>
> Purging is idempotent, and purging an already-purged cgroup is very
> cheap. Also, this allows using cgroup-setup without a finish script,
> while still ensuring that if the service restarts there are no stale
> processes left behind.
Alright.
>>> + let Some(program_name) = args.next() else {
>>> + return;
>>> + };
>>
>> This could be moved closer to where it's used.
>
> Will fix in v3.
>
>>> + if let Err(e) = cgroup.make_child(cgroup_relative_path) {
>>> + fail!("{prog_name:?}: cannot create child cgroup: {e}")
>>> + }
>>> + let child = cgroup
>>> + .open_cgroup_at(cgroup_relative_path)
>>> + .unwrap_or_else(|e| {
>>> + fail!("{prog_name:?}: cannot create child cgroup {cgroup_relative_path:?}: {e}")
>>> + });
>>> + child
>>> + .make_child(Path::new("@inner.service"))
>>> + .unwrap_or_else(|e| {
>>> + fail!("{prog_name:?}: cannot create child cgroup {cgroup_relative_path:?}/@inner.service: {e}")
>>> + });
>>> + let child = child
>>> + .open_cgroup_at(Path::new("@inner.service"))
>>> + .unwrap_or_else(|e| {
>>> + fail!("{prog_name:?}: cannot open child cgroup {cgroup_relative_path:?}/@inner.service: {e}")
>>> + });
>>
>> This would really benefit from having been written up, so I don't have
>> to guess what it's for. We only really need it for supervisors, right?
>
> It's only *needed* when the program might be called recursively,
> but it's easier to just use it everywhere.
Yeah, that makes sense. Would just be nice to have that documented
somewhere so the next person doesn't have to spend the time I did
figuring it out!
>>> +
>>> + // SAFETY: safe FFI call
>>> + let pid = unsafe { libc::getpid() };
>>
>> Not std::process::id()?
>
> Only because I was not aware of it.
>
>>> + write_cgroup_value(&prog_name, &child, "cgroup.procs", &pid.to_string());
>>> +
>>> + let e = std::process::Command::new(&program_name).args(args).exec();
>>> + fail!(
>>> + "{prog_name:?}: cannot spawn child {:?}: {}",
>>> + program_name,
>>> + e
>>> + );
>>> +}
>>> +
>>> +fn write_cgroup_value(prog_name: &OsStr, child: &cgroup::LeafCgroup, name: &str, value: &str) {
>>
>> Perhaps it would be nicer for this to be a method on LeafCgroup?
>
> Sure!
>
>>> + let path = Path::new(name);
>>> + let fd = rustix::fs::openat2(
>>> + child.as_fd(),
>>> + path,
>>> + OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::WRONLY,
>>> + Mode::empty(),
>>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>>> + )
>>> + .unwrap_or_else(|e| fail!("{prog_name:?}: cannot open {:?}: {}", path, e));
>>> + let () = File::from(fd)
>>> + .write_all(value.as_bytes())
>>> + .unwrap_or_else(|e| {
>>> + fail!(
>>> + "{prog_name:?}: Cannot write {:?} to {:?}: {}",
>>> + name,
>>> + path,
>>> + e
>>> + )
>>> + });
>>
>> Similarly to above, if we don't need openat2, this could be done more
>> nicely as std::fs::write.
>
> std::fs doesn't even expose openat(), and I'm much more comfortable
> using a directory FD here. This is a fairly severe limitation in
> the stdlib. Fixing it requires using the Native API on Windows,
> but that's easy enough.
We have a lock on the cgroup though. What are we protecting against by
using openat?
>>> +}
>>> +
>>> +fn purge_cgroup(prog_name: &OsStr, cgroup: &cgroup::LeafCgroup, cgroup_relative_path: &Path) {
>>> + let child = match cgroup.open_cgroup_at(cgroup_relative_path) {
>>> + Ok(child_cgroup) => child_cgroup,
>>> + Err(Errno::NOENT) => return,
>>> + Err(other) => {
>>> + fail!("{prog_name:?}: cannot open child cgroup {cgroup_relative_path:?}: {other}")
>>> + }
>>> + };
>>> + child.kill_processes().unwrap_or_else(|e| {
>>> + fail!("{prog_name:?}: cannot kill programs in {cgroup_relative_path:?}: {e}")
>>> + });
>>> +
>>> + cgroup
>>> + .delete_child(cgroup_relative_path)
>>> + .unwrap_or_else(|e| {
>>> + fail!("{prog_name:?}: delete child cgroup {cgroup_relative_path:?}: {e}")
>>> + });
>>> +}
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v2] Set up control groups for most services
2026-06-25 3:03 ` Demi Marie Obenour
@ 2026-06-25 9:55 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-06-25 9:55 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 7740 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> On 6/24/26 22:03, Demi Marie Obenour wrote:
>> On 6/24/26 08:13, Alyssa Ross wrote:
>>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>>
>>>> The cgroups are handled by a Rust tool. The name of the cgroup is
>>>> autogenerated from the service name.
>>>>
>>>> Using cgroups for process control is not implemented yet.
>>>>
>>>
>>> Perhaps you could include some more details of how the cgroup hierarchy
>>> is supposed to be set up and why, either in the patch body or in
>>> documentation?
>>
>> Will fix in v3.
>>
>>> As I understood it, the point of using cgroups was that we could bundle
>>> all services for a VM, including the VMM, into a single cgroup, but I
>>> don't see that here, just a pure translation of the service hierarchy
>>> (which the VMM might not even be part of). Are per-VM cgroups coming
>>> later?
>>
>> I think it would be significantly simpler to have the VMM be
>> just another VM service. This would naturally put it under the
>> vm-services cgroup, solving this problem. Since this would mostly
>> involve changes to execline scripting, I think it would be better if
>> you wrote this code.
>>
>>>> diff --git a/host/rootfs/image/etc/fstab b/host/rootfs/image/etc/fstab
>>>> index 18bb5e45abafeaf43871306ce8233239e5c07f76..d92364d83f4d26f766fcd5b494614c06a630d2fe 100644
>>>> --- a/host/rootfs/image/etc/fstab
>>>> +++ b/host/rootfs/image/etc/fstab
>>>> @@ -6,3 +6,4 @@ tmpfs /dev/shm tmpfs nosuid,nodev 0 0
>>>> tmpfs /media tmpfs nosuid,nodev,noexec,nosymfollow,mode=755 0 0
>>>> sysfs /sys sysfs nosuid,nodev,noexec 0 0
>>>> tmpfs /tmp tmpfs nosuid,nodev 0 0
>>>> +cgroup2 /sys/fs/cgroup cgroup2 nosuid,nodev,noexec,nosymfollow 0 0
>>>
>>> Alignment is off here. The rest of the file uses tabs at 8 characters.
>>> (I know, I know, but it's traditional.)
>>>
>>>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
>>>> index bf66ab0878fc6d8e77ca71a4a89e50c139a6bb11..5c9988c994ed9b942329055673b332d0aa918957 100755
>>>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
>>>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
>>>> @@ -1,7 +1,8 @@
>>>> -#!/bin/execlineb -WP
>>>> +#!/bin/execlineb -WS1
>>>> # SPDX-License-Identifier: EUPL-1.2+
>>>> # SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
>>>>
>>>> +cgroup-setup -- $1
>>>> piperw 3 4
>>>> background {
>>>> fdclose 3
>>>
>>> What benefit does this cgroup provide?
>>
>> None, I'll remove it.
>>
>> The only requirement of the current code is that if a service is under
>> a cgroup, its parent should *also* be under a cgroup. Otherwise you
>> can get conflicting cgroup paths.
>>
>>>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
>>>> index 78f794202bf174f3c036f3e20755ac087a988277..ed0808b9e45b077023f237a0f9c0e5c830015ac2 100755
>>>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
>>>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
>>>> @@ -1,5 +1,6 @@
>>>> -#!/bin/execlineb -WP
>>>> +#!/bin/execlineb -WS1
>>>> # SPDX-License-Identifier: EUPL-1.2+
>>>> # SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
>>>>
>>>> +cgroup-setup -- $1
>>>> s6-svscan -d3 instance
>>>
>>> We're putting supervisors of instances services in a cgroup? Can you
>>> explain to me why that's useful?
>>
>> This allows configuring limits that apply to the whole instance, and
>> means that the whole instance can be reliably killed. Services spawned
>> by the instance will be in their own sub-cgroups, so one can apply
>> separate limits to them so long as those limits do not exceed those
>> of the containing cgroup.
>>
>> The hierarchy works like this:
>>
>> /sys/fs/cgroup
>> service1.service
>> @inner.service # processes go here
>> service2.service
>> @inner.service # and here
>> service3.service
>> @inner.service # and here
>>
>> You can apply resource limits at any level of the hierarchy. Limits in
>> nested cgroups can be smaller but not larger. Terminating processes
>> in a cgroup with cgroup.kill also terminates all processes in child
>> cgroups. After terminating the processes in a cgroup, cgroup-setup
>> will delete everything in the hierarchy recursively.
>
> More generally:
(I already replied to the previous email before seeing this, but I'll
try to respond in depth here too.)
> The design I implemented is based on a model where cgroups and services
> are very tightly coupled: most services have an associated cgroup and
> every cgroup is attached to a service. This is like systemd service
> units, except that it can be nested many layers deep.
>
> Since a VM may have many processes that serve it, this model requires
> that all of these processes be under a single service. In this case,
> that service would be the vm-services service.
>
> That said, I'm not sure which of these service, if any, should be
> automatically restarted if they crash. Automatic restart of services
> is actually very useful for attackers, as it lets them retry an exploit
> over and over until they get the memory layout they need for success.
This is exactly why this model doesn't fit. It does not make sense to
use an s6 service for something that doesn't get restarted. Oneshot
services are not a concept that exist in s6. For a clean, repeatable
environment etc. one can use s6-sudod, but even if we had an s6-sudo
service for starting VMMs, they still wouldn't be under a vm-services
instance. The service↔cgroup equivalence from systemd does not make
sense here.
> If you restart the VMM, you might as well restart every other
> service as well. Unless Cloud Hypervisor automatically reconnects to
> vhost-user devices (and I don't think it does), there's no point in
> restarting virtiofsd or crosvm without either re-adding the device
> or restarting the VMM. Even if you did re-add the device, I don't
> know if the guest would be able to recover.
Guests are able to recover from e.g. dbus crashing trivially, and I've
seen virtio-fs working fine after a virtiofsd restart too. That's why
we run them as services. It's set up the way it is for a reason!
> Without automatic restart, there's no real reason to use s6 to manage
> the per-VM services. s6 assumes you do want to restart services
> that crash.
Which we do, generally.
> An alternative approach is to have cgroups be freestanding objects
> that exist independently of whether there are processes in them.
> This corresponds to systemd scope units. It's not hard to implement
> this: just disable automatic purging and pass the full path to the
> cgroup. But it requires each VM service to have identical resource
> limits, whereas one typically wants per-process limits. For instance,
> none of the services should be able to create child processes, and
> per-service cgroups make this easy: just set the maximum amount of
> processes in the cgroup to 1.
Why would you be restricted to identical resource limits.
> Incidentally, this works very well with PID namespaces. One can have
> the "main" process of the service run in a PID namespace, and have
> all the others enter that namespace. Then if the main process dies,
> so do all of the others.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* [PATCH v3 00/22] Control group support
2026-06-20 17:27 ` [PATCH v2] " Demi Marie Obenour
2026-06-24 12:13 ` Alyssa Ross
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 01/22] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
` (21 more replies)
1 sibling, 22 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Changes in v3:
- Remove the implicit .service suffix on leaf control groups.
- Make cgroup-setup acts as an s6 finish script when called as "finish".
- Put the VMMs in the same cgroups as the per-VM services.
- Add common helper script for the per-VM services.
- Significantly refactor the Rust code.
- Link to v2: https://spectrum-os.org/lists/archives/spectrum-devel/20260620-cgroups-v2-1-ccae224b6c85@gmail.com
Changes in v2:
- Omit resource control support. It was completely broken, and the way
Spectrum run scripts work means that it is better to setup resource
controls just before the execve() into the final service process.
This will be done by a separate tool.
- Link to v1: https://spectrum-os.org/lists/archives/spectrum-devel/20260620-cgroups-v1-1-0e5abf35101b@gmail.com
---
Demi Marie Obenour (22):
host/rootfs: Mount filesystems before s6-rc-init
scripts: Support symlinks in s6-rc-compile inputs
tools: Add control group manager
Documentation: Mention control groups
Mount cgroup2 filesystem at /sys/fs/cgroup
host/rootfs: Add helper program for per-VM services
host/rootfs: Enable controllers in sub-cgroups
host/rootfs: Add comments where cgroups are intentionally not used
host/rootfs: serial-getty-generator: Use cgroups
host/rootfs: Set up parent cgroup for all per-VM services
host/rootfs: Create per-VM cgroup for all of the VM's services
host/rootfs: run-vmm: Create per-VM cgroup
host/rootfs: run-appimage: Purge the per-VM cgroup
host/rootfs: run-flatpak: Purge the per-VM cgroup
host/rootfs: dbus: Run in cgroup
host/rootfs: vhost-user-fs: Run in cgroup
host/rootfs: vhost-user-gpu: Run in cgroup
host/rootfs: xdg-desktop-portal-spectrum-host: Run in cgroup
host/rootfs: systemd-udevd: Run in cgroup
host/rootfs: weston: Run in cgroup
host/rootfs: spectrum-router: Run in cgroup
host/rootfs: vm-import: Use elglob -w
.codespellrc | 2 +-
Documentation/doc/development/control-groups.adoc | 82 +++++
host/rootfs/Makefile | 32 +-
host/rootfs/default.nix | 7 +-
host/rootfs/file-list.mk | 12 +
host/rootfs/image/etc/fstab | 1 +
host/rootfs/image/etc/init | 15 +-
.../s6-linux-init/run-image/service/getty-tty2/run | 1 +
.../s6-linux-init/run-image/service/getty-tty3/run | 1 +
.../s6-linux-init/run-image/service/getty-tty4/run | 1 +
.../run-image/service/root-terminal/run | 1 +
.../run-image/service/s6-linux-init-shutdownd/run | 1 +
.../run-image/service/s6-svscan-log/run | 1 +
.../service/serial-getty-generator/finish | 1 +
.../run-image/service/serial-getty-generator/run | 4 +-
.../run-image/service/serial-getty/run | 1 +
.../run-image/service/serial-getty/template/run | 1 +
.../run-image/service/vm-services/run | 3 +
.../vm-services/template/data/service/dbus/finish | 1 +
.../vm-services/template/data/service/dbus/run | 4 +-
.../template/data/service/spectrum-router/finish | 1 +
.../template/data/service/spectrum-router/run | 4 +-
.../template/data/service/vhost-user-fs/finish | 1 +
.../template/data/service/vhost-user-fs/run | 5 +-
.../template/data/service/vhost-user-gpu/finish | 1 +
.../template/data/service/vhost-user-gpu/run | 9 +-
.../xdg-desktop-portal-spectrum-host/finish | 1 +
.../service/xdg-desktop-portal-spectrum-host/run | 4 +-
.../run-image/service/vm-services/template/finish | 1 +
.../run-image/service/vm-services/template/run | 4 +
.../rootfs/image/etc/s6-linux-init/scripts/rc.init | 9 +-
host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 1 +
host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +-
host/rootfs/image/etc/s6-rc/weston/finish | 1 +
host/rootfs/image/etc/s6-rc/weston/run | 9 +-
host/rootfs/image/usr/bin/run-appimage | 5 +-
host/rootfs/image/usr/bin/run-flatpak | 4 +-
host/rootfs/image/usr/bin/run-vmm | 2 +
host/rootfs/image/usr/bin/vm-import | 2 +-
host/rootfs/image/usr/bin/vm-service-run | 34 ++
img/app/Makefile | 23 +-
img/app/default.nix | 1 +
img/app/file-list.mk | 2 +
lib/targets.mk | 30 ++
pkgs/default.nix | 1 +
release/live/Makefile | 12 +-
release/live/default.nix | 1 +
scripts/genfiles.awk | 13 +-
tools/cgroup-setup/Cargo.lock | 67 ++++
tools/cgroup-setup/Cargo.lock.license | 2 +
tools/cgroup-setup/Cargo.toml | 11 +
tools/cgroup-setup/default.nix | 18 +
tools/cgroup-setup/src/cgroup.rs | 203 +++++++++++
tools/cgroup-setup/src/main.rs | 406 +++++++++++++++++++++
vm/sys/net/Makefile | 19 +-
vm/sys/net/default.nix | 1 +
vm/sys/net/file-list.mk | 2 +
57 files changed, 979 insertions(+), 108 deletions(-)
---
base-commit: 89f8c9238616b43e48b21e2eade41e061b006cd8
change-id: 20260528-cgroups-d609e270e649
--
Sincerely,
Demi Marie Obenour (she/her/hers)
^ permalink raw reply [flat|nested] 132+ messages in thread
* [PATCH v3 01/22] host/rootfs: Mount filesystems before s6-rc-init
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-13 9:39 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 02/22] scripts: Support symlinks in s6-rc-compile inputs Demi Marie Obenour
` (20 subsequent siblings)
21 siblings, 2 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
cgroup-setup is written in Rust and uses the Rust standard library.
It also relies on cgroupfs being mounted at /sys/fs/cgroup.
Furthermore, it runs very, *very* early on, since it sets up the cgroups
in which s6-svscan and s6-supervise processes run. This allows setting
up sub-cgroups without having to worry about name collisions.
While it would be possible to only mount the bare minimum necessary,
it's simpler to just mount everything before starting any services.
These mounts are all pseudo-filesystems or tmpfs instances, and none
have any dependencies.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/init | 11 ++++++++++-
host/rootfs/image/etc/s6-linux-init/scripts/rc.init | 9 +--------
2 files changed, 11 insertions(+), 9 deletions(-)
diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
index db8a6d9c747e3d212c21b6c983c8f7e299822abe..8ec7859ab00e24007a93d4000c8f34e353de50ae 100755
--- a/host/rootfs/image/etc/init
+++ b/host/rootfs/image/etc/init
@@ -1,6 +1,15 @@
#!/bin/execlineb -Ws0
# SPDX-License-Identifier: EUPL-1.2+
-# SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2020-2022, 2024 Alyssa Ross <hi@alyssa.is>
/bin/setpriv --no-new-privs --
+
+if { /bin/ln -s /proc/self/fd /dev }
+if { /bin/ln -s /proc/self/fd/0 /dev/stdin }
+if { /bin/ln -s /proc/self/fd/1 /dev/stdout }
+if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
+
+if { /bin/mount --make-shared / }
+if { /bin/mount -a --mkdir }
+
/bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
diff --git a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
index f638e373589884acb959c868462fdf532380a851..fba305c4e9cbb9f7535f11bb4c06d8da8a34837b 100755
--- a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
+++ b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
@@ -4,13 +4,6 @@
if { s6-rc-init -c /etc/s6-rc /run/service }
-if { ln -s /proc/self/fd /dev }
-if { ln -s /proc/self/fd/0 /dev/stdin }
-if { ln -s /proc/self/fd/1 /dev/stdout }
-if { ln -s /proc/self/fd/2 /dev/stderr }
-
-if { mount --make-shared / }
-if { mount --make-shared /run }
-if { mount -a --mkdir }
+if { /bin/mount --make-shared /run }
s6-rc change ok-all
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 02/22] scripts: Support symlinks in s6-rc-compile inputs
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 01/22] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-13 9:42 ` Alyssa Ross
2026-07-11 20:12 ` [PATCH v3 03/22] tools: Add control group manager Demi Marie Obenour
` (19 subsequent siblings)
21 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Finish scripts can (and often will) be symbolic links to
/usr/bin/cgroup-setup.
This requires makefile changes, so take the opportunity to move some of
the make code to a new lib/targets.mk.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/Makefile | 32 ++++----------------------------
host/rootfs/default.nix | 1 +
host/rootfs/file-list.mk | 2 ++
img/app/Makefile | 23 +++--------------------
img/app/default.nix | 1 +
img/app/file-list.mk | 2 ++
lib/targets.mk | 30 ++++++++++++++++++++++++++++++
release/live/Makefile | 12 ++++--------
release/live/default.nix | 1 +
scripts/genfiles.awk | 13 ++++++++++++-
vm/sys/net/Makefile | 19 ++++---------------
vm/sys/net/default.nix | 1 +
vm/sys/net/file-list.mk | 2 ++
13 files changed, 67 insertions(+), 72 deletions(-)
diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile
index 4f01e3ecce83558b786542ebbeedb7ff3b4c17a5..4c6233ba355ed7208de64df6c1b929aacfb52bbe 100644
--- a/host/rootfs/Makefile
+++ b/host/rootfs/Makefile
@@ -4,7 +4,8 @@
.POSIX:
-include ../../lib/common.mk
+ROOT = ../..
+include $(ROOT)/lib/common.mk
include file-list.mk
ROOT_FS = build
@@ -85,33 +86,6 @@ build/etc/os-release:
mkdir -p build/etc
sed 's/@VERSION@/$(VERSION)/g' < os-release.in > build/etc/os-release
-build/fifo:
- mkdir -p build
- mkfifo -m 0600 $@
-
-build/empty:
- mkdir -p $@
-
-# s6-rc-compile's input is a directory, but that doesn't play nice
-# with Make, because it won't know to update if some file in the
-# directory is changed, or a file is created or removed in a
-# subdirectory. Using the whole source directory could also end up
-# including files that aren't intended to be part of the input, like
-# temporary editor files or .license files. So for all these reasons,
-# only explicitly listed files are made available to s6-rc-compile.
-build/etc/s6-rc: $(S6_RC_FILES) file-list.mk
- mkdir -p $$(dirname $@)
- rm -rf $@
- set -uo pipefail && dir=$$(mktemp -d) && \
- { tar -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 3; } && \
- s6-rc-compile $@ $$dir; \
- exit=$$?; rm -r $$dir; exit $$exit
-
-clean:
- -chmod -Rf +w build
- rm -rf build
-.PHONY: clean
-
build/live.img: ../../scripts/format-uuid.sh ../../scripts/make-gpt.sh ../../scripts/sfdisk-field.awk build/verity-timestamp $(ROOT_FS_IMAGES)
../../scripts/make-gpt.sh $@.tmp \
$(ROOT_FS_VERITY):verity:$$(../../scripts/format-uuid.sh "$$(dd if=$(ROOT_FS_VERITY_ROOTHASH) bs=32 skip=1 count=1 status=none)"):Spectrum_'$(VERSION).verity' \
@@ -155,3 +129,5 @@ run: build/empty build/live.img $(ROOTFS_VERITY_ROOTHASH)
-vga none \
-device vhost-vsock-pci,guest-cid=3
.PHONY: run
+
+include $(ROOT)/lib/targets.mk
diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix
index 6bfeefbe0a5f76c1538ccb40e5eb8f291f5d3592..4dcc10b9933cdfaf87f87417db1de0daa57b73f5 100644
--- a/host/rootfs/default.nix
+++ b/host/rootfs/default.nix
@@ -116,6 +116,7 @@ stdenvNoCC.mkDerivation {
fileset = fileset.intersection src (fileset.unions [
./.
../../lib/common.mk
+ ../../lib/targets.mk
../../scripts/make-erofs.sh
]);
};
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 3899d620717fc97f42e669e5313c4100dcf5b1cd..7e44fc612a5a9ac8767be9148cdc71869ef16964 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -111,3 +111,5 @@ S6_RC_FILES = \
image/etc/s6-rc/weston/notification-fd \
image/etc/s6-rc/weston/run \
image/etc/s6-rc/weston/type
+
+S6_RC_LINKS =
diff --git a/img/app/Makefile b/img/app/Makefile
index 3996831120c1aa831e44a18579b1f5bbac7cb776..42793725f26af43605c5ae95a2dc78ca1c49efe9 100644
--- a/img/app/Makefile
+++ b/img/app/Makefile
@@ -4,7 +4,8 @@
.POSIX:
-include ../../lib/common.mk
+ROOT = ../..
+include $(ROOT)/lib/common.mk
include file-list.mk
prefix = build/host
@@ -39,13 +40,6 @@ FIFOS = etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/fifo
BUILD_FILES = build/etc/s6-rc
-build/fifo:
- mkdir -p build
- mkfifo -m 0600 $@
-
-build/empty:
- mkdir -p $@
-
build/rootfs.erofs: ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty build/fifo file-list.mk
set -euo pipefail; \
{ \
@@ -56,15 +50,6 @@ build/rootfs.erofs: ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUIL
printf 'build/fifo\n%s\n' $(FIFOS) ;\
} | ../../scripts/make-erofs.sh $@
-
-build/etc/s6-rc: $(S6_RC_FILES) file-list.mk
- mkdir -p $$(dirname $@)
- rm -rf $@
- set -uo pipefail && dir=$$(mktemp -d) && \
- { tar -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 3; } && \
- s6-rc-compile $@ $$dir; \
- exit=$$?; rm -r $$dir; exit $$exit
-
debug:
$(GDB) -q \
-ex 'set substitute-path .. $(LINUX_SRC)' \
@@ -144,6 +129,4 @@ run-crosvm: $(imgdir)/appvm/blk/root.img start-vhost-user-gpu start-virtiofsd
run: run-$(VMM)
.PHONY: run
-clean:
- rm -rf build
-.PHONY: clean
+include $(ROOT)/lib/targets.mk
diff --git a/img/app/default.nix b/img/app/default.nix
index 71e6fa02e78553d027d59f5b2ff728830276d480..97f4cc75d620c93c6f5a6de7c7069cf3d39f25b5 100644
--- a/img/app/default.nix
+++ b/img/app/default.nix
@@ -104,6 +104,7 @@ stdenvNoCC.mkDerivation {
fileset = lib.fileset.intersection src (lib.fileset.unions [
./.
../../lib/common.mk
+ ../../lib/targets.mk
../../scripts/make-erofs.sh
../../scripts/make-gpt.sh
../../scripts/sfdisk-field.awk
diff --git a/img/app/file-list.mk b/img/app/file-list.mk
index 815b63ab2a4f2d11f63ce5593fecefa979c13eba..6312c361b629430629661be30332fac38ca0be73 100644
--- a/img/app/file-list.mk
+++ b/img/app/file-list.mk
@@ -71,3 +71,5 @@ S6_RC_FILES = \
image/etc/s6-rc/wireplumber/dependencies.d/pipewire \
image/etc/s6-rc/wireplumber/run \
image/etc/s6-rc/wireplumber/type
+
+S6_RC_LINKS =
diff --git a/lib/targets.mk b/lib/targets.mk
new file mode 100644
index 0000000000000000000000000000000000000000..2d6d2682df2ed070b3f6075913fcfc0e6a803bbb
--- /dev/null
+++ b/lib/targets.mk
@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2021, 2023, 2024, 2025 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
+
+build/fifo:
+ mkdir -p build
+ mkfifo -m 0600 $@
+
+build/empty:
+ mkdir -p $@
+
+# s6-rc-compile's input is a directory, but that doesn't play nice
+# with Make, because it won't know to update if some file in the
+# directory is changed, or a file is created or removed in a
+# subdirectory. Using the whole source directory could also end up
+# including files that aren't intended to be part of the input, like
+# temporary editor files or .license files. So for all these reasons,
+# only explicitly listed files are made available to s6-rc-compile.
+build/etc/s6-rc: $(S6_RC_FILES) file-list.mk
+ mkdir -p $$(dirname $@)
+ rm -rf $@
+ set -uo pipefail && dir=$$(mktemp -d) && \
+ { tar -c $(S6_RC_FILES) $(S6_RC_LINKS) | tar -C $$dir -x --strip-components 3; } && \
+ s6-rc-compile $@ $$dir; \
+ exit=$$?; rm -r $$dir; exit $$exit
+
+clean:
+ -chmod -Rf +w build
+ rm -rf build
+.PHONY: clean
diff --git a/release/live/Makefile b/release/live/Makefile
index 85319d0eec06df4e7ab1e28030e3bb78608194df..f984bc8d5a93733a246ccc901674be06bbcbb69e 100644
--- a/release/live/Makefile
+++ b/release/live/Makefile
@@ -3,7 +3,8 @@
.POSIX:
-include ../../lib/common.mk
+ROOT = ../..
+include $(ROOT)/lib/common.mk
dest = build/live.img
@@ -24,13 +25,6 @@ build/boot.fat: $(SYSTEMD_BOOT_EFI) $(SPECTRUM_EFI)
$(MCOPY) -i $@ $(SPECTRUM_EFI) ::/EFI/Linux/'Spectrum_$(VERSION).efi'
$(MCOPY) -i $@ $(SYSTEMD_BOOT_EFI) ::/EFI/BOOT/$(EFINAME)
-build/empty:
- mkdir -p $@
-
-clean:
- rm -rf build
-.PHONY: clean
-
run: build/empty $(dest)
@set -xueo pipefail && \
img="$$(mktemp build/spectrum-rootfs.XXXXXXXXX.img)" && \
@@ -61,3 +55,5 @@ run: build/empty $(dest)
-drive file=/proc/self/fd/3,id=drive1,format=raw,if=none \
-drive file=/proc/self/fd/4,if=virtio,format=raw
.PHONY: run
+
+include $(ROOT)/lib/targets.mk
diff --git a/release/live/default.nix b/release/live/default.nix
index aa8dddd68497ced322961a6b551b73fe5ab4c475..403183226570ec318fbf7fddecfd52fd7c133c7e 100644
--- a/release/live/default.nix
+++ b/release/live/default.nix
@@ -28,6 +28,7 @@ stdenv.mkDerivation {
fileset = lib.fileset.intersection src (lib.fileset.unions [
./.
../../lib/common.mk
+ ../../lib/targets.mk
../../scripts/format-uuid.sh
../../scripts/make-gpt.sh
../../scripts/sfdisk-field.awk
diff --git a/scripts/genfiles.awk b/scripts/genfiles.awk
index 2d32016b5e6d6773685b71cc04e49cf24e842ba9..35e8538e75cc67f0a2ec8bc7b1be403a3f6604f0 100644
--- a/scripts/genfiles.awk
+++ b/scripts/genfiles.awk
@@ -6,6 +6,7 @@ BEGIN {
modes["120000"] = "symlink";
modes["100644"] = "regular";
modes["100755"] = "regular";
+ rc_count = rc_link_count = file_count = symlink_count = 0;
}
function fail(msg) {
@@ -34,6 +35,10 @@ filename !~ /^[[:alnum:]_./-]+$/ {
filename ~ /\.license$/ { next }
filename ~ /^image\/etc\/s6-rc\// {
+ if (mode == "symlink") {
+ rc_links[rc_link_count++] = filename;
+ next;
+ }
if (mode != "regular") {
fail("s6-rc-compile input '" filename "' isn't a regular file");
}
@@ -77,9 +82,15 @@ END {
for (array_index = 0; array_index < symlink_count; array_index += 1) {
printf " \\\n\t%s", symlinks[array_index];
}
- printf "\n\nS6_RC_FILES =";
+ printf "\n\n" \
+"S6_RC_FILES =";
for (array_index = 0; array_index < rc_count; array_index += 1) {
printf " \\\n\t%s", rc_files[array_index];
}
+ printf "\n\n" \
+"S6_RC_LINKS =";
+ for (array_index = 0; array_index < rc_link_count; array_index += 1) {
+ printf " \\\n\t%s", rc_links[array_index];
+ }
print "";
}
diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile
index e37fe8ca161106f9c6492b2f8b5d68d48ae945de..a1258432b39175839b7f2f6c2535e75197910679 100644
--- a/vm/sys/net/Makefile
+++ b/vm/sys/net/Makefile
@@ -3,7 +3,9 @@
.POSIX:
-include ../../../lib/common.mk
+ROOT = ../../..
+ROOT_FS = build
+include $(ROOT)/lib/common.mk
include file-list.mk
prefix = build/host
@@ -33,9 +35,6 @@ DIRS = dev etc/s6-linux-init/env proc run sys
BUILD_FILES = build/etc/s6-rc
-build/empty:
- mkdir -p $@
-
build/rootfs.erofs: ../../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty file-list.mk
set -euo pipefail; \
{ \
@@ -45,14 +44,6 @@ build/rootfs.erofs: ../../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(B
printf 'build/empty\n%s\n' $(DIRS) ;\
} | ../../../scripts/make-erofs.sh $@
-build/etc/s6-rc: $(S6_RC_FILES) file-list.mk
- mkdir -p $$(dirname $@)
- rm -rf $@
- set -uo pipefail && dir=$$(mktemp -d) && \
- { tar -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 3; } && \
- s6-rc-compile $@ $$dir; \
- exit=$$?; rm -r $$dir; exit $$exit
-
debug:
$(GDB) -q \
-ex 'set substitute-path .. $(LINUX_SRC)' \
@@ -108,6 +99,4 @@ run-crosvm: $(vmdir)/netvm/blk/root.img start-vhost-user-net
run: run-$(VMM)
.PHONY: run
-clean:
- rm -rf build
-.PHONY: clean
+include $(ROOT)/lib/targets.mk
diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix
index a722b021dc30f80e01dc626393a2f0311a75dc9d..df5fe7b1aa370c8b620dc18e565ca7fabde25f38 100644
--- a/vm/sys/net/default.nix
+++ b/vm/sys/net/default.nix
@@ -110,6 +110,7 @@ stdenvNoCC.mkDerivation {
fileset = lib.fileset.intersection src (lib.fileset.unions [
./.
../../../lib/common.mk
+ ../../../lib/targets.mk
../../../scripts/make-erofs.sh
../../../scripts/make-gpt.sh
../../../scripts/sfdisk-field.awk
diff --git a/vm/sys/net/file-list.mk b/vm/sys/net/file-list.mk
index c8d6d8ccb569c988f5de804244d800fe05784d05..e13bf1c32a290e6d0a9465d14bed8da2049a65f7 100644
--- a/vm/sys/net/file-list.mk
+++ b/vm/sys/net/file-list.mk
@@ -33,3 +33,5 @@ S6_RC_FILES = \
image/etc/s6-rc/nftables/up \
image/etc/s6-rc/ok-all/contents.d/mdevd-coldplug \
image/etc/s6-rc/ok-all/type
+
+S6_RC_LINKS =
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 03/22] tools: Add control group manager
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 01/22] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 02/22] scripts: Support symlinks in s6-rc-compile inputs Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 04/22] Documentation: Mention control groups Demi Marie Obenour
` (18 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
The cgroup-setup Rust program can create and purge cgroups. It can also
wait for one to become empty, spawn a program in a cgroup, and more. In
the future, it will also support cgroup-based resource control.
This program can also act as an s6 finish script. When invoked as such,
it automatically purges the correct cgroup. It also tells s6 to not
restart the service if it dumped core. Core dumps are often due to
memory corruption, and automatically restarting a service that dumped
core makes memory corruption attacks easier.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
.codespellrc | 2 +-
host/rootfs/default.nix | 6 +-
pkgs/default.nix | 1 +
tools/cgroup-setup/Cargo.lock | 67 ++++++
tools/cgroup-setup/Cargo.lock.license | 2 +
tools/cgroup-setup/Cargo.toml | 11 +
tools/cgroup-setup/default.nix | 18 ++
tools/cgroup-setup/src/cgroup.rs | 203 +++++++++++++++++
tools/cgroup-setup/src/main.rs | 406 ++++++++++++++++++++++++++++++++++
9 files changed, 712 insertions(+), 4 deletions(-)
diff --git a/.codespellrc b/.codespellrc
index d8023afc64ec44e98a88c5e397c8d5c681dde063..ae20da8309530759ac729689825a4afc683afa09 100644
--- a/.codespellrc
+++ b/.codespellrc
@@ -2,4 +2,4 @@
# SPDX-License-Identifier: CC0-1.0
[codespell]
-ignore-words-list = crate,passt,rouge,ser
+ignore-words-list = crate,passt,rouge,ser,WRONLY
diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix
index 4dcc10b9933cdfaf87f87417db1de0daa57b73f5..f161c26e8b3cc3a46d3321d88a52c446e2faac96 100644
--- a/host/rootfs/default.nix
+++ b/host/rootfs/default.nix
@@ -8,7 +8,7 @@ import ../../lib/call-package.nix (
}:
pkgsMusl.callPackage (
-{ spectrum-host-tools, spectrum-router
+{ spectrum-host-tools, spectrum-router, spectrum-cgroup-setup
, lib, stdenvNoCC, nixos, runCommand, writeClosure, erofs-utils, s6-rc
, btrfs-progs, bubblewrap, busybox, cloud-hypervisor, cosmic-files
, crosvm, cryptsetup, dejavu_fonts, dbus, execline, foot, fuse3
@@ -27,8 +27,8 @@ let
packages = [
btrfs-progs bubblewrap cloud-hypervisor cosmic-files crosvm cryptsetup dbus
execline fuse3 inotify-tools iproute2 jq kmod mdevd mount-flatpak s6
- s6-linux-init s6-rc shadow socat spectrum-host-tools spectrum-router
- virtiofsd xdg-desktop-portal-spectrum-host
+ s6-linux-init s6-rc shadow socat spectrum-cgroup-setup spectrum-host-tools
+ spectrum-router virtiofsd xdg-desktop-portal-spectrum-host
(foot.override { allowPgo = false; })
diff --git a/pkgs/default.nix b/pkgs/default.nix
index 4dbdfee8ba330f5ba5c9fedee2d1bb8a44af5722..22c0d782bf4b5467fe77d5a2d71fb8df3af3e756 100644
--- a/pkgs/default.nix
+++ b/pkgs/default.nix
@@ -51,6 +51,7 @@ let
driverSupport = true;
};
spectrum-router = self.callSpectrumPackage ../tools/router {};
+ spectrum-cgroup-setup = self.callSpectrumPackage ../tools/cgroup-setup {};
xdg-desktop-portal-spectrum-host =
self.callSpectrumPackage ../tools/xdg-desktop-portal-spectrum-host {};
diff --git a/tools/cgroup-setup/Cargo.lock b/tools/cgroup-setup/Cargo.lock
new file mode 100644
index 0000000000000000000000000000000000000000..fe967b3aa02c296c87b6b36ac59253dbe0a32de9
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.lock
@@ -0,0 +1,67 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 4
+
+[[package]]
+name = "bitflags"
+version = "2.11.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3"
+
+[[package]]
+name = "cgroup-setup"
+version = "0.0.0"
+dependencies = [
+ "libc",
+ "rustix",
+]
+
+[[package]]
+name = "errno"
+version = "0.3.14"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
+dependencies = [
+ "libc",
+ "windows-sys",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.186"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
+
+[[package]]
+name = "linux-raw-sys"
+version = "0.12.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
+
+[[package]]
+name = "rustix"
+version = "1.1.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
+dependencies = [
+ "bitflags",
+ "errno",
+ "libc",
+ "linux-raw-sys",
+ "windows-sys",
+]
+
+[[package]]
+name = "windows-link"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
+
+[[package]]
+name = "windows-sys"
+version = "0.61.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
+dependencies = [
+ "windows-link",
+]
diff --git a/tools/cgroup-setup/Cargo.lock.license b/tools/cgroup-setup/Cargo.lock.license
new file mode 100644
index 0000000000000000000000000000000000000000..aa108acd23886d8302eaf7babff90d1b08ae19fb
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.lock.license
@@ -0,0 +1,2 @@
+SPDX-License-Identifier: EUPL-1.2+
+SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
diff --git a/tools/cgroup-setup/Cargo.toml b/tools/cgroup-setup/Cargo.toml
new file mode 100644
index 0000000000000000000000000000000000000000..7ed6d6a0ea3bbfc4064b9f39383d0788c4bd84e5
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.toml
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: CC0-1.0
+# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+[package]
+name = "cgroup-setup"
+edition = "2024"
+
+[dependencies]
+libc = "0.2.177"
+rustix = { version = "1.1.2", features = ["fs"] }
diff --git a/tools/cgroup-setup/default.nix b/tools/cgroup-setup/default.nix
new file mode 100644
index 0000000000000000000000000000000000000000..9e716b8f270828b733f39961e5ca37a290a872b4
--- /dev/null
+++ b/tools/cgroup-setup/default.nix
@@ -0,0 +1,18 @@
+# SPDX-FileCopyrightText: 2024 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
+# SPDX-License-Identifier: MIT
+
+import ../../lib/call-package.nix (
+{ src, lib, rustPlatform }:
+
+rustPlatform.buildRustPackage {
+ name = "spectrum-cgroup-setup";
+
+ src = lib.fileset.toSource {
+ root = ../..;
+ fileset = lib.fileset.intersection src ./.;
+ };
+ sourceRoot = "source/tools/cgroup-setup";
+
+ cargoLock.lockFile = ./Cargo.lock;
+}) (_: {})
diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs
new file mode 100644
index 0000000000000000000000000000000000000000..eb9b892ab6f562cffa738abaf9c55dc18001012a
--- /dev/null
+++ b/tools/cgroup-setup/src/cgroup.rs
@@ -0,0 +1,203 @@
+// SPDX-License-Identifier: EUPL-1.2+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+use std::fs::File;
+use std::io::{Read as _, Seek as _, Write as _};
+use std::os::unix::prelude::*;
+
+use std::path::{Path, PathBuf};
+
+use rustix::fs::{AtFlags, XattrFlags};
+use rustix::path::Arg;
+use rustix::{
+ fs::{Mode, OFlags, ResolveFlags},
+ io::Errno,
+};
+
+pub(crate) struct LeafCgroup {
+ path: PathBuf,
+ fd: OwnedFd,
+}
+
+pub enum Access {
+ Read,
+ Write,
+}
+
+impl LeafCgroup {
+ pub fn enable_delegation(&self) -> Result<(), Errno> {
+ rustix::fs::fsetxattr(self.fd.as_fd(), c"user.delegate", b"1", XattrFlags::empty())
+ }
+
+ pub fn open_subtree(&self, path: &std::path::Path, access: Access) -> Result<OwnedFd, Errno> {
+ rustix::fs::openat2(
+ self.fd.as_fd(),
+ path,
+ OFlags::NOATIME
+ | OFlags::CLOEXEC
+ | OFlags::NOFOLLOW
+ | match access {
+ Access::Write => OFlags::WRONLY,
+ Access::Read => OFlags::RDONLY,
+ },
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )
+ }
+
+ pub fn wait_for_empty(&self, kill: bool) -> std::io::Result<()> {
+ let kill_fd = if kill {
+ Some(self.open_subtree(std::path::Path::new("cgroup.kill"), Access::Write)?)
+ } else {
+ None
+ };
+ let wait_file = self.open_subtree(std::path::Path::new("cgroup.events"), Access::Read)?;
+ let poll_fd = wait_file.as_raw_fd();
+ let mut wait_fd = File::from(wait_file);
+ if let Some(kill_fd) = kill_fd {
+ assert_eq!(rustix::io::write(kill_fd.as_fd(), b"1")?, 1, "kernel bug");
+ }
+ let mut fds = libc::pollfd {
+ fd: poll_fd,
+ events: libc::POLLPRI | libc::POLLERR,
+ revents: 0,
+ };
+ let mut v = vec![];
+ loop {
+ v.clear();
+ // SAFETY: FFI call, valid arguments, fds contains 1 element
+ if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
+ panic!("poll failed");
+ }
+ wait_fd
+ .seek(std::io::SeekFrom::Start(0))
+ .expect("Seek on control group file should succeed");
+ wait_fd
+ .read_to_end(&mut v)
+ .expect("reading from control group should work");
+ if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
+ break;
+ }
+ }
+ Ok(())
+ }
+}
+
+impl AsFd for LeafCgroup {
+ fn as_fd(&self) -> std::os::fd::BorrowedFd<'_> {
+ self.fd.as_fd()
+ }
+}
+
+impl LeafCgroup {
+ pub(crate) fn open_cgroup_at(&self, p: &Path) -> Result<Self, Errno> {
+ let fd = rustix::fs::openat2(
+ self.fd.as_fd(),
+ p,
+ OFlags::NOATIME
+ | OFlags::CLOEXEC
+ | OFlags::NOFOLLOW
+ | OFlags::RDONLY
+ | OFlags::DIRECTORY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )?;
+ Ok(Self {
+ fd,
+ path: self.path.join(p),
+ })
+ }
+ pub(crate) fn open_cgroup(fd: BorrowedFd, p: &Path) -> Result<Self, Errno> {
+ let fd = rustix::fs::openat2(
+ fd,
+ p,
+ OFlags::NOATIME
+ | OFlags::CLOEXEC
+ | OFlags::NOFOLLOW
+ | OFlags::RDONLY
+ | OFlags::DIRECTORY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS,
+ )?;
+ Ok(Self {
+ fd,
+ path: p.to_owned(),
+ })
+ }
+ pub(crate) fn make_child(&self, p: &Path) -> Result<(), Errno> {
+ let r = rustix::fs::mkdirat(
+ self.fd.as_fd(),
+ p,
+ Mode::RUSR
+ | Mode::WUSR
+ | Mode::XUSR
+ | Mode::RGRP
+ | Mode::XGRP
+ | Mode::ROTH
+ | Mode::XOTH,
+ );
+ if r == Err(Errno::EXIST) { Ok(()) } else { r }
+ }
+
+ pub(crate) fn delete_child(&self, path: &Path) -> Result<(), Errno> {
+ remove_all(100, self.as_fd(), &path.as_cow_c_str().unwrap())
+ }
+
+ pub(crate) fn write_cgroup_value(&self, name: &str, value: &str) -> Result<(), String> {
+ let path = Path::new(name);
+ let fd = rustix::fs::openat2(
+ self.as_fd(),
+ path,
+ OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::WRONLY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )
+ .map_err(|e| format!("Cannot open {:?}: {}", path, e))?;
+ File::from(fd)
+ .write_all(value.as_bytes())
+ .map_err(|e| format!("Cannot write {:?} to {:?}: {}", value, path, e))
+ }
+}
+
+fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> {
+ if remaining_depth < 1 {
+ panic!("control groups too deeply nested");
+ }
+ let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating");
+ while let Some(element) = d.next() {
+ let element = element.expect("Iterating through a cgroup directory failed?");
+ if element.file_type() != rustix::fs::FileType::Directory {
+ continue;
+ }
+
+ let remaining_depth = remaining_depth - 1;
+ let d: &rustix::fs::Dir = &d;
+ let dirfd = d.fd().unwrap();
+ let path = element.file_name();
+ remove_all(remaining_depth, dirfd, path)?;
+ }
+ Ok(())
+}
+
+fn remove_all(
+ remaining_depth: usize,
+ dirfd: BorrowedFd<'_>,
+ path: &std::ffi::CStr,
+) -> Result<(), Errno> {
+ if path == c"." || path == c".." {
+ return Ok(());
+ }
+ if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() {
+ return Ok(());
+ }
+ let fd = rustix::fs::openat2(
+ dirfd,
+ path,
+ OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )?;
+ remove_recursively(fd, remaining_depth)?;
+ rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?;
+ Ok(())
+}
diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
new file mode 100644
index 0000000000000000000000000000000000000000..5b2d2eb9831b0c635914a5e4dc90e79a262588a5
--- /dev/null
+++ b/tools/cgroup-setup/src/main.rs
@@ -0,0 +1,406 @@
+// SPDX-License-Identifier: EUPL-1.2+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+use std::{
+ ffi::{OsStr, OsString},
+ fs::File,
+ io::Read as _,
+ os::unix::prelude::*,
+ path::{Component, Path, PathBuf},
+};
+
+use rustix::{
+ fs::{FlockOperation, Mode, OFlags, ResolveFlags},
+ io::Errno,
+};
+
+use crate::cgroup::Access;
+
+mod cgroup;
+
+fn main() {
+ let mut args = std::env::args_os();
+ let Some(prog_name) = args.next() else {
+ eprintln!("No command line arguments (argv[0] is NULL)");
+ std::process::exit(1);
+ };
+ match main_(&prog_name, args) {
+ Ok(()) => {}
+ Err(e) => {
+ eprintln!("{prog_name:?}: {}", e);
+ std::process::exit(1);
+ }
+ }
+}
+
+fn main_(prog_name: &OsStr, mut args: std::env::ArgsOs) -> Result<(), String> {
+ let mut purge = false;
+ match prog_name
+ .as_bytes()
+ .split(|&b| b == b'/')
+ .next_back()
+ .unwrap()
+ {
+ b"finish" => {
+ return s6_finish(&mut args);
+ }
+ b"cgroup-setup" => {}
+ e => {
+ return Err(format!(
+ "must be invoked as \"cgroup-setup\" or \"finish\", got {:?}",
+ e
+ ));
+ }
+ };
+ let mut leaf = false;
+ let mut cgroup_relative_path;
+ let mut delegate = false;
+ let mut init_subtree = false;
+ let mut child_name: Option<&'static OsStr> = None;
+ let mut wait = None;
+ loop {
+ cgroup_relative_path = args.next();
+ let Some(ref arg_) = cgroup_relative_path else {
+ break;
+ };
+ let arg_ = arg_.as_bytes();
+ if arg_ == b"--" {
+ cgroup_relative_path = args.next();
+ break;
+ }
+ if !arg_.starts_with(b"-") {
+ break;
+ }
+
+ if !arg_.starts_with(b"--") {
+ return Err("takes no short options".to_owned());
+ }
+
+ match &arg_[2..] {
+ b"purge" => purge = true,
+ b"leaf" => leaf = true,
+ b"delegate" => delegate = true,
+ b"init-subtree" => init_subtree = true,
+ b"wait" => wait = Some(true),
+ b"no-wait" => wait = Some(false),
+ b"child-name" if child_name.is_none() => match args.next() {
+ Some(arg) => child_name = Some(arg.leak()),
+ None => return Err("--child-name: missing argument".to_owned()),
+ },
+ b"child-name" => return Err("--child-name: cannot be used twice".to_owned()),
+ arg => match str::from_utf8(arg) {
+ Ok(e) => return Err(format!("unknown long option {e:?}")),
+ Err(_) => return Err("long option isn't UTF-8".to_owned()),
+ },
+ }
+ }
+
+ let default_child_name = OsStr::from_bytes(b"$inner.service");
+
+ let child_name = Path::new(child_name.unwrap_or(default_child_name));
+
+ let Some(cgroup_relative_path) = cgroup_relative_path else {
+ return Err("have no positional arguments, expected at least 1".to_owned());
+ };
+
+ // The kernel doesn't care, but displaying messages does.
+ let cgroup_path = String::try_from(cgroup_relative_path.into_vec())
+ .map_err(|e| format!("non-UTF-8 cgroup path not supported (error is {e})"))?;
+
+ if cgroup_path.is_empty() {
+ return Err("cgroup name is empty".to_owned());
+ }
+ if cgroup_path == ".." {
+ return Err("cgroup name is ..".to_owned());
+ }
+ // If we aren't asked to create a child process, don't wait for existing
+ // processes to die unless explicitly asked to. Waiting for a cgroup
+ // we are in to be empty is a guaranteed deadlock.
+ let may_wait = cgroup_path != "." && cgroup_path != "/";
+ let wait = match wait {
+ Some(false) => false,
+ None => may_wait && args.len() > 0,
+ Some(true) if !may_wait => {
+ let msg = "Cannot wait for the program's own cgroup or \
+ root cgroup to be empty";
+ return Err(msg.to_owned());
+ }
+ Some(true) => true,
+ };
+
+ let params = CgroupParams {
+ purge,
+ leaf,
+ delegate,
+ init_subtree,
+ wait,
+ };
+
+ let (full_path, cgroup_target, cgroup) = cgroup_parse(cgroup_path)?;
+ if params.purge {
+ purge_cgroup(&cgroup, &cgroup_target)?;
+ }
+ let child = match cgroup.open_cgroup_at(&cgroup_target) {
+ Ok(child_cgroup) => child_cgroup,
+ Err(Errno::NOENT) => {
+ if let Err(e) = cgroup.make_child(&cgroup_target) {
+ return Err(format!("Cannot create child cgroup {full_path:?}: {e}"));
+ }
+ cgroup
+ .open_cgroup_at(&cgroup_target)
+ .map_err(|e| format!("Cannot open child cgroup {full_path:?}: {e}"))?
+ }
+ Err(other) => {
+ return Err(format!("Cannot open child cgroup {full_path:?}: {other}"));
+ }
+ };
+ if params.wait {
+ child
+ .wait_for_empty(false)
+ .map_err(|e| format!("Cannot wait for {full_path:?} to be empty: {e}"))?;
+ }
+ let pid = std::process::id().to_string();
+ if params.leaf {
+ // If we aren't delegating any cgroups, don't create a sub-cgroup.
+ child
+ .write_cgroup_value("cgroup.procs", &pid)
+ .map_err(|e| format!("Cannot write to {full_path:?}/cgroup.procs: {e}"))?;
+ } else {
+ child.make_child(child_name).map_err(|e| {
+ format!(
+ "Cannot create child cgroup {}/{}: {e}",
+ full_path.display(),
+ child_name.display()
+ )
+ })?;
+
+ // If the child process will need to manage cgroups itself, it will need
+ // to set up a sub-cgroup due to the "no internal processes" rule. It's
+ // simplest to just do it automatically.
+ let grandchild = child.open_cgroup_at(Path::new(child_name)).map_err(|e| {
+ format!(
+ "Cannot open child cgroup {}/{}: {e}",
+ full_path.display(),
+ child_name.display()
+ )
+ })?;
+ grandchild
+ .write_cgroup_value("cgroup.procs", &pid)
+ .map_err(|e| {
+ format!(
+ "Cannot write to {}/{}/cgroup.procs: {e}",
+ full_path.display(),
+ child_name.display()
+ )
+ })?;
+ }
+ if params.init_subtree {
+ enable_subtree_control(&cgroup)?;
+ }
+ if !params.leaf {
+ enable_subtree_control(&child)?;
+ }
+ if params.delegate {
+ child
+ .enable_delegation()
+ .map_err(|e| format!("Cannot enable cgroup delegation in {full_path:?}: {e}"))?;
+ }
+ let Some(program_name) = args.next() else {
+ return Ok(());
+ };
+ let e = std::process::Command::new(&program_name).args(args).exec();
+ Err(format!("Cannot spawn child {:?}: {}", program_name, e))
+}
+
+fn s6_finish(args: &mut std::env::ArgsOs) -> Result<(), String> {
+ if args.len() < 3 {
+ return Err(format!(
+ "s6 finish scripts take 3 arguments, got {}",
+ args.len()
+ ));
+ }
+ let status = parse_digit_string(&args.next().unwrap(), "exit status")?;
+ let signal = args.next().unwrap();
+ let signal = if status == 256 {
+ Some(parse_digit_string(&signal, "signal number")?)
+ } else {
+ None
+ };
+ let service = args
+ .next()
+ .unwrap()
+ .into_string()
+ .map_err(|e| format!("Service name {e:?} is not UTF-8"))?;
+ let (_full_path, cgroup_target, cgroup) = cgroup_parse(service)?;
+ let r = purge_cgroup(&cgroup, &cgroup_target);
+ if let Some(signal) = signal {
+ match signal as libc::c_int {
+ libc::SIGBUS
+ | libc::SIGFPE
+ | libc::SIGABRT
+ | libc::SIGTRAP
+ | libc::SIGSEGV
+ | libc::SIGILL => {
+ // Process *crashed*, indicating a *possible exploit attempt*.
+ // s6 should *not* restart it. This is distinct from a Rust panic,
+ // which is much less likely to indicate memory corruption.
+ if let Err(e) = r {
+ // do not panic on stderr write failure
+ eprintln!("Could not purge cgroup: {e}");
+ }
+ std::process::exit(125)
+ }
+ _ => return r,
+ }
+ }
+ r
+}
+
+fn parse_digit_string(digits: &OsStr, msg: &str) -> Result<u16, String> {
+ let checked = match str::from_utf8(digits.as_bytes()) {
+ Ok(s) => s,
+ Err(e) => return Err(format!("{msg} is not UTF-8: {e}")),
+ };
+ let r = checked
+ .parse::<u16>()
+ .map_err(|e| format!("{msg} {digits:?} is a bad 16-bit number: {e}"))?;
+ match checked.as_bytes() {
+ b"0" | [b'1'..=b'9', ..] => Ok(r),
+ [b'0', ..] => Err(format!("{msg} {} has a leading 0", digits.display())),
+ _ => Err(format!("{msg} {} starts with +", digits.display())),
+ }
+}
+
+fn cgroup_parse(mut arg: String) -> Result<(PathBuf, PathBuf, cgroup::LeafCgroup), String> {
+ let cgroup_root = rustix::fs::openat2(
+ rustix::fs::CWD,
+ Path::new("/sys/fs/cgroup"),
+ OFlags::DIRECTORY | OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW,
+ Mode::empty(),
+ ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_SYMLINKS,
+ )
+ .map_err(|e| format!("cannot open /sys/fs/cgroup: {e}"))?;
+ let cgroup_absolute = arg.starts_with("/");
+ let (full_path, mut cgroup_parent, cgroup_target) = if cgroup_absolute {
+ if arg.contains("//") {
+ return Err(format!("Cgroup path {arg:?} contains //"));
+ }
+ arg.remove(0);
+ let to_create = PathBuf::from(arg);
+ for component in to_create.components() {
+ match component {
+ Component::Prefix(_) => unreachable!("not present on Unix"),
+ Component::CurDir | Component::ParentDir => {
+ return Err("Cgroup path has . or .. components".to_owned());
+ }
+ Component::RootDir | Component::Normal(_) => {}
+ }
+ }
+ let Some(last_component) = to_create.file_name() else {
+ return Err(format!("Cgroup path {to_create:?} has no file name"));
+ };
+ if to_create.parent().is_none() {
+ return Err(format!("Cgroup path {to_create:?} has no parent name"));
+ }
+ let r = last_component.as_bytes().to_owned();
+ let mut prefix = to_create.clone();
+ prefix.pop();
+ (to_create, prefix, r)
+ } else {
+ if arg.is_empty() {
+ return Err("Cgroup path is empty".to_string());
+ }
+
+ let mut local_cgroup: Vec<u8> = std::fs::read("/proc/thread-self/cgroup")
+ .map_err(|e| format!("cannot read /proc/thread-self/cgroup: {e}"))?;
+ let local_cgroup_len = local_cgroup.len();
+ if local_cgroup_len < 5
+ || local_cgroup[..4] != *b"0::/"
+ || local_cgroup[local_cgroup_len - 1] != b'\n'
+ {
+ return Err(format!(
+ "Invalid contents {local_cgroup:?} of /proc/thread-self/cgroup - \
+ do you have cgroups v1 mounted instead of cgroups v2?"
+ ));
+ }
+
+ local_cgroup.copy_within(4..local_cgroup_len - 1, 0);
+ local_cgroup.truncate(local_cgroup_len - 5);
+ let total_path = PathBuf::from(<OsString as OsStringExt>::from_vec(local_cgroup));
+ let mut r = total_path.clone();
+ r.push(&arg);
+ (r, total_path, arg.into())
+ };
+ if cgroup_parent.file_name() == Some(OsStr::from_bytes(b"$inner.service")) {
+ cgroup_parent.pop();
+ }
+ if cgroup_parent.as_os_str().is_empty() {
+ cgroup_parent = ".".into();
+ }
+ let cgroup_target = OsString::from_vec(cgroup_target).into();
+ let cgroup = cgroup::LeafCgroup::open_cgroup(cgroup_root.as_fd(), &cgroup_parent)
+ .map_err(|e| format!("Failed to open {}: {e:?}", cgroup_parent.display()))?;
+ match rustix::fs::flock(cgroup.as_fd(), FlockOperation::LockExclusive) {
+ Ok(()) => {}
+ Err(e) => return Err(format!("Cannot lock cgroup: {e}")),
+ }
+ Ok((full_path, cgroup_target, cgroup))
+}
+
+struct CgroupParams {
+ purge: bool,
+ leaf: bool,
+ delegate: bool,
+ init_subtree: bool,
+ wait: bool,
+}
+
+fn enable_subtree_control(cgroup: &cgroup::LeafCgroup) -> Result<(), String> {
+ let mut buf = Vec::new();
+ File::from(
+ cgroup
+ .open_subtree(Path::new("cgroup.controllers"), Access::Read)
+ .map_err(|e| format!("Cannot open cgroup.controllers: {e}"))?,
+ )
+ .read_to_end(&mut buf)
+ .map_err(|e| format!("cannot read cgroup.controllers: {e}"))?;
+ let mut subtree = vec![];
+ if buf.ends_with(b"\n") {
+ buf.pop();
+ }
+ for controller in buf.split(|&b| b == b' ').filter(|e| !e.is_empty()) {
+ for &c in controller {
+ if c <= b' ' || c >= 0x7F {
+ return Err(format!("Bad byte {c} in cgroup.controllers"));
+ }
+ }
+ if !subtree.is_empty() {
+ subtree.push(b' ');
+ }
+ subtree.push(b'+');
+ subtree.extend_from_slice(controller);
+ }
+ if !subtree.is_empty() {
+ cgroup.write_cgroup_value("cgroup.subtree_control", str::from_utf8(&subtree).unwrap())?;
+ }
+ Ok(())
+}
+
+fn purge_cgroup(cgroup: &cgroup::LeafCgroup, cgroup_target: &Path) -> Result<(), String> {
+ let child = match cgroup.open_cgroup_at(cgroup_target) {
+ Ok(child_cgroup) => child_cgroup,
+ Err(Errno::NOENT) => return Ok(()),
+ Err(other) => {
+ return Err(format!(
+ "Cannot open child cgroup {cgroup_target:?}: {other}"
+ ));
+ }
+ };
+ child
+ .wait_for_empty(true)
+ .map_err(|e| format!("Cannot kill programs in {cgroup_target:?}: {e}"))?;
+
+ cgroup
+ .delete_child(cgroup_target)
+ .map_err(|e| format!("Delete child cgroup {cgroup_target:?}: {e}"))
+}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 04/22] Documentation: Mention control groups
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (2 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 03/22] tools: Add control group manager Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 05/22] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
` (17 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Documentation/doc/development/control-groups.adoc | 82 +++++++++++++++++++++++
1 file changed, 82 insertions(+)
diff --git a/Documentation/doc/development/control-groups.adoc b/Documentation/doc/development/control-groups.adoc
new file mode 100644
index 0000000000000000000000000000000000000000..9866761723ba8c77f9242d4ffdabaebad3f45aa4
--- /dev/null
+++ b/Documentation/doc/development/control-groups.adoc
@@ -0,0 +1,82 @@
+= Control groups in Spectrum
+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0
+
+Linux control groups (cgroups) can be used for several purposes:
+
+1. They allow waiting for a group of processes to exit.
+2. They allow terminating a group of processes.
+3. They allow limiting a group of processes' access to resources.
+
+Spectrum currently uses the first two. The third is not yet used,
+but will be in the future.
+
+== Control Group Hierarchy
+
+Spectrum uses the following cgroup hierarchy:
+
+1. There is a `/vm-services.slice` cgroup that contains all the per-VM
+ services on the system.
+2. The per-VM services for each VM are under `/vm-services.slice/vm-${VM}.slice`,
+ where `${VM}` is replaced by the VM's ID.
+3. Each per-VM service is under `/vm-services.slice/vm-${VM}.slice/${SERVICE_NAME}`,
+ where `${VM}` is replaced by the VM's ID and `${SERVICE_NAME}` is replaced by
+ the name of the service.
+4. The VMM runs under `/vm-services.slice/vm-${VM}.slice/vmm`.
+
+If a cgroup contains child cgroups, it likely contains a `$inner.service`
+cgroup. This is where programs that would otherwise run in the cgroup itself
+are placed. Generally, these programs are instances of `s6-svscan` and/or
+`s6-supervise`.
+
+== Using Control Groups
+
+When adding a new s6 service, one should carefully consider whether it
+should be placed in a control group. Most services should be placed in
+a control group, with only a few exceptions:
+
+1. Services, such as `getty`, that spawn background processes.
+2. Loggers.
+3. Trivial services that don't do anything.
+
+Generally, it's best to set the control group up as the first thing
+the service does. To do that, use `cgroup-setup --leaf -- $1 COMMAND_LINE`,
+where `$1` should be the service name and `COMMAND_LINE` is the program
+to run in a cgroup.
+
+If you use execline for your run script, this is as simple as:
+
+.run
+....
+#!/bin/execlineb -WS1
+
+cgroup-setup --leaf -- $1
+# rest of script comes here
+....
+
+If the services exits, it's usually best to terminate any programs left
+behind with SIGKILL and remove the control group. To do that, make
+the `finish` script be a symbolic link to `/usr/bin/cgroup-setup`.
+`cgroup-setup` recognizes when it has been invoked as `finish` and
+purges the cgroup automatically.
+
+When invoked as `finish`, `cgroup-setup` checks if the service exited
+due to a signal that caused it to dump core. If it did, `cgroup-setup`
+exits with status 125, ensuring that `s6-supervise` will *not* restart it.
+This is intentional: if a service crashes due to a fatal signal, this
+is possibly a sign of memory corruption. Restarting the service in this
+case can turn an unreliable memory corruption exploit into a reliable one.
+Rust panics do not cause core dumps, so the service will be restarted
+afterwards.
+
+== Future plans
+
+Control groups are designed around a single writer process controlling each
+of them. Many Linux distros use systemd for this, but Spectrum doesn't use
+systemd. The only persistent per-service process is s6-supervise, but that
+doesn't have control group support.
+
+Instead, the plan is to have a database containing this information.
+Whether this will be in the `data/` subdirectory of the service directory
+or a separate system-wide database has not yet been determined.
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 05/22] Mount cgroup2 filesystem at /sys/fs/cgroup
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (3 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 04/22] Documentation: Mention control groups Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 06/22] host/rootfs: Add helper program for per-VM services Demi Marie Obenour
` (16 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Needed for cgroups to work at all.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/fstab | 1 +
1 file changed, 1 insertion(+)
diff --git a/host/rootfs/image/etc/fstab b/host/rootfs/image/etc/fstab
index 4d52a1da8c6d673cbbcfe2e9c0ec93085f06c40f..cab97cf9c49f864845492a1e524d43b7d546317f 100644
--- a/host/rootfs/image/etc/fstab
+++ b/host/rootfs/image/etc/fstab
@@ -6,3 +6,4 @@ tmpfs /dev/shm tmpfs nosuid,nodev 0 0
tmpfs /media tmpfs nosuid,nodev,noexec,nosymfollow,mode=755 0 0
sysfs /sys sysfs nosuid,nodev,noexec 0 0
tmpfs /tmp tmpfs nosuid,nodev 0 0
+cgroup2 /sys/fs/cgroup cgroup2 nosuid,nodev,noexec,nosymfollow 0 0
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 06/22] host/rootfs: Add helper program for per-VM services
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (4 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 05/22] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 07/22] host/rootfs: Enable controllers in sub-cgroups Demi Marie Obenour
` (15 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
The program handles cgroups and common substitutions. This is better
than repeating the boilerplate multiple times.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
host/rootfs/image/usr/bin/vm-service-run | 34 ++++++++++++++++++++++++++++++++
2 files changed, 35 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 7e44fc612a5a9ac8767be9148cdc71869ef16964..9375a05a6847ed2b799a13d7be74313459fc9f0e 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -65,6 +65,7 @@ FILES = \
image/usr/bin/spectrum-update \
image/usr/bin/vm-console \
image/usr/bin/vm-import \
+ image/usr/bin/vm-service-run \
image/usr/bin/vm-start \
image/usr/bin/vm-stop \
image/usr/bin/xdg-open \
diff --git a/host/rootfs/image/usr/bin/vm-service-run b/host/rootfs/image/usr/bin/vm-service-run
new file mode 100755
index 0000000000000000000000000000000000000000..c788574cf95ec7fd0d93b656557f85fa80b31690
--- /dev/null
+++ b/host/rootfs/image/usr/bin/vm-service-run
@@ -0,0 +1,34 @@
+#!/usr/bin/execlineb -WS1
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+# Substitute all of the needed environment variables
+# into both this script and its arguments
+# (the subsequent command to run). This substitutes
+# the *caller*'s arguments.
+multisubstitute {
+ importas -iS VM
+ importas -iS "#"
+ importas -iS "1"
+ importas -iS WAYLAND_DISPLAY
+}
+
+# Check that the VM name and service are reasonable.
+# Then run the cgroup-setup program and the provided
+# command line. Avoid premature substitution by
+# escaping ${#} and ${1} with backslashes. Otherwise,
+# they would be values for this script, not for the caller.
+case -- "\\${#}@${VM}@\\${1}" {
+ # s6-supervise might start passing extra arguments in the future, but
+ # 0 arguments is not okay.
+ "[1-9][0-9]*@[A-Za-z0-9_][A-Za-z0-9_.-]*@[A-Za-z_][A-Za-z0-9_-]*" {
+ # The caller has been migrated from using -WS1 to using -Wp.
+ # Pop the environment to preserve the original behavior.
+ emptyenv -P cgroup-setup --leaf -- "\\${1}" $@
+ }
+}
+fdmove -c 1 2
+if { printf "Wrong VM name %s, \
+parent argument count %s, or parent argument %s\n"
+ $VM "\\$#" "\\${1}" }
+exit 100
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 07/22] host/rootfs: Enable controllers in sub-cgroups
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (5 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 06/22] host/rootfs: Add helper program for per-VM services Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 08/22] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
` (14 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Actually use the cgroup manager for the first time.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/init | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
index 8ec7859ab00e24007a93d4000c8f34e353de50ae..5d31a23bae2f29f35bfeced68242dededf6ae0c7 100755
--- a/host/rootfs/image/etc/init
+++ b/host/rootfs/image/etc/init
@@ -12,4 +12,8 @@ if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
if { /bin/mount --make-shared / }
if { /bin/mount -a --mkdir }
+# Enable subtree control of all cgroups and move
+# process to a child cgroup.
+/usr/bin/cgroup-setup --init-subtree .
+
/bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 08/22] host/rootfs: Add comments where cgroups are intentionally not used
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (6 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 07/22] host/rootfs: Enable controllers in sub-cgroups Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 09/22] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
` (13 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Most services should use cgroups, but some do not.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run | 1 +
.../etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run | 1 +
.../image/etc/s6-linux-init/run-image/service/serial-getty/template/run | 1 +
8 files changed, 8 insertions(+)
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
index 4d2cd7172e887e90d9ce87489cdacf41e51f77e8..5b2421a3dedf7c282ea818f4064535d0fc29d511 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty2 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
index 06abf5422365ff4edc9379e5cbcab716e6830399..3ca382f76368174ee1f99ea880827958c57b602c 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty3 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
index 3dd235fc93d403fc8971e923ec62cb025684b67a..983bd82927e468bab6ae20f61411680d173e4924 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty4 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
index 86b9a1ef70300de52f33a84cbfac22a77e00df36..fb55327dff5f189f543fb3629702caad9ff247e2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
@@ -2,6 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# cgroups make no sense here
s6-ipcserver-socketbinder -a 0700 /run/root-terminal
if { chown wayland /run/root-terminal }
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
index 327f4ccc5056e7ef87c089e666090307845cae3a..a6733604e91692f8440fd68efeb1a6b7378be167 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# cgroups make no sense for shutdownd
s6-linux-init-shutdownd -Bc /etc/s6-linux-init
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
index 8f7bd889e50d76a49b98d8ba9e2c6172f287277c..cf3f5c7b499d997fd426062651eca64b3accc146 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
@@ -2,5 +2,6 @@
# SPDX-License-Identifier: ISC
# SPDX-FileCopyrightText: Copyright (c) 2015-2024 Laurent Bercot <ska-skaware@skarnet.org>
+# cgroups make no sense for a logger
redirfd -rnb 0 fifo
s6-log -bpd3 -- T /run/log
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
index 78f794202bf174f3c036f3e20755ac087a988277..8fb1b6779c0bb1e3cdf67d1fa14722caacc25994 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# No need for a cgroup here.
s6-svscan -d3 instance
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
index f3ed9a7f08e60206afd222f4fc02c5fb19527561..775b828c72b9d73bef8a3355fb3ae0536975f474 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021, 2024 Alyssa Ross <hi@alyssa.is>
+# Don't put this in a cgroup. We want nohup to work.
getty -i -n -l /etc/login 0,115200,57600,38400,9600 $1 dumb
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 09/22] host/rootfs: serial-getty-generator: Use cgroups
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (7 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 08/22] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 10/22] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
` (12 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Avoids concurrent execution and leaked child processes.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../etc/s6-linux-init/run-image/service/serial-getty-generator/finish | 1 +
.../etc/s6-linux-init/run-image/service/serial-getty-generator/run | 4 +++-
3 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 9375a05a6847ed2b799a13d7be74313459fc9f0e..6144ca829abe4f561b95504b9f77a893c30a37cb 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -77,6 +77,7 @@ LINKS = \
image/etc/group \
image/etc/passwd \
image/etc/s6-linux-init/run-image/opengl-driver \
+ image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish \
image/etc/s6-linux-init/run-image/service/vmm/template/run \
image/lib \
image/sbin \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
new file mode 120000
index 0000000000000000000000000000000000000000..9e39002e3ae4063bdef05676efb55416020cf7aa
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
@@ -0,0 +1 @@
+/usr/bin/cgroup-setup
\ No newline at end of file
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
index bf66ab0878fc6d8e77ca71a4a89e50c139a6bb11..a1846a29cf55892127feb7ecfc0adcd12f0497bd 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
@@ -1,7 +1,9 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+# avoid concurrent operation
+cgroup-setup --leaf -- $1
piperw 3 4
background {
fdclose 3
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 10/22] host/rootfs: Set up parent cgroup for all per-VM services
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (8 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 09/22] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 11/22] host/rootfs: Create per-VM cgroup for all of the VM's services Demi Marie Obenour
` (11 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run | 3 +++
1 file changed, 3 insertions(+)
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
index 78f794202bf174f3c036f3e20755ac087a988277..430065b1efae8ce1fe7c7a07b67920daa88bf98c 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
@@ -1,5 +1,8 @@
#!/bin/execlineb -WP
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# Set up the parent cgroup of all VMMs and VM services
+cgroup-setup --no-wait -- /vm-services.slice
s6-svscan -d3 instance
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 11/22] host/rootfs: Create per-VM cgroup for all of the VM's services
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (9 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 10/22] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 12/22] host/rootfs: run-vmm: Create per-VM cgroup Demi Marie Obenour
` (10 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../etc/s6-linux-init/run-image/service/vm-services/template/finish | 1 +
.../etc/s6-linux-init/run-image/service/vm-services/template/run | 4 ++++
3 files changed, 6 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 6144ca829abe4f561b95504b9f77a893c30a37cb..2b0a254ab1be1dcd7bcaafcffdb0b8aebb5b3697 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -78,6 +78,7 @@ LINKS = \
image/etc/passwd \
image/etc/s6-linux-init/run-image/opengl-driver \
image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
image/etc/s6-linux-init/run-image/service/vmm/template/run \
image/lib \
image/sbin \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
new file mode 120000
index 0000000000000000000000000000000000000000..9e39002e3ae4063bdef05676efb55416020cf7aa
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
@@ -0,0 +1 @@
+/usr/bin/cgroup-setup
\ No newline at end of file
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
index bf90e99eef4106a9ad17dfba310b6edb3eaa64a3..d3a5f3490e447448929383a6b5d36dcef723712a 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
@@ -2,6 +2,10 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# Move into the cgroup with all per-VM services
+# Do not wait for the existing processes in the slice to finish.
+cgroup-setup --no-wait -- /vm-services.slice/vm-${1}.slice
+
export VM $1
s6-svscan -d3 data/service
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 12/22] host/rootfs: run-vmm: Create per-VM cgroup
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (10 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 11/22] host/rootfs: Create per-VM cgroup for all of the VM's services Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 13/22] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
` (9 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/usr/bin/run-vmm | 2 ++
1 file changed, 2 insertions(+)
diff --git a/host/rootfs/image/usr/bin/run-vmm b/host/rootfs/image/usr/bin/run-vmm
index 6967dc5586335d999f8030371fe0f38502409919..607f798dbb5eee4b30bcf18f80d3cb5ef2857a98 100755
--- a/host/rootfs/image/usr/bin/run-vmm
+++ b/host/rootfs/image/usr/bin/run-vmm
@@ -5,6 +5,7 @@
if {
backtick -D "" mnt {
importas -Siu 1
+ if { cgroup-setup --no-wait /vm-services.slice/vm-${1}.slice }
nsenter --mount=/run/vm/by-id/${1}/ns/mnt
findmnt -no FSTYPE,SOURCE /run/fs/${1}/disk
}
@@ -86,6 +87,7 @@ redirfd -r 0 /dev/null
s6-softlimit -H -l 18446744073709551615
if { udevadm wait /dev/kvm }
+cgroup-setup --leaf --no-wait /vm-services.slice/vm-${1}.slice/vmm
s6-envuidgid vmm-${1}
s6-applyuidgid -Uz
bwrap
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 13/22] host/rootfs: run-appimage: Purge the per-VM cgroup
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (11 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 12/22] host/rootfs: run-vmm: Create per-VM cgroup Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 14/22] host/rootfs: run-flatpak: " Demi Marie Obenour
` (8 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/usr/bin/run-appimage | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/image/usr/bin/run-appimage b/host/rootfs/image/usr/bin/run-appimage
index 672ec22d4b815fb9c2940e13ae08ed4093baee4e..869375fb63c780cb24ecfe4a272159d543a8adff 100755
--- a/host/rootfs/image/usr/bin/run-appimage
+++ b/host/rootfs/image/usr/bin/run-appimage
@@ -89,7 +89,10 @@ fdclose 4
foreground { run-vmm $id }
fdclose 3
-if { s6-instance-delete /run/service/vm-services $id }
+if {
+ if { s6-instance-delete /run/service/vm-services $id }
+ cgroup-setup --purge /vm-services.slice/vm-${id}.slice
+}
if { umount -R /run/vm/by-id/${id}/ns }
rm -r -- $diskdir /run/vm/by-id/${id} /run/configs/${id}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 14/22] host/rootfs: run-flatpak: Purge the per-VM cgroup
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (12 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 13/22] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 15/22] host/rootfs: dbus: Run in cgroup Demi Marie Obenour
` (7 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/usr/bin/run-flatpak | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/image/usr/bin/run-flatpak b/host/rootfs/image/usr/bin/run-flatpak
index fafbc82ff6966d8be9c8083a17618f1fb192bd07..f435b4f4be9ce764a60936a60d25ee79ace7f982 100755
--- a/host/rootfs/image/usr/bin/run-flatpak
+++ b/host/rootfs/image/usr/bin/run-flatpak
@@ -87,9 +87,9 @@ if {
fdclose 4
foreground { run-vmm $id }
+ if { s6-instance-delete -- /run/service/vm-services $id }
+ cgroup-setup --purge /vm-services.slice/vm-${id}.slice
}
-if { s6-instance-delete -- /run/service/vm-services $id }
-
if { umount -R /run/vm/by-id/${id}/ns }
rm -r -- $diskdir /run/vm/by-id/${id} /run/configs/${id}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 15/22] host/rootfs: dbus: Run in cgroup
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (13 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 14/22] host/rootfs: run-flatpak: " Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 16/22] host/rootfs: vhost-user-fs: " Demi Marie Obenour
` (6 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../run-image/service/vm-services/template/data/service/dbus/finish | 1 +
.../run-image/service/vm-services/template/data/service/dbus/run | 4 ++--
3 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 2b0a254ab1be1dcd7bcaafcffdb0b8aebb5b3697..96960f7e58bef0ed081f4b2ef060765e412a871f 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -78,6 +78,7 @@ LINKS = \
image/etc/passwd \
image/etc/s6-linux-init/run-image/opengl-driver \
image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
image/etc/s6-linux-init/run-image/service/vmm/template/run \
image/lib \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish
new file mode 120000
index 0000000000000000000000000000000000000000..9e39002e3ae4063bdef05676efb55416020cf7aa
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish
@@ -0,0 +1 @@
+/usr/bin/cgroup-setup
\ No newline at end of file
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
index 331d9a57e8d99eb8d6f7b33031cd130b410a6157..38d5836acb9db2d8a8bee25e3e9527a9061ed837 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
@@ -1,8 +1,8 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -Wp
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
-importas -i VM VM
+vm-service-run
if {
redirfd -w 1 data/dbus.conf
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 16/22] host/rootfs: vhost-user-fs: Run in cgroup
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (14 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 15/22] host/rootfs: dbus: Run in cgroup Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 17/22] host/rootfs: vhost-user-gpu: " Demi Marie Obenour
` (5 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Use the vm-service-run script.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../service/vm-services/template/data/service/vhost-user-fs/finish | 1 +
.../service/vm-services/template/data/service/vhost-user-fs/run | 5 +++--
3 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 96960f7e58bef0ed081f4b2ef060765e412a871f..31a536c5174c20e8a566b36471372fd97e579bd1 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -79,6 +79,7 @@ LINKS = \
image/etc/s6-linux-init/run-image/opengl-driver \
image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
image/etc/s6-linux-init/run-image/service/vmm/template/run \
image/lib \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish
new file mode 120000
index 0000000000000000000000000000000000000000..9e39002e3ae4063bdef05676efb55416020cf7aa
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish
@@ -0,0 +1 @@
+/usr/bin/cgroup-setup
\ No newline at end of file
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
index cb059aebc8dfe2c8a5b059d5bfea884de53bf922..66aeef5d1bd7399c9969e38ce1a9ad32f4ba56b8 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
@@ -1,10 +1,11 @@
-#!/bin/execlineb -WS1
+#!/bin/execlineb -Wp
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2021-2025 Alyssa Ross <hi@alyssa.is>
+vm-service-run
+
s6-ipcserver-socketbinder -a 0700 -B env/virtiofsd.sock
-importas -i VM VM
if { chown vmm-${VM} env/virtiofsd.sock }
if { fdmove 1 3 echo }
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 17/22] host/rootfs: vhost-user-gpu: Run in cgroup
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (15 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 16/22] host/rootfs: vhost-user-fs: " Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 18/22] host/rootfs: xdg-desktop-portal-spectrum-host: " Demi Marie Obenour
` (4 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Use vm-service-run for this.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../vm-services/template/data/service/vhost-user-gpu/finish | 1 +
.../service/vm-services/template/data/service/vhost-user-gpu/run | 9 +++------
3 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 31a536c5174c20e8a566b36471372fd97e579bd1..15e92bd32887beb547fc88d431c36b09b220c3ad 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -80,6 +80,7 @@ LINKS = \
image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
image/etc/s6-linux-init/run-image/service/vmm/template/run \
image/lib \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish
new file mode 120000
index 0000000000000000000000000000000000000000..9e39002e3ae4063bdef05676efb55416020cf7aa
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish
@@ -0,0 +1 @@
+/usr/bin/cgroup-setup
\ No newline at end of file
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
index c3bfafe02ec6d2ee418fe56b033bbc4c7a73b186..49978bf9e4e600198cd3f8704c33229518a8fa60 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
@@ -1,14 +1,11 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -Wp
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
-s6-ipcserver-socketbinder -a 0700 -b 1 env/crosvm.sock
+vm-service-run
-multisubstitute {
- importas -Siu VM
- importas -Si WAYLAND_DISPLAY
-}
+s6-ipcserver-socketbinder -a 0700 -b 1 env/crosvm.sock
if { chown vmm-${VM} env/crosvm.sock }
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 18/22] host/rootfs: xdg-desktop-portal-spectrum-host: Run in cgroup
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (16 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 17/22] host/rootfs: vhost-user-gpu: " Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 19/22] host/rootfs: systemd-udevd: " Demi Marie Obenour
` (3 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../template/data/service/xdg-desktop-portal-spectrum-host/finish | 1 +
.../template/data/service/xdg-desktop-portal-spectrum-host/run | 4 ++--
3 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 15e92bd32887beb547fc88d431c36b09b220c3ad..cf2b05c3265b27734dd29a72c5f687a46e16cae2 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -81,6 +81,7 @@ LINKS = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
image/etc/s6-linux-init/run-image/service/vmm/template/run \
image/lib \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish
new file mode 120000
index 0000000000000000000000000000000000000000..9e39002e3ae4063bdef05676efb55416020cf7aa
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish
@@ -0,0 +1 @@
+/usr/bin/cgroup-setup
\ No newline at end of file
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
index 909957b817136149dd01c3b98847238ecda8032d..d9df94b6c6b73d0ee385864691524cedd887a2e2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
@@ -1,8 +1,8 @@
-#!/bin/execlineb -WS1
+#!/bin/execlineb -Wp
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
-importas -i VM VM
+vm-service-run
export DBUS_SESSION_BUS_ADDRESS unix:path=/run/portal-bus/${VM}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 19/22] host/rootfs: systemd-udevd: Run in cgroup
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (17 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 18/22] host/rootfs: xdg-desktop-portal-spectrum-host: " Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 20/22] host/rootfs: weston: " Demi Marie Obenour
` (2 subsequent siblings)
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 3 ++-
host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 1 +
host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +++--
3 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index cf2b05c3265b27734dd29a72c5f687a46e16cae2..28866876e7a936e92f97e87b5108f6ad50b0005a 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -119,4 +119,5 @@ S6_RC_FILES = \
image/etc/s6-rc/weston/run \
image/etc/s6-rc/weston/type
-S6_RC_LINKS =
+S6_RC_LINKS = \
+ image/etc/s6-rc/systemd-udevd/finish
diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/finish b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
new file mode 120000
index 0000000000000000000000000000000000000000..9e39002e3ae4063bdef05676efb55416020cf7aa
--- /dev/null
+++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
@@ -0,0 +1 @@
+/usr/bin/cgroup-setup
\ No newline at end of file
diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/run b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
old mode 100644
new mode 100755
index aec6444e951503eae988e666b77fda8f2ae33d72..901f3e6667a8de04a6af39945406a1757455f089
--- a/host/rootfs/image/etc/s6-rc/systemd-udevd/run
+++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
@@ -1,7 +1,8 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
-s6-setlock /run/sd-notify-wrapper/systemd-udevd.lock
+
+cgroup-setup --delegate --child-name udev -- $1
s6-ipcserver-socketbinder -b0 -m -a 0600 /run/sd-notify-wrapper/systemd-udevd.sock
background -d {
fdmove 1 3
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 20/22] host/rootfs: weston: Run in cgroup
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (18 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 19/22] host/rootfs: systemd-udevd: " Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 21/22] host/rootfs: spectrum-router: " Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 22/22] host/rootfs: vm-import: Use elglob -w Demi Marie Obenour
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 3 ++-
host/rootfs/image/etc/s6-rc/weston/finish | 1 +
host/rootfs/image/etc/s6-rc/weston/run | 9 +++++++--
3 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 28866876e7a936e92f97e87b5108f6ad50b0005a..65fd1cf5071f9eba2d989d802f7641e74816816e 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -120,4 +120,5 @@ S6_RC_FILES = \
image/etc/s6-rc/weston/type
S6_RC_LINKS = \
- image/etc/s6-rc/systemd-udevd/finish
+ image/etc/s6-rc/systemd-udevd/finish \
+ image/etc/s6-rc/weston/finish
diff --git a/host/rootfs/image/etc/s6-rc/weston/finish b/host/rootfs/image/etc/s6-rc/weston/finish
new file mode 120000
index 0000000000000000000000000000000000000000..9e39002e3ae4063bdef05676efb55416020cf7aa
--- /dev/null
+++ b/host/rootfs/image/etc/s6-rc/weston/finish
@@ -0,0 +1 @@
+/usr/bin/cgroup-setup
\ No newline at end of file
diff --git a/host/rootfs/image/etc/s6-rc/weston/run b/host/rootfs/image/etc/s6-rc/weston/run
old mode 100644
new mode 100755
index 8b065ee9a535d0a08e9255900c73b88a0c376d4b..a306d9903da6da920efa55eccdfb4f3d14698a79
--- a/host/rootfs/image/etc/s6-rc/weston/run
+++ b/host/rootfs/image/etc/s6-rc/weston/run
@@ -1,8 +1,13 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -Wp
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2021, 2025 Alyssa Ross <hi@alyssa.is>
-importas -Siu WAYLAND_DISPLAY
+multisubstitute {
+ importas -Siu WAYLAND_DISPLAY
+ importas -Si 1
+}
+
+cgroup-setup --leaf --no-wait -- $1
piperw 4 3
background {
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 21/22] host/rootfs: spectrum-router: Run in cgroup
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (19 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 20/22] host/rootfs: weston: " Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 22/22] host/rootfs: vm-import: Use elglob -w Demi Marie Obenour
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../service/vm-services/template/data/service/spectrum-router/finish | 1 +
.../service/vm-services/template/data/service/spectrum-router/run | 4 ++--
3 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 65fd1cf5071f9eba2d989d802f7641e74816816e..ce16d9a514a0750805e0badc49fe7efb9586339c 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -79,6 +79,7 @@ LINKS = \
image/etc/s6-linux-init/run-image/opengl-driver \
image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish
new file mode 120000
index 0000000000000000000000000000000000000000..9e39002e3ae4063bdef05676efb55416020cf7aa
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish
@@ -0,0 +1 @@
+/usr/bin/cgroup-setup
\ No newline at end of file
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
index 3de58c27facc78c48176d8f9a6dd1827ac926f1a..dfb3b8e77c8b3a02b83f4dfeb19b3dc7a816ac45 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
@@ -1,9 +1,9 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -Wp
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
-importas -i VM VM
+vm-service-run
s6-ipcserver-socketbinder -a 0770 /run/vm/by-id/${VM}/router-driver.sock
fdmove -c 3 0
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v3 22/22] host/rootfs: vm-import: Use elglob -w
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
` (20 preceding siblings ...)
2026-07-11 20:12 ` [PATCH v3 21/22] host/rootfs: spectrum-router: " Demi Marie Obenour
@ 2026-07-11 20:12 ` Demi Marie Obenour
21 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-11 20:12 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/usr/bin/vm-import | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/host/rootfs/image/usr/bin/vm-import b/host/rootfs/image/usr/bin/vm-import
index ade1ee86527813b171e458961fadb5fb936e358d..ffaff7e5a7cb47b9aedcebca84a62634bd5256e3 100755
--- a/host/rootfs/image/usr/bin/vm-import
+++ b/host/rootfs/image/usr/bin/vm-import
@@ -3,7 +3,7 @@
# SPDX-FileCopyrightText: 2023-2024 Alyssa Ross <hi@alyssa.is>
cd $2
-elglob -0 names *
+elglob -w0 names *
forx -po0 -E name { $names }
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* Re: [PATCH v3 01/22] host/rootfs: Mount filesystems before s6-rc-init
2026-07-11 20:12 ` [PATCH v3 01/22] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
@ 2026-07-13 9:39 ` Alyssa Ross
2026-07-13 17:27 ` Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
1 sibling, 1 reply; 132+ messages in thread
From: Alyssa Ross @ 2026-07-13 9:39 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 2678 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> cgroup-setup is written in Rust and uses the Rust standard library.
> It also relies on cgroupfs being mounted at /sys/fs/cgroup.
> Furthermore, it runs very, *very* early on, since it sets up the cgroups
> in which s6-svscan and s6-supervise processes run. This allows setting
> up sub-cgroups without having to worry about name collisions.
>
> While it would be possible to only mount the bare minimum necessary,
> it's simpler to just mount everything before starting any services.
> These mounts are all pseudo-filesystems or tmpfs instances, and none
> have any dependencies.
>
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> host/rootfs/image/etc/init | 11 ++++++++++-
> host/rootfs/image/etc/s6-linux-init/scripts/rc.init | 9 +--------
> 2 files changed, 11 insertions(+), 9 deletions(-)
>
> diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
> index db8a6d9c747e3d212c21b6c983c8f7e299822abe..8ec7859ab00e24007a93d4000c8f34e353de50ae 100755
> --- a/host/rootfs/image/etc/init
> +++ b/host/rootfs/image/etc/init
> @@ -1,6 +1,15 @@
> #!/bin/execlineb -Ws0
> # SPDX-License-Identifier: EUPL-1.2+
> -# SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is>
> +# SPDX-FileCopyrightText: 2020-2022, 2024 Alyssa Ross <hi@alyssa.is>
>
> /bin/setpriv --no-new-privs --
> +
> +if { /bin/ln -s /proc/self/fd /dev }
> +if { /bin/ln -s /proc/self/fd/0 /dev/stdin }
> +if { /bin/ln -s /proc/self/fd/1 /dev/stdout }
> +if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
I guess the idea is that the Rust standard library might want these?
> +
> +if { /bin/mount --make-shared / }
> +if { /bin/mount -a --mkdir }
> +
> /bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
> diff --git a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
> index f638e373589884acb959c868462fdf532380a851..fba305c4e9cbb9f7535f11bb4c06d8da8a34837b 100755
> --- a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
> +++ b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
> @@ -4,13 +4,6 @@
>
> if { s6-rc-init -c /etc/s6-rc /run/service }
>
> -if { ln -s /proc/self/fd /dev }
> -if { ln -s /proc/self/fd/0 /dev/stdin }
> -if { ln -s /proc/self/fd/1 /dev/stdout }
> -if { ln -s /proc/self/fd/2 /dev/stderr }
> -
> -if { mount --make-shared / }
> -if { mount --make-shared /run }
> -if { mount -a --mkdir }
> +if { /bin/mount --make-shared /run }
Presumably this does not actually suddenly need an absolute path.
>
> s6-rc change ok-all
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v3 02/22] scripts: Support symlinks in s6-rc-compile inputs
2026-07-11 20:12 ` [PATCH v3 02/22] scripts: Support symlinks in s6-rc-compile inputs Demi Marie Obenour
@ 2026-07-13 9:42 ` Alyssa Ross
2026-07-13 14:19 ` Demi Marie Obenour
0 siblings, 1 reply; 132+ messages in thread
From: Alyssa Ross @ 2026-07-13 9:42 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 453 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Finish scripts can (and often will) be symbolic links to
> /usr/bin/cgroup-setup.
As we've previously discussed, using symlinks in s6-rc services is not a
good idea. s6-rc-compile copies the contents of symlinks, meaning that
if you have a symlink to a big binary, you end up with lots of copies of
it in /run. It's better to use a regular execline script that execs the
program you want to run.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v3 02/22] scripts: Support symlinks in s6-rc-compile inputs
2026-07-13 9:42 ` Alyssa Ross
@ 2026-07-13 14:19 ` Demi Marie Obenour
2026-07-15 18:30 ` Alyssa Ross
0 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-13 14:19 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1: Type: text/plain, Size: 729 bytes --]
On 7/13/26 05:42, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> Finish scripts can (and often will) be symbolic links to
>> /usr/bin/cgroup-setup.
>
> As we've previously discussed, using symlinks in s6-rc services is not a
> good idea. s6-rc-compile copies the contents of symlinks, meaning that
> if you have a symlink to a big binary, you end up with lots of copies of
> it in /run. It's better to use a regular execline script that execs the
> program you want to run.
What about having it be a symlink to an execline script? That's small
enough to safely duplicate, and it keeps the code in one place in the
repository.
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v3 01/22] host/rootfs: Mount filesystems before s6-rc-init
2026-07-13 9:39 ` Alyssa Ross
@ 2026-07-13 17:27 ` Demi Marie Obenour
2026-07-15 18:28 ` Alyssa Ross
0 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-13 17:27 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1: Type: text/plain, Size: 3118 bytes --]
On 7/13/26 05:39, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> cgroup-setup is written in Rust and uses the Rust standard library.
>> It also relies on cgroupfs being mounted at /sys/fs/cgroup.
>> Furthermore, it runs very, *very* early on, since it sets up the cgroups
>> in which s6-svscan and s6-supervise processes run. This allows setting
>> up sub-cgroups without having to worry about name collisions.
>>
>> While it would be possible to only mount the bare minimum necessary,
>> it's simpler to just mount everything before starting any services.
>> These mounts are all pseudo-filesystems or tmpfs instances, and none
>> have any dependencies.
>>
>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>> ---
>> host/rootfs/image/etc/init | 11 ++++++++++-
>> host/rootfs/image/etc/s6-linux-init/scripts/rc.init | 9 +--------
>> 2 files changed, 11 insertions(+), 9 deletions(-)
>>
>> diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
>> index db8a6d9c747e3d212c21b6c983c8f7e299822abe..8ec7859ab00e24007a93d4000c8f34e353de50ae 100755
>> --- a/host/rootfs/image/etc/init
>> +++ b/host/rootfs/image/etc/init
>> @@ -1,6 +1,15 @@
>> #!/bin/execlineb -Ws0
>> # SPDX-License-Identifier: EUPL-1.2+
>> -# SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is>
>> +# SPDX-FileCopyrightText: 2020-2022, 2024 Alyssa Ross <hi@alyssa.is>
>>
>> /bin/setpriv --no-new-privs --
>> +
>> +if { /bin/ln -s /proc/self/fd /dev }
>> +if { /bin/ln -s /proc/self/fd/0 /dev/stdin }
>> +if { /bin/ln -s /proc/self/fd/1 /dev/stdout }
>> +if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
>
> I guess the idea is that the Rust standard library might want these?
I don't know if it actually does, though I would not be surprised if
it is not tested without them. What *is* necessary is for a cgroup
filesystem to be mounted at /sys/fs/cgroup. This allows setting up
cgroup controllers and moving PID 1 to a sub-cgroup.
>> +
>> +if { /bin/mount --make-shared / }
>> +if { /bin/mount -a --mkdir }
>> +
>> /bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
>> diff --git a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
>> index f638e373589884acb959c868462fdf532380a851..fba305c4e9cbb9f7535f11bb4c06d8da8a34837b 100755
>> --- a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
>> +++ b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
>> @@ -4,13 +4,6 @@
>>
>> if { s6-rc-init -c /etc/s6-rc /run/service }
>>
>> -if { ln -s /proc/self/fd /dev }
>> -if { ln -s /proc/self/fd/0 /dev/stdin }
>> -if { ln -s /proc/self/fd/1 /dev/stdout }
>> -if { ln -s /proc/self/fd/2 /dev/stderr }
>> -
>> -if { mount --make-shared / }
>> -if { mount --make-shared /run }
>> -if { mount -a --mkdir }
>> +if { /bin/mount --make-shared /run }
>
> Presumably this does not actually suddenly need an absolute path.
You are correct.
>>
>> s6-rc change ok-all
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v3 01/22] host/rootfs: Mount filesystems before s6-rc-init
2026-07-13 17:27 ` Demi Marie Obenour
@ 2026-07-15 18:28 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-07-15 18:28 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 3511 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> On 7/13/26 05:39, Alyssa Ross wrote:
>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>
>>> cgroup-setup is written in Rust and uses the Rust standard library.
FWIW it might be good to say "cgroup-setup, a tool which will be
introduced in a later patch" or something. Otherwise I have to guess
what this is referring to.
>>> It also relies on cgroupfs being mounted at /sys/fs/cgroup.
>>> Furthermore, it runs very, *very* early on, since it sets up the cgroups
>>> in which s6-svscan and s6-supervise processes run. This allows setting
>>> up sub-cgroups without having to worry about name collisions.
>>>
>>> While it would be possible to only mount the bare minimum necessary,
>>> it's simpler to just mount everything before starting any services.
>>> These mounts are all pseudo-filesystems or tmpfs instances, and none
>>> have any dependencies.
>>>
>>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>>> ---
>>> host/rootfs/image/etc/init | 11 ++++++++++-
>>> host/rootfs/image/etc/s6-linux-init/scripts/rc.init | 9 +--------
>>> 2 files changed, 11 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
>>> index db8a6d9c747e3d212c21b6c983c8f7e299822abe..8ec7859ab00e24007a93d4000c8f34e353de50ae 100755
>>> --- a/host/rootfs/image/etc/init
>>> +++ b/host/rootfs/image/etc/init
>>> @@ -1,6 +1,15 @@
>>> #!/bin/execlineb -Ws0
>>> # SPDX-License-Identifier: EUPL-1.2+
>>> -# SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is>
>>> +# SPDX-FileCopyrightText: 2020-2022, 2024 Alyssa Ross <hi@alyssa.is>
>>>
>>> /bin/setpriv --no-new-privs --
>>> +
>>> +if { /bin/ln -s /proc/self/fd /dev }
>>> +if { /bin/ln -s /proc/self/fd/0 /dev/stdin }
>>> +if { /bin/ln -s /proc/self/fd/1 /dev/stdout }
>>> +if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
>>
>> I guess the idea is that the Rust standard library might want these?
>
> I don't know if it actually does, though I would not be surprised if
> it is not tested without them. What *is* necessary is for a cgroup
> filesystem to be mounted at /sys/fs/cgroup. This allows setting up
> cgroup controllers and moving PID 1 to a sub-cgroup.
Yeah, that's what I thought. It doesn't do any harm to create them
early anyway, so this is fine.
>>> +
>>> +if { /bin/mount --make-shared / }
>>> +if { /bin/mount -a --mkdir }
>>> +
>>> /bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
>>> diff --git a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
>>> index f638e373589884acb959c868462fdf532380a851..fba305c4e9cbb9f7535f11bb4c06d8da8a34837b 100755
>>> --- a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
>>> +++ b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
>>> @@ -4,13 +4,6 @@
>>>
>>> if { s6-rc-init -c /etc/s6-rc /run/service }
>>>
>>> -if { ln -s /proc/self/fd /dev }
>>> -if { ln -s /proc/self/fd/0 /dev/stdin }
>>> -if { ln -s /proc/self/fd/1 /dev/stdout }
>>> -if { ln -s /proc/self/fd/2 /dev/stderr }
>>> -
>>> -if { mount --make-shared / }
>>> -if { mount --make-shared /run }
>>> -if { mount -a --mkdir }
>>> +if { /bin/mount --make-shared /run }
>>
>> Presumably this does not actually suddenly need an absolute path.
>
> You are correct.
Alright then, with that fixed:
Reviewed-by: Alyssa Ross <hi@alyssa.is>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v3 02/22] scripts: Support symlinks in s6-rc-compile inputs
2026-07-13 14:19 ` Demi Marie Obenour
@ 2026-07-15 18:30 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-07-15 18:30 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 1096 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> On 7/13/26 05:42, Alyssa Ross wrote:
>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>
>>> Finish scripts can (and often will) be symbolic links to
>>> /usr/bin/cgroup-setup.
>>
>> As we've previously discussed, using symlinks in s6-rc services is not a
>> good idea. s6-rc-compile copies the contents of symlinks, meaning that
>> if you have a symlink to a big binary, you end up with lots of copies of
>> it in /run. It's better to use a regular execline script that execs the
>> program you want to run.
>
> What about having it be a symlink to an execline script? That's small
> enough to safely duplicate, and it keeps the code in one place in the
> repository.
What code? It's just a single line script that runs another program
with what, one argument? And it's not like fundamentally no other
service is ever going to want to do anything else in its finish script.
It makes more sense for them to not be shared, so that it's not
inconvenient to make service-specific finish script modifications.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* [PATCH v4 00/20] Control group support
2026-07-11 20:12 ` [PATCH v3 01/22] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
2026-07-13 9:39 ` Alyssa Ross
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 01/20] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
` (20 more replies)
1 sibling, 21 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Changes in v4:
- Implement proper locking to make concurrent operations safe.
- Purge VMM cgroup in vmm service finish script.
- Delete /run/vsock/${VM}/vsock before running Cloud Hypervisor.
- Massively refactor cgroup-setup tool.
- Link to v3: https://spectrum-os.org/lists/archives/spectrum-devel/20260711-cgroups-v3-1-5cba61a20cba@gmail.com
Changes in v3:
- Remove the implicit .service suffix on leaf control groups.
- Make cgroup-setup acts as an s6 finish script when called as "finish".
- Put the VMMs in the same cgroups as the per-VM services.
- Add common helper script for the per-VM services.
- Significantly refactor the Rust code.
- Link to v2: https://spectrum-os.org/lists/archives/spectrum-devel/20260620-cgroups-v2-1-ccae224b6c85@gmail.com
Changes in v2:
- Omit resource control support. It was completely broken, and the way
Spectrum run scripts work means that it is better to setup resource
controls just before the execve() into the final service process.
This will be done by a separate tool.
- Link to v1: https://spectrum-os.org/lists/archives/spectrum-devel/20260620-cgroups-v1-1-0e5abf35101b@gmail.com
---
Demi Marie Obenour (20):
host/rootfs: Mount filesystems before s6-rc-init
tools: Add control group manager
Documentation: Mention control groups
Mount cgroup2 filesystem at /sys/fs/cgroup
host/rootfs: Add helper program for per-VM services
host/rootfs: Enable controllers in sub-cgroups
host/rootfs: Add comments where cgroups are intentionally not used
host/rootfs: serial-getty-generator: Use cgroups
host/rootfs: Set up parent cgroup for all per-VM services
host/rootfs: Create per-VM cgroup for all of the VM's services
host/rootfs: run-vmm: Create per-VM cgroup
host/rootfs: run-appimage: Purge the per-VM cgroup
host/rootfs: run-flatpak: Purge the per-VM cgroup
host/rootfs: dbus: Run in cgroup
host/rootfs: vhost-user-fs: Run in cgroup
host/rootfs: vhost-user-gpu: Run in cgroup
host/rootfs: xdg-desktop-portal-spectrum-host: Run in cgroup
host/rootfs: systemd-udevd: Run in cgroup
host/rootfs: weston: Run in cgroup
host/rootfs: spectrum-router: Run in cgroup
.codespellrc | 2 +-
Documentation/doc/development/control-groups.adoc | 88 ++++++
host/rootfs/default.nix | 6 +-
host/rootfs/file-list.mk | 13 +
host/rootfs/image/etc/fstab | 1 +
host/rootfs/image/etc/init | 15 +-
.../s6-linux-init/run-image/service/getty-tty2/run | 1 +
.../s6-linux-init/run-image/service/getty-tty3/run | 1 +
.../s6-linux-init/run-image/service/getty-tty4/run | 1 +
.../run-image/service/root-terminal/run | 1 +
.../run-image/service/s6-linux-init-shutdownd/run | 1 +
.../run-image/service/s6-svscan-log/run | 1 +
.../service/serial-getty-generator/finish | 5 +
.../run-image/service/serial-getty-generator/run | 4 +-
.../run-image/service/serial-getty/run | 1 +
.../run-image/service/serial-getty/template/run | 1 +
.../run-image/service/vm-services/run | 3 +
.../vm-services/template/data/service/dbus/finish | 5 +
.../vm-services/template/data/service/dbus/run | 4 +-
.../template/data/service/spectrum-router/finish | 5 +
.../template/data/service/spectrum-router/run | 4 +-
.../template/data/service/vhost-user-fs/finish | 5 +
.../template/data/service/vhost-user-fs/run | 5 +-
.../template/data/service/vhost-user-gpu/finish | 5 +
.../template/data/service/vhost-user-gpu/run | 9 +-
.../xdg-desktop-portal-spectrum-host/finish | 5 +
.../service/xdg-desktop-portal-spectrum-host/run | 4 +-
.../run-image/service/vm-services/template/finish | 5 +
.../run-image/service/vm-services/template/run | 4 +
.../run-image/service/vmm/template/finish | 5 +
.../rootfs/image/etc/s6-linux-init/scripts/rc.init | 7 -
host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +
host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +-
host/rootfs/image/etc/s6-rc/weston/finish | 5 +
host/rootfs/image/etc/s6-rc/weston/run | 9 +-
host/rootfs/image/usr/bin/cgroup-purge | 1 +
host/rootfs/image/usr/bin/cgroup-s6-finish | 1 +
host/rootfs/image/usr/bin/run-appimage | 7 +-
host/rootfs/image/usr/bin/run-flatpak | 7 +-
host/rootfs/image/usr/bin/run-vmm | 4 +
host/rootfs/image/usr/bin/vm-service-run | 34 ++
host/rootfs/image/usr/bin/vm-stop | 5 +-
pkgs/default.nix | 1 +
tools/cgroup-setup/Cargo.lock | 67 ++++
tools/cgroup-setup/Cargo.lock.license | 2 +
tools/cgroup-setup/Cargo.toml | 11 +
tools/cgroup-setup/default.nix | 18 ++
tools/cgroup-setup/src/cgroup.rs | 349 +++++++++++++++++++++
tools/cgroup-setup/src/main.rs | 347 ++++++++++++++++++++
49 files changed, 1060 insertions(+), 35 deletions(-)
---
base-commit: c9726df5ae78c880a19226e958e5f1f4e3eae28a
change-id: 20260528-cgroups-d609e270e649
--
Sincerely,
Demi Marie Obenour (she/her/hers)
^ permalink raw reply [flat|nested] 132+ messages in thread
* [PATCH v4 01/20] host/rootfs: Mount filesystems before s6-rc-init
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 02/20] tools: Add control group manager Demi Marie Obenour
` (19 subsequent siblings)
20 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
cgroup-setup, a tool which will be introduced in a later patch, is
written in Rust and uses the Rust standard library. It also relies on
cgroupfs being mounted at /sys/fs/cgroup. Furthermore, it runs very,
*very* early on, since it sets up the cgroups in which s6-svscan and
s6-supervise processes run. This allows setting up sub-cgroups without
having to worry about name collisions.
While it would be possible to only mount the bare minimum necessary,
it's simpler to just mount everything before starting any services.
These mounts are all pseudo-filesystems or tmpfs instances, and none
have any dependencies.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/image/etc/init | 11 ++++++++++-
host/rootfs/image/etc/s6-linux-init/scripts/rc.init | 7 -------
2 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
index db8a6d9c747e3d212c21b6c983c8f7e299822abe..8ec7859ab00e24007a93d4000c8f34e353de50ae 100755
--- a/host/rootfs/image/etc/init
+++ b/host/rootfs/image/etc/init
@@ -1,6 +1,15 @@
#!/bin/execlineb -Ws0
# SPDX-License-Identifier: EUPL-1.2+
-# SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2020-2022, 2024 Alyssa Ross <hi@alyssa.is>
/bin/setpriv --no-new-privs --
+
+if { /bin/ln -s /proc/self/fd /dev }
+if { /bin/ln -s /proc/self/fd/0 /dev/stdin }
+if { /bin/ln -s /proc/self/fd/1 /dev/stdout }
+if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
+
+if { /bin/mount --make-shared / }
+if { /bin/mount -a --mkdir }
+
/bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
diff --git a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
index f638e373589884acb959c868462fdf532380a851..db766ba0ca0a11982f5b55497c17eaf5e95afb2c 100755
--- a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
+++ b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
@@ -4,13 +4,6 @@
if { s6-rc-init -c /etc/s6-rc /run/service }
-if { ln -s /proc/self/fd /dev }
-if { ln -s /proc/self/fd/0 /dev/stdin }
-if { ln -s /proc/self/fd/1 /dev/stdout }
-if { ln -s /proc/self/fd/2 /dev/stderr }
-
-if { mount --make-shared / }
if { mount --make-shared /run }
-if { mount -a --mkdir }
s6-rc change ok-all
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 02/20] tools: Add control group manager
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 01/20] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-22 16:01 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 03/20] Documentation: Mention control groups Demi Marie Obenour
` (18 subsequent siblings)
20 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
The cgroup-setup Rust program can create and purge cgroups. It can also
wait for one to become empty, spawn a program in a cgroup, and more. In
the future, it will also support cgroup-based resource control. Locking
is used to ensure that concurrent invocations are safe.
This program can also be used in an s6 finish script. When passed the
args of such a script, it automatically purges the correct cgroup. It
also tells s6 to not restart the service if it dumped core. Core dumps
are often due to memory corruption, and automatically restarting a
service that dumped core makes memory corruption attacks easier.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
.codespellrc | 2 +-
host/rootfs/default.nix | 6 +-
host/rootfs/file-list.mk | 2 +
host/rootfs/image/usr/bin/cgroup-purge | 1 +
host/rootfs/image/usr/bin/cgroup-s6-finish | 1 +
pkgs/default.nix | 1 +
tools/cgroup-setup/Cargo.lock | 67 ++++++
tools/cgroup-setup/Cargo.lock.license | 2 +
tools/cgroup-setup/Cargo.toml | 11 +
tools/cgroup-setup/default.nix | 18 ++
tools/cgroup-setup/src/cgroup.rs | 349 +++++++++++++++++++++++++++++
tools/cgroup-setup/src/main.rs | 347 ++++++++++++++++++++++++++++
12 files changed, 803 insertions(+), 4 deletions(-)
diff --git a/.codespellrc b/.codespellrc
index d8023afc64ec44e98a88c5e397c8d5c681dde063..ae20da8309530759ac729689825a4afc683afa09 100644
--- a/.codespellrc
+++ b/.codespellrc
@@ -2,4 +2,4 @@
# SPDX-License-Identifier: CC0-1.0
[codespell]
-ignore-words-list = crate,passt,rouge,ser
+ignore-words-list = crate,passt,rouge,ser,WRONLY
diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix
index 6bfeefbe0a5f76c1538ccb40e5eb8f291f5d3592..ccf626e2ec0f4bf96573dc5edf058c9375bb65f6 100644
--- a/host/rootfs/default.nix
+++ b/host/rootfs/default.nix
@@ -8,7 +8,7 @@ import ../../lib/call-package.nix (
}:
pkgsMusl.callPackage (
-{ spectrum-host-tools, spectrum-router
+{ spectrum-host-tools, spectrum-router, spectrum-cgroup-setup
, lib, stdenvNoCC, nixos, runCommand, writeClosure, erofs-utils, s6-rc
, btrfs-progs, bubblewrap, busybox, cloud-hypervisor, cosmic-files
, crosvm, cryptsetup, dejavu_fonts, dbus, execline, foot, fuse3
@@ -27,8 +27,8 @@ let
packages = [
btrfs-progs bubblewrap cloud-hypervisor cosmic-files crosvm cryptsetup dbus
execline fuse3 inotify-tools iproute2 jq kmod mdevd mount-flatpak s6
- s6-linux-init s6-rc shadow socat spectrum-host-tools spectrum-router
- virtiofsd xdg-desktop-portal-spectrum-host
+ s6-linux-init s6-rc shadow socat spectrum-cgroup-setup spectrum-host-tools
+ spectrum-router virtiofsd xdg-desktop-portal-spectrum-host
(foot.override { allowPgo = false; })
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 3899d620717fc97f42e669e5313c4100dcf5b1cd..e1280ab56d8797e40b9b1c584ab0daef3cda41d7 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -79,6 +79,8 @@ LINKS = \
image/etc/s6-linux-init/run-image/service/vmm/template/run \
image/lib \
image/sbin \
+ image/usr/bin/cgroup-purge \
+ image/usr/bin/cgroup-s6-finish \
image/usr/bin/systemd-udevd
S6_RC_FILES = \
diff --git a/host/rootfs/image/usr/bin/cgroup-purge b/host/rootfs/image/usr/bin/cgroup-purge
new file mode 120000
index 0000000000000000000000000000000000000000..a0c8d8e144d72b69c613eb0613e39acc9df979df
--- /dev/null
+++ b/host/rootfs/image/usr/bin/cgroup-purge
@@ -0,0 +1 @@
+cgroup-setup
\ No newline at end of file
diff --git a/host/rootfs/image/usr/bin/cgroup-s6-finish b/host/rootfs/image/usr/bin/cgroup-s6-finish
new file mode 120000
index 0000000000000000000000000000000000000000..a0c8d8e144d72b69c613eb0613e39acc9df979df
--- /dev/null
+++ b/host/rootfs/image/usr/bin/cgroup-s6-finish
@@ -0,0 +1 @@
+cgroup-setup
\ No newline at end of file
diff --git a/pkgs/default.nix b/pkgs/default.nix
index 44f7b5ff78cb6b9e755292a6a417d0b627ed3fb0..0a13393164ad5d7f752e630763f3f97166479af5 100644
--- a/pkgs/default.nix
+++ b/pkgs/default.nix
@@ -51,6 +51,7 @@ let
driverSupport = true;
};
spectrum-router = self.callSpectrumPackage ../tools/router {};
+ spectrum-cgroup-setup = self.callSpectrumPackage ../tools/cgroup-setup {};
xdg-desktop-portal-spectrum-host =
self.callSpectrumPackage ../tools/xdg-desktop-portal-spectrum-host {};
diff --git a/tools/cgroup-setup/Cargo.lock b/tools/cgroup-setup/Cargo.lock
new file mode 100644
index 0000000000000000000000000000000000000000..fe967b3aa02c296c87b6b36ac59253dbe0a32de9
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.lock
@@ -0,0 +1,67 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 4
+
+[[package]]
+name = "bitflags"
+version = "2.11.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3"
+
+[[package]]
+name = "cgroup-setup"
+version = "0.0.0"
+dependencies = [
+ "libc",
+ "rustix",
+]
+
+[[package]]
+name = "errno"
+version = "0.3.14"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
+dependencies = [
+ "libc",
+ "windows-sys",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.186"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
+
+[[package]]
+name = "linux-raw-sys"
+version = "0.12.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
+
+[[package]]
+name = "rustix"
+version = "1.1.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
+dependencies = [
+ "bitflags",
+ "errno",
+ "libc",
+ "linux-raw-sys",
+ "windows-sys",
+]
+
+[[package]]
+name = "windows-link"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
+
+[[package]]
+name = "windows-sys"
+version = "0.61.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
+dependencies = [
+ "windows-link",
+]
diff --git a/tools/cgroup-setup/Cargo.lock.license b/tools/cgroup-setup/Cargo.lock.license
new file mode 100644
index 0000000000000000000000000000000000000000..aa108acd23886d8302eaf7babff90d1b08ae19fb
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.lock.license
@@ -0,0 +1,2 @@
+SPDX-License-Identifier: EUPL-1.2+
+SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
diff --git a/tools/cgroup-setup/Cargo.toml b/tools/cgroup-setup/Cargo.toml
new file mode 100644
index 0000000000000000000000000000000000000000..7ed6d6a0ea3bbfc4064b9f39383d0788c4bd84e5
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.toml
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: CC0-1.0
+# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+[package]
+name = "cgroup-setup"
+edition = "2024"
+
+[dependencies]
+libc = "0.2.177"
+rustix = { version = "1.1.2", features = ["fs"] }
diff --git a/tools/cgroup-setup/default.nix b/tools/cgroup-setup/default.nix
new file mode 100644
index 0000000000000000000000000000000000000000..9e716b8f270828b733f39961e5ca37a290a872b4
--- /dev/null
+++ b/tools/cgroup-setup/default.nix
@@ -0,0 +1,18 @@
+# SPDX-FileCopyrightText: 2024 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
+# SPDX-License-Identifier: MIT
+
+import ../../lib/call-package.nix (
+{ src, lib, rustPlatform }:
+
+rustPlatform.buildRustPackage {
+ name = "spectrum-cgroup-setup";
+
+ src = lib.fileset.toSource {
+ root = ../..;
+ fileset = lib.fileset.intersection src ./.;
+ };
+ sourceRoot = "source/tools/cgroup-setup";
+
+ cargoLock.lockFile = ./Cargo.lock;
+}) (_: {})
diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs
new file mode 100644
index 0000000000000000000000000000000000000000..c953d26badfdac0a1e3d7057a867aec3b3247e18
--- /dev/null
+++ b/tools/cgroup-setup/src/cgroup.rs
@@ -0,0 +1,349 @@
+// SPDX-License-Identifier: EUPL-1.2+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+use std::ffi::OsStr;
+use std::fmt::Display;
+use std::fs::File;
+use std::io::{Read as _, Seek as _, Write as _};
+use std::os::unix::prelude::*;
+
+use std::path::{Component, Path, PathBuf};
+
+use rustix::fs::{AtFlags, FlockOperation, XattrFlags};
+use rustix::{
+ fs::{Mode, OFlags, ResolveFlags},
+ io::Errno,
+};
+
+#[derive(Debug)]
+pub(crate) struct Cgroup {
+ path: PathBuf,
+ fd: Vec<(OwnedFd, bool)>,
+}
+
+impl AsFd for Cgroup {
+ fn as_fd(&self) -> BorrowedFd<'_> {
+ self.fd.last().unwrap().0.as_fd()
+ }
+}
+
+fn assert_single_component(component: &[u8]) {
+ match component {
+ b"" | b"." | b".." => panic!("bad component"),
+ _ if component.contains(&b'\0') => panic!("NUL in component"),
+ _ if component.contains(&b'/') => panic!("/ in component"),
+ _ => {}
+ }
+}
+
+impl Cgroup {
+ pub fn new(exclusive: bool) -> Result<Self, String> {
+ let cgroup_root = rustix::fs::openat2(
+ rustix::fs::CWD,
+ Path::new("/sys/fs/cgroup"),
+ OFlags::DIRECTORY | OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW,
+ Mode::empty(),
+ ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_SYMLINKS,
+ )
+ .map_err(|e| format!("Cannot open /sys/fs/cgroup: {e}"))?;
+
+ let lock_operation = if exclusive {
+ FlockOperation::LockExclusive
+ } else {
+ FlockOperation::LockShared
+ };
+ rustix::fs::flock(cgroup_root.as_fd(), lock_operation)
+ .map_err(|e| format!("Cannot lock /sys/fs/cgroup: {e}"))?;
+ Ok(Self {
+ path: PathBuf::from("/sys/fs/cgroup"),
+ fd: vec![(cgroup_root, exclusive)],
+ })
+ }
+
+ pub fn enable_delegation(&self, depth: usize) -> Result<(), Errno> {
+ let (fd, exclusive) = &self.fd[self.fd.len() - depth];
+ assert!(exclusive);
+ rustix::fs::fsetxattr(fd.as_fd(), c"user.delegate", b"1", XattrFlags::empty())
+ }
+
+ pub fn enable_subtree_control(&self, depth: usize) -> Result<(), String> {
+ let (fd, exclusive) = &self.fd[self.fd.len() - depth];
+ assert!(exclusive);
+ let p = Path::new("cgroup.controllers");
+ let mut buf = self.read_control_file(fd.as_fd(), p)?;
+ let mut subtree = vec![];
+ if buf.ends_with(b"\n") {
+ buf.pop();
+ }
+ for controller in buf.split(|&b| b == b' ').filter(|e| !e.is_empty()) {
+ for &c in controller {
+ if c <= b' ' || c >= 0x7F {
+ return Err(format!("Bad byte {c} in cgroup.controllers"));
+ }
+ }
+ if !subtree.is_empty() {
+ subtree.push(b' ');
+ }
+ subtree.push(b'+');
+ subtree.extend_from_slice(controller);
+ }
+ if !subtree.is_empty() {
+ self.write_cgroup_value("cgroup.subtree_control", str::from_utf8(&subtree).unwrap())?;
+ }
+ Ok(())
+ }
+
+ pub fn read_control_file(&self, fd: BorrowedFd, p: &Path) -> Result<Vec<u8>, String> {
+ let mut buf = Vec::new();
+ let err = |e: &dyn Display, p: &Path, msg: &str| {
+ let path = self.path.join(p);
+ format!("Cannot {msg} {path:?}: {e}")
+ };
+ File::from(open_subtree_raw(Path::new(p), fd.as_fd()).map_err(|e| err(&e, p, "open"))?)
+ .read_to_end(&mut buf)
+ .map_err(|e| err(&e, p, "read"))?;
+ Ok(buf)
+ }
+
+ /// Open a single component as a sub-cgroup
+ fn open_sub_cgroup_raw(&self, access: OFlags, component: &[u8]) -> Result<OwnedFd, Errno> {
+ assert_single_component(component);
+ rustix::fs::openat2(
+ self.as_fd(),
+ Path::new(OsStr::from_bytes(component)),
+ OFlags::CLOEXEC | OFlags::NOFOLLOW | access,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS
+ | ResolveFlags::NO_MAGICLINKS
+ | ResolveFlags::BENEATH
+ | ResolveFlags::NO_XDEV,
+ )
+ }
+
+ pub fn open_sub_cgroup(
+ &mut self,
+ path: &std::path::Path,
+ exclusive: bool,
+ allow_missing: bool,
+ ) -> Result<bool, String> {
+ let mut iter = path.components().peekable();
+ while let Some(component) = iter.next() {
+ let component = match component {
+ Component::Normal(component) => component,
+ _ => unreachable!(),
+ };
+ let sub_fd = match self
+ .open_sub_cgroup_raw(OFlags::DIRECTORY | OFlags::RDONLY, component.as_bytes())
+ {
+ Ok(sub_fd) => {
+ self.path.push(component);
+ sub_fd
+ }
+ Err(Errno::NOENT) if allow_missing => return Ok(false),
+ Err(e) => {
+ return Err(format!(
+ "Cannot open sub-cgroup {component:?} of {:?}: {e}",
+ self.path
+ ));
+ }
+ };
+ let exclusive = exclusive && iter.peek().is_none();
+ let lock_operation = if exclusive {
+ FlockOperation::LockExclusive
+ } else {
+ FlockOperation::LockShared
+ };
+ rustix::fs::flock(sub_fd.as_fd(), lock_operation).map_err(|e| {
+ let msg = format!("Cannot lock sub-cgroup {:?}: {e}", self.path);
+ self.path.pop();
+ msg
+ })?;
+ self.fd.push((sub_fd, exclusive));
+ }
+ Ok(true)
+ }
+
+ pub fn open_subtree(&self, path: &std::path::Path) -> Result<OwnedFd, Errno> {
+ let dirfd = self.as_fd();
+ open_subtree_raw(path, dirfd)
+ }
+
+ fn exclusive(&self) -> bool {
+ self.fd.last().unwrap().1
+ }
+
+ pub fn joined_path(&self, p: &Path) -> PathBuf {
+ let mut owned_p = self.path.clone();
+ owned_p.push(p);
+ owned_p
+ }
+
+ pub fn wait_for_empty(&self) -> std::io::Result<()> {
+ assert!(self.exclusive());
+ let wait_file = self.open_subtree(std::path::Path::new("cgroup.events"))?;
+ let poll_fd = wait_file.as_raw_fd();
+ let mut wait_fd = File::from(wait_file);
+ let mut fds = libc::pollfd {
+ fd: poll_fd,
+ events: libc::POLLPRI | libc::POLLERR,
+ revents: 0,
+ };
+ let mut v = vec![];
+ loop {
+ v.clear();
+ wait_fd
+ .seek(std::io::SeekFrom::Start(0))
+ .expect("Seek on control group file should succeed");
+ wait_fd
+ .read_to_end(&mut v)
+ .expect("reading from control group should work");
+ if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
+ break;
+ }
+ // SAFETY: FFI call, valid arguments, fds contains 1 element
+ if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
+ panic!("poll failed");
+ }
+ }
+ Ok(())
+ }
+
+ pub(crate) fn make_child(&mut self, path: &Path) -> Result<(), Errno> {
+ assert!(self.exclusive());
+ let component = path.as_os_str().as_bytes();
+ assert_single_component(component);
+ match rustix::fs::mkdirat(
+ self.as_fd(),
+ path,
+ Mode::RUSR
+ | Mode::WUSR
+ | Mode::XUSR
+ | Mode::RGRP
+ | Mode::XGRP
+ | Mode::ROTH
+ | Mode::XOTH,
+ ) {
+ Ok(()) | Err(Errno::EXIST) => {}
+ bad => return bad,
+ }
+ let p = self.open_sub_cgroup_raw(OFlags::RDONLY | OFlags::DIRECTORY, component)?;
+ // exclusive lock on parent acts as exclusive lock on child
+ self.fd.push((p, true));
+ self.path.push(path);
+ Ok(())
+ }
+
+ pub(super) fn purge(&mut self, path: &Path) -> Result<(), String> {
+ assert!(self.exclusive());
+ match rustix::fs::unlinkat(self.as_fd(), Path::new(path), AtFlags::REMOVEDIR) {
+ // Trying to purge a deleted cgroup is not an error.
+ Ok(()) | Err(Errno::NOENT) => return Ok(()),
+ Err(Errno::BUSY) => {}
+ Err(e) => return Err(format!("Cannot purge {:?}: {e}", self.joined_path(path))),
+ }
+ if !self.open_sub_cgroup(path, true, true)? {
+ return Ok(());
+ }
+
+ rustix::fs::flock(
+ self.fd[self.fd.len() - 2].0.as_fd(),
+ FlockOperation::LockShared,
+ )
+ .map_err(|e| format!("Cannot relock {:?}: {e}", self.path.parent()))?;
+ self.write_cgroup_value("cgroup.kill", "1")?;
+ self.wait_for_empty()
+ .map_err(|e| format!("Cannot wait for cgroup to become empty: {e}"))?;
+ let fd = self.fd.pop().unwrap().0;
+ let v = (|| {
+ remove_recursively(fd, 1000)
+ .map_err(|e| format!("Cannot remove {:?}: {e}", self.path))?;
+ rustix::fs::flock(self.as_fd(), FlockOperation::LockExclusive)
+ .map_err(|e| format!("Cannot lock {:?}: {e}", self.path))?;
+ match rustix::fs::unlinkat(
+ self.as_fd(),
+ Path::new(self.path.file_name().unwrap()),
+ AtFlags::REMOVEDIR,
+ ) {
+ // something might have re-created the cgroup in the meantime, which is okay
+ Ok(()) | Err(Errno::BUSY) => Ok(()),
+ Err(e) => Err(format!("Cannot lock {:?}: {e}", self.path)),
+ }
+ })();
+ assert!(self.path.pop());
+ v
+ }
+
+ pub(crate) fn write_cgroup_value(&self, name: &str, value: &str) -> Result<(), String> {
+ let path = Path::new(name);
+ let fd = rustix::fs::openat2(
+ self.as_fd(),
+ path,
+ OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::WRONLY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )
+ .map_err(|e| format!("Cannot open {:?}: {}", self.joined_path(Path::new(name)), e))?;
+ File::from(fd).write_all(value.as_bytes()).map_err(|e| {
+ format!(
+ "Cannot write {:?} to {:?}: {}",
+ value,
+ self.joined_path(Path::new(name)),
+ e
+ )
+ })
+ }
+}
+
+fn open_subtree_raw(path: &Path, dirfd: BorrowedFd<'_>) -> Result<OwnedFd, Errno> {
+ rustix::fs::openat2(
+ dirfd,
+ path,
+ OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )
+}
+
+fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> {
+ if remaining_depth < 1 {
+ panic!("control groups too deeply nested");
+ }
+ let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating");
+ while let Some(element) = d.next() {
+ let element = element.expect("Iterating through a cgroup directory failed?");
+ if element.file_type() != rustix::fs::FileType::Directory {
+ continue;
+ }
+
+ let remaining_depth = remaining_depth - 1;
+ let d: &rustix::fs::Dir = &d;
+ let dirfd = d.fd().unwrap();
+ let path = element.file_name();
+ remove_all(remaining_depth, dirfd, path)?;
+ }
+ drop(d);
+ Ok(())
+}
+
+fn remove_all(
+ remaining_depth: usize,
+ dirfd: BorrowedFd<'_>,
+ path: &std::ffi::CStr,
+) -> Result<(), Errno> {
+ if path == c"." || path == c".." {
+ return Ok(());
+ }
+ if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() {
+ return Ok(());
+ }
+ let fd = rustix::fs::openat2(
+ dirfd,
+ path,
+ OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
+ )?;
+ remove_recursively(fd, remaining_depth)?;
+ rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?;
+ Ok(())
+}
diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
new file mode 100644
index 0000000000000000000000000000000000000000..2e7a4e25213a4aa2449b292f8005e1da23cbb1f9
--- /dev/null
+++ b/tools/cgroup-setup/src/main.rs
@@ -0,0 +1,347 @@
+// SPDX-License-Identifier: EUPL-1.2+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+use std::{
+ ffi::{OsStr, OsString},
+ os::unix::prelude::*,
+ path::{Path, PathBuf},
+};
+
+use crate::cgroup::Cgroup;
+
+mod cgroup;
+
+fn check_path(path: &OsStr) -> Result<(), String> {
+ if path.is_empty() {
+ return Ok(());
+ }
+
+ for component in path.as_bytes().split(|&b| b == b'/') {
+ match component {
+ b"" | b"." | b".." => {
+ return Err(format!("Path {path:?} has empty, ., or .. component"));
+ }
+ // Cannot happen: command line arguments have no NUL byte,
+ // and /proc/self/cgroup having a NUL byte is a kernel bug.
+ _ if component.contains(&b'\0') => panic!("Path {path:?} has NUL byte"),
+ _ if component.len() > 255 => {
+ return Err(format!(
+ "Path {path:?} has component {:?} that is longer than 255 bytes",
+ OsStr::from_bytes(component)
+ ));
+ }
+ _ => {}
+ }
+ }
+
+ Ok(())
+}
+
+/// Get the path of the cgroup for the provided command-line argument.
+/// Returns an empty path if the path is "/", or if it is "." and the
+/// current cgroup is "/".
+///
+/// # Errors
+///
+/// Fails if the provided path is invalid or empty, or if it is relative
+/// and the local cgroup cannot be determined.
+fn get_cgroup(cgroup_path: OsString) -> Result<PathBuf, String> {
+ if cgroup_path.as_bytes().starts_with(b"/") {
+ let mut cgroup_path = cgroup_path.into_vec();
+ cgroup_path.remove(0);
+ if cgroup_path.is_empty() {
+ return Err("cgroup path cannot be /".to_owned());
+ }
+ let cgroup_path = OsString::from_vec(cgroup_path);
+ check_path(&cgroup_path)?;
+ Ok(cgroup_path.into())
+ } else if cgroup_path.is_empty() {
+ Err("cgroup path cannot be empty".to_owned())
+ } else {
+ check_path(&cgroup_path)?;
+ let mut local_cgroup = local_cgroup()?;
+ local_cgroup.push(cgroup_path);
+ Ok(local_cgroup)
+ }
+}
+
+/// Open the cgroup corresponding to the provided path.
+/// It must have already been made relative to `/sys/fs/cgroup`.
+///
+/// # Errors
+///
+/// Fails if the cgroup operation fails.
+fn open_cgroup(path: &Path, exclusive: bool) -> Result<Cgroup, String> {
+ if path.as_os_str().is_empty() {
+ Cgroup::new(exclusive)
+ } else {
+ let mut cgroup = Cgroup::new(false)?;
+ cgroup.open_sub_cgroup(path, exclusive, false)?;
+ Ok(cgroup)
+ }
+}
+
+/// Open the cgroup corresponding to the provided path's parent.
+/// It is made relative to the process's own cgroup if needed.
+///
+/// # Errors
+///
+/// Fails if the cgroup operation fails.
+fn open_relative_cgroup(arg: OsString) -> Result<(PathBuf, Cgroup), String> {
+ let path = get_cgroup(arg)?;
+ let cgroup = open_cgroup(path.parent().expect("always has a parent"), true)?;
+ Ok((path, cgroup))
+}
+
+fn main() {
+ let mut args = std::env::args_os();
+ let Some(prog_name) = args.next() else {
+ eprintln!("No command line arguments (argv[0] is NULL)");
+ std::process::exit(1);
+ };
+ match main_(&prog_name, args) {
+ Ok(()) => {}
+ Err(e) => {
+ eprintln!("{prog_name:?}: {}", e);
+ std::process::exit(1);
+ }
+ }
+}
+
+fn main_(prog_name: &OsStr, mut args: std::env::ArgsOs) -> Result<(), String> {
+ match prog_name
+ .as_bytes()
+ .split(|&b| b == b'/')
+ .next_back()
+ .unwrap()
+ {
+ b"cgroup-s6-finish" => {
+ return s6_finish(&mut args);
+ }
+ b"cgroup-setup" => {}
+ b"cgroup-purge" => {
+ if args.len() != 1 {
+ return Err(format!(
+ "cgroup-purge takes one argument, got {}",
+ args.len()
+ ));
+ }
+ let cgroup_path = args.next().unwrap();
+ let (path, mut cgroup) = open_relative_cgroup(cgroup_path)?;
+ let cgroup_target = Path::new(path.file_name().unwrap());
+ return cgroup.purge(cgroup_target);
+ }
+ _ => {
+ return Err(format!(
+ "must be invoked as \"cgroup-setup\" \
+ \"cgroup-purge\", or \"cgroup-s6-finish\", \
+ got {prog_name:?}",
+ ));
+ }
+ };
+ let mut leaf = false;
+ let mut cgroup_path;
+ let mut delegate = false;
+ let mut init_subtree = false;
+ let mut child_name: Option<&'static OsStr> = None;
+ let mut wait = true;
+ loop {
+ cgroup_path = args.next();
+ let Some(ref arg_) = cgroup_path else {
+ break;
+ };
+ let arg_ = arg_.as_bytes();
+ if arg_ == b"--" {
+ cgroup_path = args.next();
+ break;
+ }
+ if !arg_.starts_with(b"-") {
+ break;
+ }
+
+ if !arg_.starts_with(b"--") {
+ return Err("takes no short options".to_owned());
+ }
+
+ match &arg_[2..] {
+ b"leaf" => leaf = true,
+ b"delegate" => delegate = true,
+ b"init-subtree" => init_subtree = true,
+ b"wait" => wait = true,
+ b"no-wait" => wait = false,
+ b"child-name" if child_name.is_none() => match args.next() {
+ Some(arg) => child_name = Some(arg.leak()),
+ None => return Err("--child-name: missing argument".to_owned()),
+ },
+ b"child-name" => return Err("--child-name: cannot be used twice".to_owned()),
+ arg => match str::from_utf8(arg) {
+ Ok(e) => return Err(format!("unknown long option {e:?}")),
+ Err(_) => return Err("long option isn't UTF-8".to_owned()),
+ },
+ }
+ }
+
+ let default_child_name = OsStr::from_bytes(b"$inner.service");
+
+ let child_name = Path::new(child_name.unwrap_or(default_child_name));
+
+ let Some(mut cgroup_path) = cgroup_path else {
+ return Err("have no positional arguments, expected at least 1".to_owned());
+ };
+
+ // Allow --init-subtree .
+ if cgroup_path.as_bytes() == b"." && init_subtree && !leaf {
+ cgroup_path = child_name.to_owned().into();
+ leaf = true;
+ }
+
+ let (path, mut cgroup) = open_relative_cgroup(cgroup_path)?;
+ let cgroup_target = Path::new(path.file_name().unwrap());
+ cgroup
+ .make_child(cgroup_target)
+ .map_err(|e| format!("Cannot make child cgroup: {e}"))?;
+ if wait {
+ cgroup
+ .wait_for_empty()
+ .map_err(|e| format!("Cannot wait for {path:?} to be empty: {e}"))?;
+ }
+ let pid = std::process::id().to_string();
+ if leaf {
+ if args.len() != 0 {
+ // If we aren't delegating any cgroups, don't create a sub-cgroup.
+ cgroup
+ .write_cgroup_value("cgroup.procs", &pid)
+ .map_err(|e| format!("Cannot write to {path:?}/cgroup.procs: {e}"))?;
+ }
+ } else {
+ // If the child process will need to manage cgroups itself, it will need
+ // to set up a sub-cgroup due to the "no internal processes" rule. It's
+ // simplest to just do it automatically.
+ cgroup.make_child(Path::new(child_name)).map_err(|e| {
+ format!(
+ "Cannot create child cgroup {}/{}: {e}",
+ path.display(),
+ child_name.display()
+ )
+ })?;
+ if args.len() != 0 {
+ cgroup
+ .write_cgroup_value("cgroup.procs", &pid)
+ .map_err(|e| {
+ format!(
+ "Cannot write to {}/{}/cgroup.procs: {e}",
+ path.display(),
+ child_name.display()
+ )
+ })?;
+ }
+ }
+ if !leaf {
+ cgroup.enable_subtree_control(2)?;
+ }
+ if init_subtree {
+ cgroup.enable_subtree_control(1)?;
+ }
+ if delegate {
+ cgroup
+ .enable_delegation(1)
+ .map_err(|e| format!("Cannot enable cgroup delegation in {path:?}: {e}"))?;
+ }
+ let Some(program_name) = args.next() else {
+ return Ok(());
+ };
+ let e = std::process::Command::new(&program_name).args(args).exec();
+ Err(format!("Cannot spawn child {:?}: {}", program_name, e))
+}
+
+fn s6_finish(args: &mut std::env::ArgsOs) -> Result<(), String> {
+ if args.len() < 3 {
+ return Err(format!(
+ "s6 finish scripts take at least 3 arguments, got {}",
+ args.len()
+ ));
+ }
+ let status = parse_digit_string(&args.next().unwrap(), "exit status")?;
+ let signal = args.next().unwrap();
+ let signal = if status == 256 {
+ Some(parse_digit_string(&signal, "signal number")?)
+ } else {
+ None
+ };
+ let service = args.next().unwrap();
+
+ let (path, mut cgroup) = open_relative_cgroup(service)?;
+ let cgroup_target = Path::new(path.file_name().unwrap());
+ let exit_125 = if let Some(signal) = signal {
+ match signal as libc::c_int {
+ libc::SIGBUS
+ | libc::SIGFPE
+ | libc::SIGABRT
+ | libc::SIGTRAP
+ | libc::SIGSEGV
+ | libc::SIGILL => {
+ // Process *crashed*, indicating a *possible exploit attempt*.
+ // s6 should *not* restart it. This is distinct from a Rust panic,
+ // which is much less likely to indicate memory corruption.
+ true
+ }
+ _ => false,
+ }
+ } else {
+ false
+ };
+ if exit_125 {
+ // Ignore panics. Exit status is more important.
+ // We already had a core dump.
+ let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
+ match cgroup.purge(cgroup_target) {
+ Ok(()) => {}
+ Err(e) => {
+ eprintln!("cgroup-s6-finish: Failed to purge cgroup: {e}")
+ }
+ };
+ }));
+ std::process::exit(125)
+ } else {
+ cgroup.purge(cgroup_target)
+ }
+}
+
+fn parse_digit_string(digits: &OsStr, msg: &str) -> Result<u16, String> {
+ let checked = match str::from_utf8(digits.as_bytes()) {
+ Ok(s) => s,
+ Err(e) => return Err(format!("{msg} is not UTF-8: {e}")),
+ };
+ let r = checked
+ .parse::<u16>()
+ .map_err(|e| format!("{msg} {digits:?} is a bad 16-bit number: {e}"))?;
+ match checked.as_bytes() {
+ b"0" | [b'1'..=b'9', ..] => Ok(r),
+ [b'0', ..] => Err(format!("{msg} {} has a leading 0", digits.display())),
+ _ => Err(format!("{msg} {} starts with +", digits.display())),
+ }
+}
+
+fn local_cgroup() -> Result<PathBuf, String> {
+ let mut local_cgroup: Vec<u8> = std::fs::read("/proc/thread-self/cgroup")
+ .map_err(|e| format!("cannot read /proc/thread-self/cgroup: {e}"))?;
+ let local_cgroup_len = local_cgroup.len();
+ if local_cgroup_len < 5
+ || local_cgroup[..4] != *b"0::/"
+ || local_cgroup[local_cgroup_len - 1] != b'\n'
+ || local_cgroup[4..local_cgroup_len - 1].contains(&b'\n')
+ {
+ // It's possible to get here if the cgroup path contains a newline,
+ // but that never happens in Spectrum.
+ return Err(format!(
+ "Invalid contents {local_cgroup:?} of /proc/thread-self/cgroup - \
+ do you have cgroups v1 mounted instead of cgroups v2?"
+ ));
+ }
+
+ local_cgroup.copy_within(4..local_cgroup_len - 1, 0);
+ local_cgroup.truncate(local_cgroup_len - 5);
+ let local_cgroup = OsString::from_vec(local_cgroup);
+ check_path(&local_cgroup).unwrap();
+ Ok(PathBuf::from(local_cgroup))
+}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 03/20] Documentation: Mention control groups
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 01/20] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 02/20] tools: Add control group manager Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-27 11:22 ` Alyssa Ross
2026-07-28 10:41 ` Valentin Gagarin
2026-07-22 1:59 ` [PATCH v4 04/20] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
` (17 subsequent siblings)
20 siblings, 2 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Documentation/doc/development/control-groups.adoc | 88 +++++++++++++++++++++++
1 file changed, 88 insertions(+)
diff --git a/Documentation/doc/development/control-groups.adoc b/Documentation/doc/development/control-groups.adoc
new file mode 100644
index 0000000000000000000000000000000000000000..6ce33f21a230d012a690fc5deb2ba597d1d01ef1
--- /dev/null
+++ b/Documentation/doc/development/control-groups.adoc
@@ -0,0 +1,88 @@
+= Control groups in Spectrum
+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0
+
+Linux control groups (cgroups) can be used for several purposes:
+
+1. They allow waiting for a group of processes to exit.
+2. They allow terminating a group of processes.
+3. They allow limiting a group of processes' access to resources.
+
+Spectrum currently uses the first two. The third is not yet used,
+but will be in the future.
+
+== Control Group Hierarchy
+
+Spectrum uses the following cgroup hierarchy:
+
+1. There is a `/vm-services.slice` cgroup that contains all the per-VM
+ services on the system.
+2. The per-VM services for each VM are under `/vm-services.slice/vm-${VM}.slice`,
+ where `${VM}` is replaced by the VM's ID.
+3. Each per-VM service is under `/vm-services.slice/vm-${VM}.slice/${SERVICE_NAME}`,
+ where `${VM}` is replaced by the VM's ID and `${SERVICE_NAME}` is replaced by
+ the name of the service.
+4. The VMM runs under `/vm-services.slice/vm-${VM}.slice/vmm`.
+
+If a cgroup contains child cgroups, it likely contains a `$inner.service`
+cgroup. This is where programs that would otherwise run in the cgroup itself
+are placed. Generally, these programs are instances of `s6-svscan` and/or
+`s6-supervise`.
+
+== Using Control Groups
+
+When adding a new s6 service, one should carefully consider whether it
+should be placed in a control group. Most services should be placed in
+a control group, with only a few exceptions:
+
+1. Services, such as `getty`, that spawn background processes.
+2. Loggers.
+3. Trivial services that don't do anything.
+
+Generally, it's best to set the control group up as the first thing
+the service does. To do that, use `cgroup-setup --leaf -- $1 COMMAND_LINE`,
+where `$1` should be the service name and `COMMAND_LINE` is the program
+to run in a cgroup.
+
+If you use execline for your run script, this is as simple as:
+
+.run
+....
+#!/bin/execlineb -WS1
+
+cgroup-setup --leaf -- $1
+# rest of script comes here
+....
+
+If the service exits, it's usually best to terminate any programs left
+behind with SIGKILL and remove the control group. In Spectrum, this is
+called "purging" the cgroup. To purge the cgroup when a service exits,
+make the `finish` script invoke `/usr/bin/cgroup-s6-finish`. The first
+two command line arguments must be the first two arguments passed to the
+`finish` script. The third argument must be the path to the cgroup to
+be purged relative to the cgroup the program itself is in. This is
+usually, but not always, the third argument to the `finish` script.
+
+When invoked as `cgroup-s6-finish`, `cgroup-setup` checks if
+the service exited due to a signal that caused it to dump core. If it
+did, `cgroup-s6-finish` exits with status 125, ensuring that
+`s6-supervise` will *not* restart it. This is intentional: if a service
+crashes due to a fatal signal, this is possibly a sign of memory
+corruption. Restarting the service in this case can turn an unreliable
+memory corruption exploit into a reliable one. Rust panics do not cause
+core dumps, so the service will be restarted afterwards.
+
+One can also use `cgroup-purge` to purge a cgroup explicitly. This is
+used to stop the VMM and all per-VM services when a VM is shut down.
+
+== Future plans
+
+Control groups are designed around a single writer process controlling each
+of them. Many Linux distros use systemd for this, but Spectrum doesn't use
+systemd. The only persistent per-service process is s6-supervise, but that
+doesn't have control group support.
+
+Instead, the plan is to have a database containing this information.
+Whether this will be in the `data/` subdirectory of the service directory
+or a separate system-wide database has not yet been determined.
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 04/20] Mount cgroup2 filesystem at /sys/fs/cgroup
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (2 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 03/20] Documentation: Mention control groups Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-27 11:23 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 05/20] host/rootfs: Add helper program for per-VM services Demi Marie Obenour
` (16 subsequent siblings)
20 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Needed for cgroups to work at all.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/fstab | 1 +
1 file changed, 1 insertion(+)
diff --git a/host/rootfs/image/etc/fstab b/host/rootfs/image/etc/fstab
index 4d52a1da8c6d673cbbcfe2e9c0ec93085f06c40f..cab97cf9c49f864845492a1e524d43b7d546317f 100644
--- a/host/rootfs/image/etc/fstab
+++ b/host/rootfs/image/etc/fstab
@@ -6,3 +6,4 @@ tmpfs /dev/shm tmpfs nosuid,nodev 0 0
tmpfs /media tmpfs nosuid,nodev,noexec,nosymfollow,mode=755 0 0
sysfs /sys sysfs nosuid,nodev,noexec 0 0
tmpfs /tmp tmpfs nosuid,nodev 0 0
+cgroup2 /sys/fs/cgroup cgroup2 nosuid,nodev,noexec,nosymfollow 0 0
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 05/20] host/rootfs: Add helper program for per-VM services
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (3 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 04/20] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-27 11:27 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 06/20] host/rootfs: Enable controllers in sub-cgroups Demi Marie Obenour
` (15 subsequent siblings)
20 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
The program handles cgroups and common substitutions. This is better
than repeating the boilerplate multiple times.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
host/rootfs/image/usr/bin/vm-service-run | 34 ++++++++++++++++++++++++++++++++
2 files changed, 35 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index e1280ab56d8797e40b9b1c584ab0daef3cda41d7..c707c01179238231f70164ffda5b7c940b34192a 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -65,6 +65,7 @@ FILES = \
image/usr/bin/spectrum-update \
image/usr/bin/vm-console \
image/usr/bin/vm-import \
+ image/usr/bin/vm-service-run \
image/usr/bin/vm-start \
image/usr/bin/vm-stop \
image/usr/bin/xdg-open \
diff --git a/host/rootfs/image/usr/bin/vm-service-run b/host/rootfs/image/usr/bin/vm-service-run
new file mode 100755
index 0000000000000000000000000000000000000000..c788574cf95ec7fd0d93b656557f85fa80b31690
--- /dev/null
+++ b/host/rootfs/image/usr/bin/vm-service-run
@@ -0,0 +1,34 @@
+#!/usr/bin/execlineb -WS1
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+# Substitute all of the needed environment variables
+# into both this script and its arguments
+# (the subsequent command to run). This substitutes
+# the *caller*'s arguments.
+multisubstitute {
+ importas -iS VM
+ importas -iS "#"
+ importas -iS "1"
+ importas -iS WAYLAND_DISPLAY
+}
+
+# Check that the VM name and service are reasonable.
+# Then run the cgroup-setup program and the provided
+# command line. Avoid premature substitution by
+# escaping ${#} and ${1} with backslashes. Otherwise,
+# they would be values for this script, not for the caller.
+case -- "\\${#}@${VM}@\\${1}" {
+ # s6-supervise might start passing extra arguments in the future, but
+ # 0 arguments is not okay.
+ "[1-9][0-9]*@[A-Za-z0-9_][A-Za-z0-9_.-]*@[A-Za-z_][A-Za-z0-9_-]*" {
+ # The caller has been migrated from using -WS1 to using -Wp.
+ # Pop the environment to preserve the original behavior.
+ emptyenv -P cgroup-setup --leaf -- "\\${1}" $@
+ }
+}
+fdmove -c 1 2
+if { printf "Wrong VM name %s, \
+parent argument count %s, or parent argument %s\n"
+ $VM "\\$#" "\\${1}" }
+exit 100
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 06/20] host/rootfs: Enable controllers in sub-cgroups
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (4 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 05/20] host/rootfs: Add helper program for per-VM services Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-27 12:11 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 07/20] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
` (14 subsequent siblings)
20 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Actually use the cgroup manager for the first time.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/init | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
index 8ec7859ab00e24007a93d4000c8f34e353de50ae..5d31a23bae2f29f35bfeced68242dededf6ae0c7 100755
--- a/host/rootfs/image/etc/init
+++ b/host/rootfs/image/etc/init
@@ -12,4 +12,8 @@ if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
if { /bin/mount --make-shared / }
if { /bin/mount -a --mkdir }
+# Enable subtree control of all cgroups and move
+# process to a child cgroup.
+/usr/bin/cgroup-setup --init-subtree .
+
/bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 07/20] host/rootfs: Add comments where cgroups are intentionally not used
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (5 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 06/20] host/rootfs: Enable controllers in sub-cgroups Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-27 12:12 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 08/20] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
` (13 subsequent siblings)
20 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Most services should use cgroups, but some do not.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run | 1 +
.../etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run | 1 +
.../image/etc/s6-linux-init/run-image/service/serial-getty/template/run | 1 +
8 files changed, 8 insertions(+)
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
index 4d2cd7172e887e90d9ce87489cdacf41e51f77e8..5b2421a3dedf7c282ea818f4064535d0fc29d511 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty2 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
index 06abf5422365ff4edc9379e5cbcab716e6830399..3ca382f76368174ee1f99ea880827958c57b602c 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty3 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
index 3dd235fc93d403fc8971e923ec62cb025684b67a..983bd82927e468bab6ae20f61411680d173e4924 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty4 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
index 86b9a1ef70300de52f33a84cbfac22a77e00df36..fb55327dff5f189f543fb3629702caad9ff247e2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
@@ -2,6 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# cgroups make no sense here
s6-ipcserver-socketbinder -a 0700 /run/root-terminal
if { chown wayland /run/root-terminal }
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
index 327f4ccc5056e7ef87c089e666090307845cae3a..a6733604e91692f8440fd68efeb1a6b7378be167 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# cgroups make no sense for shutdownd
s6-linux-init-shutdownd -Bc /etc/s6-linux-init
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
index 8f7bd889e50d76a49b98d8ba9e2c6172f287277c..cf3f5c7b499d997fd426062651eca64b3accc146 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
@@ -2,5 +2,6 @@
# SPDX-License-Identifier: ISC
# SPDX-FileCopyrightText: Copyright (c) 2015-2024 Laurent Bercot <ska-skaware@skarnet.org>
+# cgroups make no sense for a logger
redirfd -rnb 0 fifo
s6-log -bpd3 -- T /run/log
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
index 78f794202bf174f3c036f3e20755ac087a988277..8fb1b6779c0bb1e3cdf67d1fa14722caacc25994 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# No need for a cgroup here.
s6-svscan -d3 instance
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
index f3ed9a7f08e60206afd222f4fc02c5fb19527561..775b828c72b9d73bef8a3355fb3ae0536975f474 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021, 2024 Alyssa Ross <hi@alyssa.is>
+# Don't put this in a cgroup. We want nohup to work.
getty -i -n -l /etc/login 0,115200,57600,38400,9600 $1 dumb
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 08/20] host/rootfs: serial-getty-generator: Use cgroups
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (6 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 07/20] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-27 12:14 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 09/20] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
` (12 subsequent siblings)
20 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Avoids concurrent execution and leaked child processes.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../s6-linux-init/run-image/service/serial-getty-generator/finish | 5 +++++
.../etc/s6-linux-init/run-image/service/serial-getty-generator/run | 4 +++-
3 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index c707c01179238231f70164ffda5b7c940b34192a..b899b4356d73d73ad0fc4ecd3837b7a3525d6f5a 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -21,6 +21,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run \
image/etc/s6-linux-init/run-image/service/s6-svscan-log/notification-fd \
image/etc/s6-linux-init/run-image/service/s6-svscan-log/run \
+ image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish \
image/etc/s6-linux-init/run-image/service/serial-getty-generator/run \
image/etc/s6-linux-init/run-image/service/serial-getty/notification-fd \
image/etc/s6-linux-init/run-image/service/serial-getty/run \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
new file mode 100755
index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-s6-finish $@
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
index bf66ab0878fc6d8e77ca71a4a89e50c139a6bb11..a1846a29cf55892127feb7ecfc0adcd12f0497bd 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
@@ -1,7 +1,9 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+# avoid concurrent operation
+cgroup-setup --leaf -- $1
piperw 3 4
background {
fdclose 3
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 09/20] host/rootfs: Set up parent cgroup for all per-VM services
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (7 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 08/20] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-27 12:16 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 10/20] host/rootfs: Create per-VM cgroup for all of the VM's services Demi Marie Obenour
` (11 subsequent siblings)
20 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run | 3 +++
1 file changed, 3 insertions(+)
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
index 78f794202bf174f3c036f3e20755ac087a988277..430065b1efae8ce1fe7c7a07b67920daa88bf98c 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
@@ -1,5 +1,8 @@
#!/bin/execlineb -WP
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# Set up the parent cgroup of all VMMs and VM services
+cgroup-setup --no-wait -- /vm-services.slice
s6-svscan -d3 instance
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 10/20] host/rootfs: Create per-VM cgroup for all of the VM's services
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (8 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 09/20] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 11/20] host/rootfs: run-vmm: Create per-VM cgroup Demi Marie Obenour
` (10 subsequent siblings)
20 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../etc/s6-linux-init/run-image/service/vm-services/template/finish | 5 +++++
.../etc/s6-linux-init/run-image/service/vm-services/template/run | 4 ++++
3 files changed, 10 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index b899b4356d73d73ad0fc4ecd3837b7a3525d6f5a..498c699c459515c1c7b6139aa07a89fd904509b3 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -39,6 +39,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/run \
image/etc/s6-linux-init/run-image/service/vmm/notification-fd \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
new file mode 100755
index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-s6-finish $@
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
index bf90e99eef4106a9ad17dfba310b6edb3eaa64a3..d3a5f3490e447448929383a6b5d36dcef723712a 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
@@ -2,6 +2,10 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# Move into the cgroup with all per-VM services
+# Do not wait for the existing processes in the slice to finish.
+cgroup-setup --no-wait -- /vm-services.slice/vm-${1}.slice
+
export VM $1
s6-svscan -d3 data/service
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 11/20] host/rootfs: run-vmm: Create per-VM cgroup
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (9 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 10/20] host/rootfs: Create per-VM cgroup for all of the VM's services Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 12/20] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
` (9 subsequent siblings)
20 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../etc/s6-linux-init/run-image/service/vm-services/template/finish | 4 ++--
.../image/etc/s6-linux-init/run-image/service/vmm/template/finish | 5 +++++
host/rootfs/image/usr/bin/run-vmm | 4 ++++
host/rootfs/image/usr/bin/vm-stop | 5 ++++-
5 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 498c699c459515c1c7b6139aa07a89fd904509b3..33b7f869da7dfdb8bde6c70ebd9acd5e0f2856c1 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -44,6 +44,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/run \
image/etc/s6-linux-init/run-image/service/vmm/notification-fd \
image/etc/s6-linux-init/run-image/service/vmm/run \
+ image/etc/s6-linux-init/run-image/service/vmm/template/finish \
image/etc/s6-linux-init/run-image/service/vmm/template/notification-fd \
image/etc/s6-linux-init/scripts/rc.init \
image/etc/s6-linux-init/scripts/rc.shutdown \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
index 4d5b454e97c3584d0644c2d738514f13b1c53957..72d071291f5235e17b9a94c15bd2c0a287e5cbfb 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
@@ -1,5 +1,5 @@
#!/usr/bin/execlineb -WS3
-# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# SPDX-License-Identifier: EUPL-1.2+
-cgroup-s6-finish $@
+cgroup-s6-finish $1 $2 /vm-services.slice/vm-${3}.slice
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/template/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/template/finish
new file mode 100755
index 0000000000000000000000000000000000000000..bb9647c0f4a501a83874adfba8765954438d0b11
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/template/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-s6-finish $1 $2 /vm-services.slice/vm-${3}.slice/vmm
diff --git a/host/rootfs/image/usr/bin/run-vmm b/host/rootfs/image/usr/bin/run-vmm
index 6967dc5586335d999f8030371fe0f38502409919..12896c276fbffa024fa0f89f42a2967506b1cb74 100755
--- a/host/rootfs/image/usr/bin/run-vmm
+++ b/host/rootfs/image/usr/bin/run-vmm
@@ -5,6 +5,7 @@
if {
backtick -D "" mnt {
importas -Siu 1
+ if { cgroup-setup --no-wait /vm-services.slice/vm-${1}.slice }
nsenter --mount=/run/vm/by-id/${1}/ns/mnt
findmnt -no FSTYPE,SOURCE /run/fs/${1}/disk
}
@@ -33,6 +34,8 @@ if {
elgetpositionals
+if { rm -f /run/vsock/${1}/vsock }
+
s6-ipcserver-socketbinder -B /run/vm/by-id/${1}/vmm
getpid -E vmm_pid
@@ -86,6 +89,7 @@ redirfd -r 0 /dev/null
s6-softlimit -H -l 18446744073709551615
if { udevadm wait /dev/kvm }
+cgroup-setup --leaf /vm-services.slice/vm-${1}.slice/vmm
s6-envuidgid vmm-${1}
s6-applyuidgid -Uz
bwrap
diff --git a/host/rootfs/image/usr/bin/vm-stop b/host/rootfs/image/usr/bin/vm-stop
index 6fb6aaed46de684eb035243e4f7c79111e93bf8c..23a815c0c260452ff9b1bcce7a091519b7420196 100755
--- a/host/rootfs/image/usr/bin/vm-stop
+++ b/host/rootfs/image/usr/bin/vm-stop
@@ -2,4 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
-ch-remote --api-socket /run/vm/by-id/${1}/vmm shutdown
+foreground { ch-remote --api-socket /run/vm/by-id/${1}/vmm shutdown }
+if { cgroup-purge /vm-services.slice/vm-${1}.slice/vmm }
+importas -iS ?
+exit $?
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 12/20] host/rootfs: run-appimage: Purge the per-VM cgroup
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (10 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 11/20] host/rootfs: run-vmm: Create per-VM cgroup Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-27 12:18 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 13/20] host/rootfs: run-flatpak: " Demi Marie Obenour
` (8 subsequent siblings)
20 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/usr/bin/run-appimage | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/image/usr/bin/run-appimage b/host/rootfs/image/usr/bin/run-appimage
index 672ec22d4b815fb9c2940e13ae08ed4093baee4e..03020c89b02f295844f5ed814034ba66be741f4b 100755
--- a/host/rootfs/image/usr/bin/run-appimage
+++ b/host/rootfs/image/usr/bin/run-appimage
@@ -89,7 +89,12 @@ fdclose 4
foreground { run-vmm $id }
fdclose 3
-if { s6-instance-delete /run/service/vm-services $id }
+if {
+ # Even if s6-instance-delete fails, cgroup-purge is enough
+ # to ensure that the VMM and all per-VM services are killed.
+ foreground { s6-instance-delete /run/service/vm-services $id }
+ cgroup-purge /vm-services.slice/vm-${id}.slice
+}
if { umount -R /run/vm/by-id/${id}/ns }
rm -r -- $diskdir /run/vm/by-id/${id} /run/configs/${id}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 13/20] host/rootfs: run-flatpak: Purge the per-VM cgroup
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (11 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 12/20] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-27 12:19 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 14/20] host/rootfs: dbus: Run in cgroup Demi Marie Obenour
` (7 subsequent siblings)
20 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/usr/bin/run-flatpak | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/image/usr/bin/run-flatpak b/host/rootfs/image/usr/bin/run-flatpak
index fafbc82ff6966d8be9c8083a17618f1fb192bd07..cdd13fb52d97751f92864d34a5221367df6d6f0d 100755
--- a/host/rootfs/image/usr/bin/run-flatpak
+++ b/host/rootfs/image/usr/bin/run-flatpak
@@ -87,9 +87,12 @@ if {
fdclose 4
foreground { run-vmm $id }
-}
-if { s6-instance-delete -- /run/service/vm-services $id }
+ # Even if s6-instance-delete fails, cgroup-purge is enough
+ # to ensure that the VMM and all per-VM services are killed.
+ foreground { s6-instance-delete /run/service/vm-services $id }
+ cgroup-purge /vm-services.slice/vm-${id}.slice
+}
if { umount -R /run/vm/by-id/${id}/ns }
rm -r -- $diskdir /run/vm/by-id/${id} /run/configs/${id}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 14/20] host/rootfs: dbus: Run in cgroup
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (12 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 13/20] host/rootfs: run-flatpak: " Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 15/20] host/rootfs: vhost-user-fs: " Demi Marie Obenour
` (6 subsequent siblings)
20 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../run-image/service/vm-services/template/data/service/dbus/finish | 5 +++++
.../run-image/service/vm-services/template/data/service/dbus/run | 4 ++--
3 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 33b7f869da7dfdb8bde6c70ebd9acd5e0f2856c1..4835a84593f5e006a7b86170a065903a9dd324aa 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -28,6 +28,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/serial-getty/template/run \
image/etc/s6-linux-init/run-image/service/vm-services/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish
new file mode 100755
index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-s6-finish $@
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
index 331d9a57e8d99eb8d6f7b33031cd130b410a6157..38d5836acb9db2d8a8bee25e3e9527a9061ed837 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
@@ -1,8 +1,8 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -Wp
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
-importas -i VM VM
+vm-service-run
if {
redirfd -w 1 data/dbus.conf
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 15/20] host/rootfs: vhost-user-fs: Run in cgroup
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (13 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 14/20] host/rootfs: dbus: Run in cgroup Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 16/20] host/rootfs: vhost-user-gpu: " Demi Marie Obenour
` (5 subsequent siblings)
20 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Use the vm-service-run script.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../service/vm-services/template/data/service/vhost-user-fs/finish | 5 +++++
.../service/vm-services/template/data/service/vhost-user-fs/run | 5 +++--
3 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 4835a84593f5e006a7b86170a065903a9dd324aa..9a0f231611e4046737141a86ed5b39523c73a022 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -34,6 +34,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/notification-fd \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish
new file mode 100755
index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-s6-finish $@
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
index cb059aebc8dfe2c8a5b059d5bfea884de53bf922..66aeef5d1bd7399c9969e38ce1a9ad32f4ba56b8 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
@@ -1,10 +1,11 @@
-#!/bin/execlineb -WS1
+#!/bin/execlineb -Wp
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2021-2025 Alyssa Ross <hi@alyssa.is>
+vm-service-run
+
s6-ipcserver-socketbinder -a 0700 -B env/virtiofsd.sock
-importas -i VM VM
if { chown vmm-${VM} env/virtiofsd.sock }
if { fdmove 1 3 echo }
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 16/20] host/rootfs: vhost-user-gpu: Run in cgroup
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (14 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 15/20] host/rootfs: vhost-user-fs: " Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 17/20] host/rootfs: xdg-desktop-portal-spectrum-host: " Demi Marie Obenour
` (4 subsequent siblings)
20 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Use vm-service-run for this.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../vm-services/template/data/service/vhost-user-gpu/finish | 5 +++++
.../service/vm-services/template/data/service/vhost-user-gpu/run | 9 +++------
3 files changed, 9 insertions(+), 6 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 9a0f231611e4046737141a86ed5b39523c73a022..616c1cdef8241158631240ea8a174bc3bf3b5e0f 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -37,6 +37,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/notification-fd \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish
new file mode 100755
index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-s6-finish $@
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
index c3bfafe02ec6d2ee418fe56b033bbc4c7a73b186..49978bf9e4e600198cd3f8704c33229518a8fa60 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
@@ -1,14 +1,11 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -Wp
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
-s6-ipcserver-socketbinder -a 0700 -b 1 env/crosvm.sock
+vm-service-run
-multisubstitute {
- importas -Siu VM
- importas -Si WAYLAND_DISPLAY
-}
+s6-ipcserver-socketbinder -a 0700 -b 1 env/crosvm.sock
if { chown vmm-${VM} env/crosvm.sock }
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 17/20] host/rootfs: xdg-desktop-portal-spectrum-host: Run in cgroup
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (15 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 16/20] host/rootfs: vhost-user-gpu: " Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 18/20] host/rootfs: systemd-udevd: " Demi Marie Obenour
` (3 subsequent siblings)
20 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../template/data/service/xdg-desktop-portal-spectrum-host/finish | 5 +++++
.../template/data/service/xdg-desktop-portal-spectrum-host/run | 4 ++--
3 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 616c1cdef8241158631240ea8a174bc3bf3b5e0f..e3411d40014342e67071a48f9e1c7bc7b3954bcf 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -40,6 +40,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish
new file mode 100755
index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-s6-finish $@
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
index 909957b817136149dd01c3b98847238ecda8032d..d9df94b6c6b73d0ee385864691524cedd887a2e2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
@@ -1,8 +1,8 @@
-#!/bin/execlineb -WS1
+#!/bin/execlineb -Wp
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
-importas -i VM VM
+vm-service-run
export DBUS_SESSION_BUS_ADDRESS unix:path=/run/portal-bus/${VM}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 18/20] host/rootfs: systemd-udevd: Run in cgroup
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (16 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 17/20] host/rootfs: xdg-desktop-portal-spectrum-host: " Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-27 12:20 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 19/20] host/rootfs: weston: " Demi Marie Obenour
` (2 subsequent siblings)
20 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +++++
host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +++--
3 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index e3411d40014342e67071a48f9e1c7bc7b3954bcf..065571ef3a5c61fa79a1d2ed878052a5f391be96 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -106,6 +106,7 @@ S6_RC_FILES = \
image/etc/s6-rc/systemd-udevd-coldplug/dependencies.d/systemd-udevd \
image/etc/s6-rc/systemd-udevd-coldplug/type \
image/etc/s6-rc/systemd-udevd-coldplug/up \
+ image/etc/s6-rc/systemd-udevd/finish \
image/etc/s6-rc/systemd-udevd/notification-fd \
image/etc/s6-rc/systemd-udevd/run \
image/etc/s6-rc/systemd-udevd/type \
diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/finish b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
new file mode 100755
index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
--- /dev/null
+++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-s6-finish $@
diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/run b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
old mode 100644
new mode 100755
index aec6444e951503eae988e666b77fda8f2ae33d72..901f3e6667a8de04a6af39945406a1757455f089
--- a/host/rootfs/image/etc/s6-rc/systemd-udevd/run
+++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
@@ -1,7 +1,8 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
-s6-setlock /run/sd-notify-wrapper/systemd-udevd.lock
+
+cgroup-setup --delegate --child-name udev -- $1
s6-ipcserver-socketbinder -b0 -m -a 0600 /run/sd-notify-wrapper/systemd-udevd.sock
background -d {
fdmove 1 3
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 19/20] host/rootfs: weston: Run in cgroup
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (17 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 18/20] host/rootfs: systemd-udevd: " Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-27 12:23 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 20/20] host/rootfs: spectrum-router: " Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
20 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
host/rootfs/image/etc/s6-rc/weston/finish | 5 +++++
host/rootfs/image/etc/s6-rc/weston/run | 9 +++++++--
3 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 065571ef3a5c61fa79a1d2ed878052a5f391be96..e890e6c211ff011e494cbdd64a1c26ff6360d94a 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -119,6 +119,7 @@ S6_RC_FILES = \
image/etc/s6-rc/vmm-env/contents.d/systemd-udevd-coldplug \
image/etc/s6-rc/vmm-env/type \
image/etc/s6-rc/weston/dependencies.d/systemd-udevd-coldplug \
+ image/etc/s6-rc/weston/finish \
image/etc/s6-rc/weston/notification-fd \
image/etc/s6-rc/weston/run \
image/etc/s6-rc/weston/type
diff --git a/host/rootfs/image/etc/s6-rc/weston/finish b/host/rootfs/image/etc/s6-rc/weston/finish
new file mode 100755
index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
--- /dev/null
+++ b/host/rootfs/image/etc/s6-rc/weston/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-s6-finish $@
diff --git a/host/rootfs/image/etc/s6-rc/weston/run b/host/rootfs/image/etc/s6-rc/weston/run
old mode 100644
new mode 100755
index 8b065ee9a535d0a08e9255900c73b88a0c376d4b..a306d9903da6da920efa55eccdfb4f3d14698a79
--- a/host/rootfs/image/etc/s6-rc/weston/run
+++ b/host/rootfs/image/etc/s6-rc/weston/run
@@ -1,8 +1,13 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -Wp
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2021, 2025 Alyssa Ross <hi@alyssa.is>
-importas -Siu WAYLAND_DISPLAY
+multisubstitute {
+ importas -Siu WAYLAND_DISPLAY
+ importas -Si 1
+}
+
+cgroup-setup --leaf --no-wait -- $1
piperw 4 3
background {
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v4 20/20] host/rootfs: spectrum-router: Run in cgroup
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (18 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 19/20] host/rootfs: weston: " Demi Marie Obenour
@ 2026-07-22 1:59 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
20 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-22 1:59 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../service/vm-services/template/data/service/spectrum-router/finish | 5 +++++
.../service/vm-services/template/data/service/spectrum-router/run | 4 ++--
3 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index e890e6c211ff011e494cbdd64a1c26ff6360d94a..459e4fffca491fa5c5f2e0d24f6ab8f91b105488 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -32,6 +32,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish
new file mode 100755
index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-s6-finish $@
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
index 3de58c27facc78c48176d8f9a6dd1827ac926f1a..dfb3b8e77c8b3a02b83f4dfeb19b3dc7a816ac45 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
@@ -1,9 +1,9 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -Wp
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
-importas -i VM VM
+vm-service-run
s6-ipcserver-socketbinder -a 0770 /run/vm/by-id/${VM}/router-driver.sock
fdmove -c 3 0
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* Re: [PATCH v4 02/20] tools: Add control group manager
2026-07-22 1:59 ` [PATCH v4 02/20] tools: Add control group manager Demi Marie Obenour
@ 2026-07-22 16:01 ` Alyssa Ross
2026-07-23 23:07 ` Demi Marie Obenour
0 siblings, 1 reply; 132+ messages in thread
From: Alyssa Ross @ 2026-07-22 16:01 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 40565 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> The cgroup-setup Rust program can create and purge cgroups. It can also
> wait for one to become empty, spawn a program in a cgroup, and more. In
> the future, it will also support cgroup-based resource control. Locking
> is used to ensure that concurrent invocations are safe.
>
> This program can also be used in an s6 finish script. When passed the
> args of such a script, it automatically purges the correct cgroup. It
> also tells s6 to not restart the service if it dumped core. Core dumps
> are often due to memory corruption, and automatically restarting a
> service that dumped core makes memory corruption attacks easier.
>
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> .codespellrc | 2 +-
> host/rootfs/default.nix | 6 +-
> host/rootfs/file-list.mk | 2 +
> host/rootfs/image/usr/bin/cgroup-purge | 1 +
> host/rootfs/image/usr/bin/cgroup-s6-finish | 1 +
> pkgs/default.nix | 1 +
> tools/cgroup-setup/Cargo.lock | 67 ++++++
> tools/cgroup-setup/Cargo.lock.license | 2 +
> tools/cgroup-setup/Cargo.toml | 11 +
> tools/cgroup-setup/default.nix | 18 ++
> tools/cgroup-setup/src/cgroup.rs | 349 +++++++++++++++++++++++++++++
> tools/cgroup-setup/src/main.rs | 347 ++++++++++++++++++++++++++++
> 12 files changed, 803 insertions(+), 4 deletions(-)
> diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
> index 3899d620717fc97f42e669e5313c4100dcf5b1cd..e1280ab56d8797e40b9b1c584ab0daef3cda41d7 100644
> --- a/host/rootfs/file-list.mk
> +++ b/host/rootfs/file-list.mk
> @@ -79,6 +79,8 @@ LINKS = \
> image/etc/s6-linux-init/run-image/service/vmm/template/run \
> image/lib \
> image/sbin \
> + image/usr/bin/cgroup-purge \
> + image/usr/bin/cgroup-s6-finish \
> image/usr/bin/systemd-udevd
>
> S6_RC_FILES = \
> diff --git a/host/rootfs/image/usr/bin/cgroup-purge b/host/rootfs/image/usr/bin/cgroup-purge
> new file mode 120000
> index 0000000000000000000000000000000000000000..a0c8d8e144d72b69c613eb0613e39acc9df979df
> --- /dev/null
> +++ b/host/rootfs/image/usr/bin/cgroup-purge
> @@ -0,0 +1 @@
> +cgroup-setup
> \ No newline at end of file
> diff --git a/host/rootfs/image/usr/bin/cgroup-s6-finish b/host/rootfs/image/usr/bin/cgroup-s6-finish
> new file mode 120000
> index 0000000000000000000000000000000000000000..a0c8d8e144d72b69c613eb0613e39acc9df979df
> --- /dev/null
> +++ b/host/rootfs/image/usr/bin/cgroup-s6-finish
> @@ -0,0 +1 @@
> +cgroup-setup
> \ No newline at end of file
Usually packages that expect to be invoked via symlinks like this
(coreutils, busybox, execline) install their own symlinks, rather than
expecting systems to create them. I think these would be more
appropriate in a postInstall in tools/cgroup-setup/default.nix.
> diff --git a/pkgs/default.nix b/pkgs/default.nix
> index 44f7b5ff78cb6b9e755292a6a417d0b627ed3fb0..0a13393164ad5d7f752e630763f3f97166479af5 100644
> --- a/pkgs/default.nix
> +++ b/pkgs/default.nix
> @@ -51,6 +51,7 @@ let
> driverSupport = true;
> };
> spectrum-router = self.callSpectrumPackage ../tools/router {};
> + spectrum-cgroup-setup = self.callSpectrumPackage ../tools/cgroup-setup {};
> xdg-desktop-portal-spectrum-host =
> self.callSpectrumPackage ../tools/xdg-desktop-portal-spectrum-host {};
>
> diff --git a/tools/cgroup-setup/Cargo.lock b/tools/cgroup-setup/Cargo.lock
> new file mode 100644
> index 0000000000000000000000000000000000000000..fe967b3aa02c296c87b6b36ac59253dbe0a32de9
> --- /dev/null
> +++ b/tools/cgroup-setup/Cargo.lock
> @@ -0,0 +1,67 @@
> +# This file is automatically @generated by Cargo.
> +# It is not intended for manual editing.
> +version = 4
> +
> +[[package]]
> +name = "bitflags"
> +version = "2.11.1"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3"
> +
> +[[package]]
> +name = "cgroup-setup"
> +version = "0.0.0"
> +dependencies = [
> + "libc",
> + "rustix",
> +]
> +
> +[[package]]
> +name = "errno"
> +version = "0.3.14"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
> +dependencies = [
> + "libc",
> + "windows-sys",
> +]
> +
> +[[package]]
> +name = "libc"
> +version = "0.2.186"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
> +
> +[[package]]
> +name = "linux-raw-sys"
> +version = "0.12.1"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
> +
> +[[package]]
> +name = "rustix"
> +version = "1.1.4"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
> +dependencies = [
> + "bitflags",
> + "errno",
> + "libc",
> + "linux-raw-sys",
> + "windows-sys",
> +]
> +
> +[[package]]
> +name = "windows-link"
> +version = "0.2.1"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
> +
> +[[package]]
> +name = "windows-sys"
> +version = "0.61.2"
> +source = "registry+https://github.com/rust-lang/crates.io-index"
> +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
> +dependencies = [
> + "windows-link",
> +]
> diff --git a/tools/cgroup-setup/Cargo.lock.license b/tools/cgroup-setup/Cargo.lock.license
> new file mode 100644
> index 0000000000000000000000000000000000000000..aa108acd23886d8302eaf7babff90d1b08ae19fb
> --- /dev/null
> +++ b/tools/cgroup-setup/Cargo.lock.license
> @@ -0,0 +1,2 @@
> +SPDX-License-Identifier: EUPL-1.2+
> +SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
I think this should be CC0-1.0 like every other Cargo.lock.license.
There's nothing copyrightable about it.
> diff --git a/tools/cgroup-setup/Cargo.toml b/tools/cgroup-setup/Cargo.toml
> new file mode 100644
> index 0000000000000000000000000000000000000000..7ed6d6a0ea3bbfc4064b9f39383d0788c4bd84e5
> --- /dev/null
> +++ b/tools/cgroup-setup/Cargo.toml
> @@ -0,0 +1,11 @@
> +# SPDX-License-Identifier: CC0-1.0
> +# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
I surely did not contribute anything copyrightable to this.
> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +
> +[package]
> +name = "cgroup-setup"
> +edition = "2024"
> +
> +[dependencies]
> +libc = "0.2.177"
> +rustix = { version = "1.1.2", features = ["fs"] }
> diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs
> new file mode 100644
> index 0000000000000000000000000000000000000000..c953d26badfdac0a1e3d7057a867aec3b3247e18
> --- /dev/null
> +++ b/tools/cgroup-setup/src/cgroup.rs
> @@ -0,0 +1,349 @@
> +// SPDX-License-Identifier: EUPL-1.2+
> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +
> +use std::ffi::OsStr;
> +use std::fmt::Display;
> +use std::fs::File;
> +use std::io::{Read as _, Seek as _, Write as _};
> +use std::os::unix::prelude::*;
> +
> +use std::path::{Component, Path, PathBuf};
> +
> +use rustix::fs::{AtFlags, FlockOperation, XattrFlags};
> +use rustix::{
> + fs::{Mode, OFlags, ResolveFlags},
> + io::Errno,
> +};
> +
> +#[derive(Debug)]
> +pub(crate) struct Cgroup {
> + path: PathBuf,
> + fd: Vec<(OwnedFd, bool)>,
There's no point storing all these exclusivity bools, is there? I think
only the last one is ever checked, so we could make things tighter and
clearer like this, where we only track the exclusivity of the last fd:
fd: Vec<OwnedFd>,
exclusive: bool,
> +}
> +
> +impl AsFd for Cgroup {
> + fn as_fd(&self) -> BorrowedFd<'_> {
> + self.fd.last().unwrap().0.as_fd()
> + }
> +}
> +
> +fn assert_single_component(component: &[u8]) {
Why not &Path, which is already guaranteed not to have a NUL byte?
Perhaps this whole thing could be simplified to
Some(component).as_os_str() == component.file_name()? Maybe that's too
clever, though…
> + match component {
> + b"" | b"." | b".." => panic!("bad component"),
> + _ if component.contains(&b'\0') => panic!("NUL in component"),
> + _ if component.contains(&b'/') => panic!("/ in component"),
> + _ => {}
> + }
> +}
> +
> +impl Cgroup {
> + pub fn new(exclusive: bool) -> Result<Self, String> {
> + let cgroup_root = rustix::fs::openat2(
> + rustix::fs::CWD,
> + Path::new("/sys/fs/cgroup"),
> + OFlags::DIRECTORY | OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW,
> + Mode::empty(),
> + ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_SYMLINKS,
> + )
I pointed out in my review of v2 that OFlags::NOFOLLOW is redundant with
ResolveFlags::NO_SYMLINKS, but now it seesm to have come back across the
board.
> + .map_err(|e| format!("Cannot open /sys/fs/cgroup: {e}"))?;
> +
> + let lock_operation = if exclusive {
> + FlockOperation::LockExclusive
> + } else {
> + FlockOperation::LockShared
> + };
> + rustix::fs::flock(cgroup_root.as_fd(), lock_operation)
> + .map_err(|e| format!("Cannot lock /sys/fs/cgroup: {e}"))?;
> + Ok(Self {
> + path: PathBuf::from("/sys/fs/cgroup"),
> + fd: vec![(cgroup_root, exclusive)],
> + })
> + }
> +
> + pub fn enable_delegation(&self, depth: usize) -> Result<(), Errno> {
> + let (fd, exclusive) = &self.fd[self.fd.len() - depth];
> + assert!(exclusive);
> + rustix::fs::fsetxattr(fd.as_fd(), c"user.delegate", b"1", XattrFlags::empty())
> + }
> +
> + pub fn enable_subtree_control(&self, depth: usize) -> Result<(), String> {
> + let (fd, exclusive) = &self.fd[self.fd.len() - depth];
> + assert!(exclusive);
> + let p = Path::new("cgroup.controllers");
> + let mut buf = self.read_control_file(fd.as_fd(), p)?;
> + let mut subtree = vec![];
> + if buf.ends_with(b"\n") {
> + buf.pop();
> + }
> + for controller in buf.split(|&b| b == b' ').filter(|e| !e.is_empty()) {
> + for &c in controller {
> + if c <= b' ' || c >= 0x7F {
> + return Err(format!("Bad byte {c} in cgroup.controllers"));
> + }
> + }
> + if !subtree.is_empty() {
> + subtree.push(b' ');
> + }
> + subtree.push(b'+');
> + subtree.extend_from_slice(controller);
> + }
> + if !subtree.is_empty() {
> + self.write_cgroup_value("cgroup.subtree_control", str::from_utf8(&subtree).unwrap())?;
> + }
> + Ok(())
> + }
> +
> + pub fn read_control_file(&self, fd: BorrowedFd, p: &Path) -> Result<Vec<u8>, String> {
> + let mut buf = Vec::new();
> + let err = |e: &dyn Display, p: &Path, msg: &str| {
> + let path = self.path.join(p);
> + format!("Cannot {msg} {path:?}: {e}")
> + };
> + File::from(open_subtree_raw(Path::new(p), fd.as_fd()).map_err(|e| err(&e, p, "open"))?)
If we're using it for opening files, open_subtree_raw is probably misnamed.
> + .read_to_end(&mut buf)
> + .map_err(|e| err(&e, p, "read"))?;
> + Ok(buf)
> + }
> +
> + /// Open a single component as a sub-cgroup
> + fn open_sub_cgroup_raw(&self, access: OFlags, component: &[u8]) -> Result<OwnedFd, Errno> {
> + assert_single_component(component);
> + rustix::fs::openat2(
> + self.as_fd(),
> + Path::new(OsStr::from_bytes(component)),
> + OFlags::CLOEXEC | OFlags::NOFOLLOW | access,
> + Mode::empty(),
> + ResolveFlags::NO_SYMLINKS
> + | ResolveFlags::NO_MAGICLINKS
> + | ResolveFlags::BENEATH
> + | ResolveFlags::NO_XDEV,
This is also doing exactly the same thing as open_subtree_raw, except it
allows changing the access mode, and also sets NO_MAGICLINKS. I don't
think any of the other callers of open_subtree_raw would need to open
magic links, so maybe this is evidence this should be unified with them?
> + )
> + }
> +
> + pub fn open_sub_cgroup(
> + &mut self,
> + path: &std::path::Path,
> + exclusive: bool,
> + allow_missing: bool,
> + ) -> Result<bool, String> {
> + let mut iter = path.components().peekable();
> + while let Some(component) = iter.next() {
Perhaps would be nicer:
let mut components = path.components().peekable();
for component in components {
> + let component = match component {
> + Component::Normal(component) => component,
> + _ => unreachable!(),
> + };
> + let sub_fd = match self
> + .open_sub_cgroup_raw(OFlags::DIRECTORY | OFlags::RDONLY, component.as_bytes())
> + {
> + Ok(sub_fd) => {
> + self.path.push(component);
I would really like to not try to store self.path. It seems very
complicated to track. It's also very unclear to me from the name (and
the code) what it is. Is it the path to the cgroup itself, or to its
parent? It looks to me like it should be the cgroup itself, but then
what's going on in purge?
We could actually improve readability of this quite complicated function
even further if you find it acceptable to just use Errno for the error
type. In that case, we'd just return Result<(), Errno>, and callers
would check for Errno::NOENT if they wanted to allow missing. Then we
could just completely drop that argument. In my opinion it would be
worth it to move complexity out of here.
> + sub_fd
> + }
> + Err(Errno::NOENT) if allow_missing => return Ok(false),
> + Err(e) => {
> + return Err(format!(
> + "Cannot open sub-cgroup {component:?} of {:?}: {e}",
> + self.path
> + ));
> + }
> + };
> + let exclusive = exclusive && iter.peek().is_none();
> + let lock_operation = if exclusive {
> + FlockOperation::LockExclusive
> + } else {
> + FlockOperation::LockShared
> + };
> + rustix::fs::flock(sub_fd.as_fd(), lock_operation).map_err(|e| {
> + let msg = format!("Cannot lock sub-cgroup {:?}: {e}", self.path);
> + self.path.pop();
> + msg
> + })?;
> + self.fd.push((sub_fd, exclusive));
> + }
> + Ok(true)
> + }
> +
> + pub fn open_subtree(&self, path: &std::path::Path) -> Result<OwnedFd, Errno> {
> + let dirfd = self.as_fd();
> + open_subtree_raw(path, dirfd)
> + }
If open_subtree_raw just took &dyn AsFd, there'd be no need for this
method.
> +
> + fn exclusive(&self) -> bool {
> + self.fd.last().unwrap().1
> + }
> +
> + pub fn joined_path(&self, p: &Path) -> PathBuf {
> + let mut owned_p = self.path.clone();
> + owned_p.push(p);
> + owned_p
> + }
> +
> + pub fn wait_for_empty(&self) -> std::io::Result<()> {
> + assert!(self.exclusive());
> + let wait_file = self.open_subtree(std::path::Path::new("cgroup.events"))?;
> + let poll_fd = wait_file.as_raw_fd();
> + let mut wait_fd = File::from(wait_file);
> + let mut fds = libc::pollfd {
> + fd: poll_fd,
> + events: libc::POLLPRI | libc::POLLERR,
> + revents: 0,
> + };
> + let mut v = vec![];
> + loop {
> + v.clear();
> + wait_fd
> + .seek(std::io::SeekFrom::Start(0))
> + .expect("Seek on control group file should succeed");
> + wait_fd
> + .read_to_end(&mut v)
> + .expect("reading from control group should work");
> + if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
> + break;
> + }
> + // SAFETY: FFI call, valid arguments, fds contains 1 element
> + if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
> + panic!("poll failed");
> + }
> + }
Are you 100% confident that this doesn't race? I don't understand why
poll would be triggered in this scenario:
1. "1" is written to cgroup.kill
2. Every process in the cgroup exits and is reaped.
3. cgroup.events is opened, with the cgroup already empty.
Are you not relying on 2 happening after 3? Presumably if you open
cgroup.events for a cgroup that's already empty, you're not going to get
a poll event to tell you it's empty.
> + Ok(())
> + }
> +
> + pub(crate) fn make_child(&mut self, path: &Path) -> Result<(), Errno> {
> + assert!(self.exclusive());
> + let component = path.as_os_str().as_bytes();
> + assert_single_component(component);
> + match rustix::fs::mkdirat(
> + self.as_fd(),
> + path,
> + Mode::RUSR
> + | Mode::WUSR
> + | Mode::XUSR
> + | Mode::RGRP
> + | Mode::XGRP
> + | Mode::ROTH
> + | Mode::XOTH,
> + ) {
> + Ok(()) | Err(Errno::EXIST) => {}
> + bad => return bad,
> + }
> + let p = self.open_sub_cgroup_raw(OFlags::RDONLY | OFlags::DIRECTORY, component)?;
> + // exclusive lock on parent acts as exclusive lock on child
> + self.fd.push((p, true));
> + self.path.push(path);
> + Ok(())
> + }
> +
> + pub(super) fn purge(&mut self, path: &Path) -> Result<(), String> {
I guess we have to call purge on the parent, rather than on the cgroup
itself, because of the unlink? Maybe we could call it purge_child? It
confused me for a while.
> + assert!(self.exclusive());
> + match rustix::fs::unlinkat(self.as_fd(), Path::new(path), AtFlags::REMOVEDIR) {
> + // Trying to purge a deleted cgroup is not an error.
> + Ok(()) | Err(Errno::NOENT) => return Ok(()),
> + Err(Errno::BUSY) => {}
This could use a comment.
> + Err(e) => return Err(format!("Cannot purge {:?}: {e}", self.joined_path(path))),
> + }
> + if !self.open_sub_cgroup(path, true, true)? {
> + return Ok(());
> + }
> +
> + rustix::fs::flock(
> + self.fd[self.fd.len() - 2].0.as_fd(),
> + FlockOperation::LockShared,
We already must have at least a shared lock on this at this point, no?
I don't think we need another one.
> + )
> + .map_err(|e| format!("Cannot relock {:?}: {e}", self.path.parent()))?;
> + self.write_cgroup_value("cgroup.kill", "1")?;
> + self.wait_for_empty()
> + .map_err(|e| format!("Cannot wait for cgroup to become empty: {e}"))?;
> + let fd = self.fd.pop().unwrap().0;
> + let v = (|| {
> + remove_recursively(fd, 1000)
> + .map_err(|e| format!("Cannot remove {:?}: {e}", self.path))?;
> + rustix::fs::flock(self.as_fd(), FlockOperation::LockExclusive)
> + .map_err(|e| format!("Cannot lock {:?}: {e}", self.path))?;
We already checked self.exclusive above, meaning we already have this
lock on self?
> + match rustix::fs::unlinkat(
> + self.as_fd(),
> + Path::new(self.path.file_name().unwrap()),
I am too confused about what self.path is to know what to make of this.
self.as_fd() should be the fd of the cgroup directory, and self.path
sounds like it should be the path to this cgroup, so how can this cgroup
directory have self.path.file_name() within it? This needs clearer
names or a refactor or something.
> + AtFlags::REMOVEDIR,
> + ) {
> + // something might have re-created the cgroup in the meantime, which is okay
> + Ok(()) | Err(Errno::BUSY) => Ok(()),
> + Err(e) => Err(format!("Cannot lock {:?}: {e}", self.path)),
> + }
> + })();
> + assert!(self.path.pop());
I don't get this. Where was it pushed? (But would prefer to just not
attempt to track this, as mentioned above.)
> + v
> + }
> +
> + pub(crate) fn write_cgroup_value(&self, name: &str, value: &str) -> Result<(), String> {
> + let path = Path::new(name);
> + let fd = rustix::fs::openat2(
> + self.as_fd(),
> + path,
> + OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::WRONLY,
> + Mode::empty(),
> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
> + )
> + .map_err(|e| format!("Cannot open {:?}: {}", self.joined_path(Path::new(name)), e))?;
Could also use ResolveFlags::MAGIC_LINKS and be unified with the other
openat2 invocations maybe? I don't understand why this is NOATIME but
others aren't.
> + File::from(fd).write_all(value.as_bytes()).map_err(|e| {
> + format!(
> + "Cannot write {:?} to {:?}: {}",
> + value,
> + self.joined_path(Path::new(name)),
> + e
> + )
> + })
> + }
> +}
> +
> +fn open_subtree_raw(path: &Path, dirfd: BorrowedFd<'_>) -> Result<OwnedFd, Errno> {
> + rustix::fs::openat2(
> + dirfd,
> + path,
> + OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY,
> + Mode::empty(),
> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
> + )
> +}
> +
> +fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> {
> + if remaining_depth < 1 {
> + panic!("control groups too deeply nested");
> + }
> + let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating");
> + while let Some(element) = d.next() {
> + let element = element.expect("Iterating through a cgroup directory failed?");
> + if element.file_type() != rustix::fs::FileType::Directory {
> + continue;
> + }
> +
> + let remaining_depth = remaining_depth - 1;
> + let d: &rustix::fs::Dir = &d;
> + let dirfd = d.fd().unwrap();
> + let path = element.file_name();
> + remove_all(remaining_depth, dirfd, path)?;
> + }
> + drop(d);
> + Ok(())
> +}
> +
> +fn remove_all(
> + remaining_depth: usize,
> + dirfd: BorrowedFd<'_>,
> + path: &std::ffi::CStr,
> +) -> Result<(), Errno> {
> + if path == c"." || path == c".." {
> + return Ok(());
> + }
> + if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() {
> + return Ok(());
> + }
> + let fd = rustix::fs::openat2(
> + dirfd,
> + path,
> + OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY,
> + Mode::empty(),
> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
> + )?;
> + remove_recursively(fd, remaining_depth)?;
> + rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?;
> + Ok(())
> +}
Could we save a lot of code by calling std::fs::remove_dir_all with a
/proc/self/fd path? It's already documented to ignore symlinks.
> diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
> new file mode 100644
> index 0000000000000000000000000000000000000000..2e7a4e25213a4aa2449b292f8005e1da23cbb1f9
> --- /dev/null
> +++ b/tools/cgroup-setup/src/main.rs
> @@ -0,0 +1,347 @@
> +// SPDX-License-Identifier: EUPL-1.2+
> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +
> +use std::{
> + ffi::{OsStr, OsString},
> + os::unix::prelude::*,
> + path::{Path, PathBuf},
> +};
> +
> +use crate::cgroup::Cgroup;
> +
> +mod cgroup;
> +
> +fn check_path(path: &OsStr) -> Result<(), String> {
> + if path.is_empty() {
> + return Ok(());
> + }
> +
> + for component in path.as_bytes().split(|&b| b == b'/') {
Would be a lot nicer to take &Path and use Path::components.
> + match component {
> + b"" | b"." | b".." => {
> + return Err(format!("Path {path:?} has empty, ., or .. component"));
> + }
> + // Cannot happen: command line arguments have no NUL byte,
> + // and /proc/self/cgroup having a NUL byte is a kernel bug.
> + _ if component.contains(&b'\0') => panic!("Path {path:?} has NUL byte"),
It is in fact an invariant of both the Path and OsStr types (on Unix)
that there are no NUL bytes, so there is no need to check for it at all.
> + _ if component.len() > 255 => {
> + return Err(format!(
> + "Path {path:?} has component {:?} that is longer than 255 bytes",
> + OsStr::from_bytes(component)
> + ));
> + }
Why on earth would we need to check for this? This limit is totally up
to the kernel. Our code won't break with an excessively long path
component.
> + _ => {}
> + }
> + }
> +
> + Ok(())
> +}
> +
> +/// Get the path of the cgroup for the provided command-line argument.
> +/// Returns an empty path if the path is "/", or if it is "." and the
> +/// current cgroup is "/".
Please standardize terminology between "current cgroup" and "local
cgroup", or if those are not the same thing use clearer phrasing.
I hope all the different modes of this function are actually needed…
(I assume they are but haven't reviewed the patches that make use of it
yet.)
> +///
> +/// # Errors
> +///
> +/// Fails if the provided path is invalid or empty, or if it is relative
> +/// and the local cgroup cannot be determined.
> +fn get_cgroup(cgroup_path: OsString) -> Result<PathBuf, String> {
Please call this function something clearer. Also using OsString where
PathBuf would be more appropriate again…
> + if cgroup_path.as_bytes().starts_with(b"/") {
… which would allow using Path::is_absolute if fixed.
> + let mut cgroup_path = cgroup_path.into_vec();
> + cgroup_path.remove(0);
Please use Path functions rather than byte manipulation like this.
> + if cgroup_path.is_empty() {
> + return Err("cgroup path cannot be /".to_owned());
> + }
> + let cgroup_path = OsString::from_vec(cgroup_path);
> + check_path(&cgroup_path)?;
> + Ok(cgroup_path.into())
> + } else if cgroup_path.is_empty() {
> + Err("cgroup path cannot be empty".to_owned())
> + } else {
> + check_path(&cgroup_path)?;
> + let mut local_cgroup = local_cgroup()?;
Can you reorder all these function definitions to be something more
sensible? get_cgroup is here, near the start of main.rs, but
local_cgroup, which it is the only direct caller of, is all the way at
the other end. (Personally I like to define utility functions directly
above their only caller — I think that creates the most natural reading
flow.)
> + local_cgroup.push(cgroup_path);
> + Ok(local_cgroup)
> + }
> +}
> +
> +/// Open the cgroup corresponding to the provided path.
> +/// It must have already been made relative to `/sys/fs/cgroup`.
> +///
> +/// # Errors
> +///
> +/// Fails if the cgroup operation fails.
This "Errors" section is just stating the obvious.
> +fn open_cgroup(path: &Path, exclusive: bool) -> Result<Cgroup, String> {
> + if path.as_os_str().is_empty() {
> + Cgroup::new(exclusive)
> + } else {
> + let mut cgroup = Cgroup::new(false)?;
> + cgroup.open_sub_cgroup(path, exclusive, false)?;
> + Ok(cgroup)
> + }
> +}
I feel like it would be more natural if Cgroup::new just took two
parameters and worked this way.
> +
> +/// Open the cgroup corresponding to the provided path's parent.
> +/// It is made relative to the process's own cgroup if needed.
> +///
> +/// # Errors
> +///
> +/// Fails if the cgroup operation fails.
> +fn open_relative_cgroup(arg: OsString) -> Result<(PathBuf, Cgroup), String> {
> + let path = get_cgroup(arg)?;
> + let cgroup = open_cgroup(path.parent().expect("always has a parent"), true)?;
> + Ok((path, cgroup))
> +}
This is a deeply confusing function. Why is opening a cgroup for the
path's _parent_ an operation that we should have a dedicated function
for? Why not have the caller pass in the actual path of the cgroup it
wants to open? This really looks like a function that's doing as many
different things at it has lines, and should just be inlined.
> +
> +fn main() {
> + let mut args = std::env::args_os();
> + let Some(prog_name) = args.next() else {
> + eprintln!("No command line arguments (argv[0] is NULL)");
> + std::process::exit(1);
> + };
> + match main_(&prog_name, args) {
> + Ok(()) => {}
> + Err(e) => {
> + eprintln!("{prog_name:?}: {}", e);
> + std::process::exit(1);
> + }
> + }
> +}
> +
> +fn main_(prog_name: &OsStr, mut args: std::env::ArgsOs) -> Result<(), String> {
main_ is a bit confusingly similar to name. I like "run" for this sort
of function, and we use that elsewhere.
> + match prog_name
> + .as_bytes()
> + .split(|&b| b == b'/')
> + .next_back()
> + .unwrap()
Another place that should use Path.
> + {
> + b"cgroup-s6-finish" => {
> + return s6_finish(&mut args);
> + }
> + b"cgroup-setup" => {}
It is inconsistent for this not to also be its own function. Then the
match could just be an expression that returned the result of the
appropriate function.
> + b"cgroup-purge" => {
> + if args.len() != 1 {
> + return Err(format!(
> + "cgroup-purge takes one argument, got {}",
> + args.len()
> + ));
> + }
> + let cgroup_path = args.next().unwrap();
> + let (path, mut cgroup) = open_relative_cgroup(cgroup_path)?;
> + let cgroup_target = Path::new(path.file_name().unwrap());
> + return cgroup.purge(cgroup_target);
> + }
> + _ => {
> + return Err(format!(
> + "must be invoked as \"cgroup-setup\" \
> + \"cgroup-purge\", or \"cgroup-s6-finish\", \
> + got {prog_name:?}",
> + ));
> + }
> + };
> + let mut leaf = false;
> + let mut cgroup_path;
> + let mut delegate = false;
> + let mut init_subtree = false;
> + let mut child_name: Option<&'static OsStr> = None;
> + let mut wait = true;
> + loop {
> + cgroup_path = args.next();
> + let Some(ref arg_) = cgroup_path else {
> + break;
> + };
> + let arg_ = arg_.as_bytes();
> + if arg_ == b"--" {
> + cgroup_path = args.next();
> + break;
> + }
> + if !arg_.starts_with(b"-") {
> + break;
> + }
> +
> + if !arg_.starts_with(b"--") {
> + return Err("takes no short options".to_owned());
> + }
> +
> + match &arg_[2..] {
> + b"leaf" => leaf = true,
> + b"delegate" => delegate = true,
> + b"init-subtree" => init_subtree = true,
> + b"wait" => wait = true,
> + b"no-wait" => wait = false,
> + b"child-name" if child_name.is_none() => match args.next() {
> + Some(arg) => child_name = Some(arg.leak()),
> + None => return Err("--child-name: missing argument".to_owned()),
> + },
> + b"child-name" => return Err("--child-name: cannot be used twice".to_owned()),
> + arg => match str::from_utf8(arg) {
> + Ok(e) => return Err(format!("unknown long option {e:?}")),
> + Err(_) => return Err("long option isn't UTF-8".to_owned()),
> + },
> + }
> + }
> +
> + let default_child_name = OsStr::from_bytes(b"$inner.service");
> +
> + let child_name = Path::new(child_name.unwrap_or(default_child_name));
> +
> + let Some(mut cgroup_path) = cgroup_path else {
> + return Err("have no positional arguments, expected at least 1".to_owned());
> + };
> +
> + // Allow --init-subtree .
> + if cgroup_path.as_bytes() == b"." && init_subtree && !leaf {
> + cgroup_path = child_name.to_owned().into();
> + leaf = true;
> + }
> +
> + let (path, mut cgroup) = open_relative_cgroup(cgroup_path)?;
> + let cgroup_target = Path::new(path.file_name().unwrap());
> + cgroup
> + .make_child(cgroup_target)
> + .map_err(|e| format!("Cannot make child cgroup: {e}"))?;
> + if wait {
> + cgroup
> + .wait_for_empty()
> + .map_err(|e| format!("Cannot wait for {path:?} to be empty: {e}"))?;
> + }
> + let pid = std::process::id().to_string();
> + if leaf {
> + if args.len() != 0 {
> + // If we aren't delegating any cgroups, don't create a sub-cgroup.
> + cgroup
> + .write_cgroup_value("cgroup.procs", &pid)
> + .map_err(|e| format!("Cannot write to {path:?}/cgroup.procs: {e}"))?;
> + }
> + } else {
> + // If the child process will need to manage cgroups itself, it will need
> + // to set up a sub-cgroup due to the "no internal processes" rule. It's
> + // simplest to just do it automatically.
> + cgroup.make_child(Path::new(child_name)).map_err(|e| {
> + format!(
> + "Cannot create child cgroup {}/{}: {e}",
> + path.display(),
> + child_name.display()
> + )
> + })?;
> + if args.len() != 0 {
> + cgroup
> + .write_cgroup_value("cgroup.procs", &pid)
> + .map_err(|e| {
> + format!(
> + "Cannot write to {}/{}/cgroup.procs: {e}",
> + path.display(),
> + child_name.display()
> + )
> + })?;
> + }
> + }
> + if !leaf {
> + cgroup.enable_subtree_control(2)?;
> + }
> + if init_subtree {
> + cgroup.enable_subtree_control(1)?;
> + }
> + if delegate {
> + cgroup
> + .enable_delegation(1)
> + .map_err(|e| format!("Cannot enable cgroup delegation in {path:?}: {e}"))?;
> + }
> + let Some(program_name) = args.next() else {
> + return Ok(());
> + };
> + let e = std::process::Command::new(&program_name).args(args).exec();
> + Err(format!("Cannot spawn child {:?}: {}", program_name, e))
> +}
> +
> +fn s6_finish(args: &mut std::env::ArgsOs) -> Result<(), String> {
> + if args.len() < 3 {
> + return Err(format!(
> + "s6 finish scripts take at least 3 arguments, got {}",
> + args.len()
> + ));
> + }
> + let status = parse_digit_string(&args.next().unwrap(), "exit status")?;
> + let signal = args.next().unwrap();
> + let signal = if status == 256 {
> + Some(parse_digit_string(&signal, "signal number")?)
> + } else {
> + None
> + };
> + let service = args.next().unwrap();
> +
> + let (path, mut cgroup) = open_relative_cgroup(service)?;
> + let cgroup_target = Path::new(path.file_name().unwrap());
> + let exit_125 = if let Some(signal) = signal {
> + match signal as libc::c_int {
> + libc::SIGBUS
> + | libc::SIGFPE
> + | libc::SIGABRT
> + | libc::SIGTRAP
> + | libc::SIGSEGV
> + | libc::SIGILL => {
> + // Process *crashed*, indicating a *possible exploit attempt*.
> + // s6 should *not* restart it. This is distinct from a Rust panic,
> + // which is much less likely to indicate memory corruption.
> + true
> + }
This has absolutely nothing to do with cgroups. If you want to have
some common finish behaviour, a program called cgroup-setup is not the
place for it. I don't think there's any need for a separate
cgroup-s6-finish mode (as opposed to cgroup-purge).
> + _ => false,
> + }
> + } else {
> + false
> + };
> + if exit_125 {
> + // Ignore panics. Exit status is more important.
> + // We already had a core dump.
> + let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
> + match cgroup.purge(cgroup_target) {
> + Ok(()) => {}
> + Err(e) => {
> + eprintln!("cgroup-s6-finish: Failed to purge cgroup: {e}")
> + }
> + };
> + }));
> + std::process::exit(125)
> + } else {
> + cgroup.purge(cgroup_target)
> + }
> +}
> +
> +fn parse_digit_string(digits: &OsStr, msg: &str) -> Result<u16, String> {
> + let checked = match str::from_utf8(digits.as_bytes()) {
> + Ok(s) => s,
> + Err(e) => return Err(format!("{msg} is not UTF-8: {e}")),
> + };
> + let r = checked
> + .parse::<u16>()
> + .map_err(|e| format!("{msg} {digits:?} is a bad 16-bit number: {e}"))?;
> + match checked.as_bytes() {
> + b"0" | [b'1'..=b'9', ..] => Ok(r),
> + [b'0', ..] => Err(format!("{msg} {} has a leading 0", digits.display())),
> + _ => Err(format!("{msg} {} starts with +", digits.display())),
> + }
Surely we trust s6 to turn a number into a string. These checks add nothing.
> +}
> +
> +fn local_cgroup() -> Result<PathBuf, String> {
> + let mut local_cgroup: Vec<u8> = std::fs::read("/proc/thread-self/cgroup")
> + .map_err(|e| format!("cannot read /proc/thread-self/cgroup: {e}"))?;
> + let local_cgroup_len = local_cgroup.len();
> + if local_cgroup_len < 5
> + || local_cgroup[..4] != *b"0::/"
> + || local_cgroup[local_cgroup_len - 1] != b'\n'
> + || local_cgroup[4..local_cgroup_len - 1].contains(&b'\n')
Last time I suggested a clearer way of doing this, but it has instead
got even less clear.
(I'm not sure why we'd care if there's a newline specifically, as
opposed to any other control character.)
> + {
> + // It's possible to get here if the cgroup path contains a newline,
> + // but that never happens in Spectrum.
> + return Err(format!(
> + "Invalid contents {local_cgroup:?} of /proc/thread-self/cgroup - \
> + do you have cgroups v1 mounted instead of cgroups v2?"
> + ));
> + }
> +
> + local_cgroup.copy_within(4..local_cgroup_len - 1, 0);
> + local_cgroup.truncate(local_cgroup_len - 5);
> + let local_cgroup = OsString::from_vec(local_cgroup);
> + check_path(&local_cgroup).unwrap();
Why do we need to do this? You're worried the kernel is going to start
including .. components in /proc/thread-self/cgroup?
> + Ok(PathBuf::from(local_cgroup))
> +}
>
> --
> 2.55.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 02/20] tools: Add control group manager
2026-07-22 16:01 ` Alyssa Ross
@ 2026-07-23 23:07 ` Demi Marie Obenour
2026-07-27 12:10 ` Alyssa Ross
0 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-23 23:07 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1: Type: text/plain, Size: 49659 bytes --]
On 7/22/26 12:01, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> The cgroup-setup Rust program can create and purge cgroups. It can also
>> wait for one to become empty, spawn a program in a cgroup, and more. In
>> the future, it will also support cgroup-based resource control. Locking
>> is used to ensure that concurrent invocations are safe.
>>
>> This program can also be used in an s6 finish script. When passed the
>> args of such a script, it automatically purges the correct cgroup. It
>> also tells s6 to not restart the service if it dumped core. Core dumps
>> are often due to memory corruption, and automatically restarting a
>> service that dumped core makes memory corruption attacks easier.
>>
>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>> ---
>> .codespellrc | 2 +-
>> host/rootfs/default.nix | 6 +-
>> host/rootfs/file-list.mk | 2 +
>> host/rootfs/image/usr/bin/cgroup-purge | 1 +
>> host/rootfs/image/usr/bin/cgroup-s6-finish | 1 +
>> pkgs/default.nix | 1 +
>> tools/cgroup-setup/Cargo.lock | 67 ++++++
>> tools/cgroup-setup/Cargo.lock.license | 2 +
>> tools/cgroup-setup/Cargo.toml | 11 +
>> tools/cgroup-setup/default.nix | 18 ++
>> tools/cgroup-setup/src/cgroup.rs | 349 +++++++++++++++++++++++++++++
>> tools/cgroup-setup/src/main.rs | 347 ++++++++++++++++++++++++++++
>> 12 files changed, 803 insertions(+), 4 deletions(-)
>
>> diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
>> index 3899d620717fc97f42e669e5313c4100dcf5b1cd..e1280ab56d8797e40b9b1c584ab0daef3cda41d7 100644
>> --- a/host/rootfs/file-list.mk
>> +++ b/host/rootfs/file-list.mk
>> @@ -79,6 +79,8 @@ LINKS = \
>> image/etc/s6-linux-init/run-image/service/vmm/template/run \
>> image/lib \
>> image/sbin \
>> + image/usr/bin/cgroup-purge \
>> + image/usr/bin/cgroup-s6-finish \
>> image/usr/bin/systemd-udevd
>>
>> S6_RC_FILES = \
>> diff --git a/host/rootfs/image/usr/bin/cgroup-purge b/host/rootfs/image/usr/bin/cgroup-purge
>> new file mode 120000
>> index 0000000000000000000000000000000000000000..a0c8d8e144d72b69c613eb0613e39acc9df979df
>> --- /dev/null
>> +++ b/host/rootfs/image/usr/bin/cgroup-purge
>> @@ -0,0 +1 @@
>> +cgroup-setup
>> \ No newline at end of file
>> diff --git a/host/rootfs/image/usr/bin/cgroup-s6-finish b/host/rootfs/image/usr/bin/cgroup-s6-finish
>> new file mode 120000
>> index 0000000000000000000000000000000000000000..a0c8d8e144d72b69c613eb0613e39acc9df979df
>> --- /dev/null
>> +++ b/host/rootfs/image/usr/bin/cgroup-s6-finish
>> @@ -0,0 +1 @@
>> +cgroup-setup
>> \ No newline at end of file
>
> Usually packages that expect to be invoked via symlinks like this
> (coreutils, busybox, execline) install their own symlinks, rather than
> expecting systems to create them. I think these would be more
> appropriate in a postInstall in tools/cgroup-setup/default.nix.
>
>> diff --git a/pkgs/default.nix b/pkgs/default.nix
>> index 44f7b5ff78cb6b9e755292a6a417d0b627ed3fb0..0a13393164ad5d7f752e630763f3f97166479af5 100644
>> --- a/pkgs/default.nix
>> +++ b/pkgs/default.nix
>> @@ -51,6 +51,7 @@ let
>> driverSupport = true;
>> };
>> spectrum-router = self.callSpectrumPackage ../tools/router {};
>> + spectrum-cgroup-setup = self.callSpectrumPackage ../tools/cgroup-setup {};
>> xdg-desktop-portal-spectrum-host =
>> self.callSpectrumPackage ../tools/xdg-desktop-portal-spectrum-host {};
>>
>> diff --git a/tools/cgroup-setup/Cargo.lock b/tools/cgroup-setup/Cargo.lock
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..fe967b3aa02c296c87b6b36ac59253dbe0a32de9
>> --- /dev/null
>> +++ b/tools/cgroup-setup/Cargo.lock
>> @@ -0,0 +1,67 @@
>> +# This file is automatically @generated by Cargo.
>> +# It is not intended for manual editing.
>> +version = 4
>> +
>> +[[package]]
>> +name = "bitflags"
>> +version = "2.11.1"
>> +source = "registry+https://github.com/rust-lang/crates.io-index"
>> +checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3"
>> +
>> +[[package]]
>> +name = "cgroup-setup"
>> +version = "0.0.0"
>> +dependencies = [
>> + "libc",
>> + "rustix",
>> +]
>> +
>> +[[package]]
>> +name = "errno"
>> +version = "0.3.14"
>> +source = "registry+https://github.com/rust-lang/crates.io-index"
>> +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
>> +dependencies = [
>> + "libc",
>> + "windows-sys",
>> +]
>> +
>> +[[package]]
>> +name = "libc"
>> +version = "0.2.186"
>> +source = "registry+https://github.com/rust-lang/crates.io-index"
>> +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
>> +
>> +[[package]]
>> +name = "linux-raw-sys"
>> +version = "0.12.1"
>> +source = "registry+https://github.com/rust-lang/crates.io-index"
>> +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
>> +
>> +[[package]]
>> +name = "rustix"
>> +version = "1.1.4"
>> +source = "registry+https://github.com/rust-lang/crates.io-index"
>> +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
>> +dependencies = [
>> + "bitflags",
>> + "errno",
>> + "libc",
>> + "linux-raw-sys",
>> + "windows-sys",
>> +]
>> +
>> +[[package]]
>> +name = "windows-link"
>> +version = "0.2.1"
>> +source = "registry+https://github.com/rust-lang/crates.io-index"
>> +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
>> +
>> +[[package]]
>> +name = "windows-sys"
>> +version = "0.61.2"
>> +source = "registry+https://github.com/rust-lang/crates.io-index"
>> +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
>> +dependencies = [
>> + "windows-link",
>> +]
>> diff --git a/tools/cgroup-setup/Cargo.lock.license b/tools/cgroup-setup/Cargo.lock.license
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..aa108acd23886d8302eaf7babff90d1b08ae19fb
>> --- /dev/null
>> +++ b/tools/cgroup-setup/Cargo.lock.license
>> @@ -0,0 +1,2 @@
>> +SPDX-License-Identifier: EUPL-1.2+
>> +SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>
> I think this should be CC0-1.0 like every other Cargo.lock.license.
> There's nothing copyrightable about it.
Will fix in v5.
>> diff --git a/tools/cgroup-setup/Cargo.toml b/tools/cgroup-setup/Cargo.toml
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..7ed6d6a0ea3bbfc4064b9f39383d0788c4bd84e5
>> --- /dev/null
>> +++ b/tools/cgroup-setup/Cargo.toml
>> @@ -0,0 +1,11 @@
>> +# SPDX-License-Identifier: CC0-1.0
>> +# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
>
> I surely did not contribute anything copyrightable to this.
Okay, I'll remove this in v5.
>> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>> +
>> +[package]
>> +name = "cgroup-setup"
>> +edition = "2024"
>> +
>> +[dependencies]
>> +libc = "0.2.177"
>> +rustix = { version = "1.1.2", features = ["fs"] }
>
>> diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..c953d26badfdac0a1e3d7057a867aec3b3247e18
>> --- /dev/null
>> +++ b/tools/cgroup-setup/src/cgroup.rs
>> @@ -0,0 +1,349 @@
>> +// SPDX-License-Identifier: EUPL-1.2+
>> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>> +
>> +use std::ffi::OsStr;
>> +use std::fmt::Display;
>> +use std::fs::File;
>> +use std::io::{Read as _, Seek as _, Write as _};
>> +use std::os::unix::prelude::*;
>> +
>> +use std::path::{Component, Path, PathBuf};
>> +
>> +use rustix::fs::{AtFlags, FlockOperation, XattrFlags};
>> +use rustix::{
>> + fs::{Mode, OFlags, ResolveFlags},
>> + io::Errno,
>> +};
>> +
>> +#[derive(Debug)]
>> +pub(crate) struct Cgroup {
>> + path: PathBuf,
>> + fd: Vec<(OwnedFd, bool)>,
>
> There's no point storing all these exclusivity bools, is there? I think
> only the last one is ever checked, so we could make things tighter and
> clearer like this, where we only track the exclusivity of the last fd:
Cgroup::enable_subtree_control() checks the exclusivity
of the caller-provided depth. Line 232 of main.rs calls
enable_subtree_control(2).
These bools are only used in assertions, so they could be removed.
I will leave that up to you. The advantage of keeping them is that a
panic is vastly easier to debug than a race condition due to improper
locking.
> fd: Vec<OwnedFd>,
> exclusive: bool,
>
>> +}
>> +
>> +impl AsFd for Cgroup {
>> + fn as_fd(&self) -> BorrowedFd<'_> {
>> + self.fd.last().unwrap().0.as_fd()
>> + }
>> +}
>> +
>> +fn assert_single_component(component: &[u8]) {
>
> Why not &Path, which is already guaranteed not to have a NUL byte?
> Perhaps this whole thing could be simplified to
> Some(component).as_os_str() == component.file_name()? Maybe that's too
> clever, though…
Path isn't actually guaranteed to not have a NUL byte.
File::open(Path::new("\0")) fails with InvalidInput rather than
panicking.
I'm very used to writing this kind of code in C, so I went with a C-like
style instead of using Rust stdlib APIs. I don't like having extra
abstractions in this kind of code, as it obscures what is going on
under the hood. That is less important here, but it's very important
in programs like mount-flatpak.
For instance, RESOLVE_BENEATH or RESOLVE_IN_ROOT can result in
spurious EAGAIN errors, which libpathrs resolves using a retry
loop that has a failure rate of about 0.1% when the system is being
hammered by repeatedly calling rename(). This is needed for container
runtimes that need to follow symlinks in their target filesystems,
but mount-flatpak doesn't need to do that when traversing OSTree
repositories. For mount-flatpak, an explicit check that the path
doesn't have "." or ".." components, and using RESOLVE_NO_SYMLINKS |
RESOLVE_NO_MAGICLINKS is just as secure and lacks this problem.
>> + match component {
>> + b"" | b"." | b".." => panic!("bad component"),
>> + _ if component.contains(&b'\0') => panic!("NUL in component"),
>> + _ if component.contains(&b'/') => panic!("/ in component"),
>> + _ => {}
>> + }
>> +}
>> +
>> +impl Cgroup {
>> + pub fn new(exclusive: bool) -> Result<Self, String> {
>> + let cgroup_root = rustix::fs::openat2(
>> + rustix::fs::CWD,
>> + Path::new("/sys/fs/cgroup"),
>> + OFlags::DIRECTORY | OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW,
>> + Mode::empty(),
>> + ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_SYMLINKS,
>> + )
>
> I pointed out in my review of v2 that OFlags::NOFOLLOW is redundant with
> ResolveFlags::NO_SYMLINKS, but now it seesm to have come back across the
> board.
Whoops, sorry about that. Will fix in v5.
>> + .map_err(|e| format!("Cannot open /sys/fs/cgroup: {e}"))?;
>> +
>> + let lock_operation = if exclusive {
>> + FlockOperation::LockExclusive
>> + } else {
>> + FlockOperation::LockShared
>> + };
>> + rustix::fs::flock(cgroup_root.as_fd(), lock_operation)
>> + .map_err(|e| format!("Cannot lock /sys/fs/cgroup: {e}"))?;
>> + Ok(Self {
>> + path: PathBuf::from("/sys/fs/cgroup"),
>> + fd: vec![(cgroup_root, exclusive)],
>> + })
>> + }
>> +
>> + pub fn enable_delegation(&self, depth: usize) -> Result<(), Errno> {
>> + let (fd, exclusive) = &self.fd[self.fd.len() - depth];
>> + assert!(exclusive);
>> + rustix::fs::fsetxattr(fd.as_fd(), c"user.delegate", b"1", XattrFlags::empty())
>> + }
>> +
>> + pub fn enable_subtree_control(&self, depth: usize) -> Result<(), String> {
>> + let (fd, exclusive) = &self.fd[self.fd.len() - depth];
>> + assert!(exclusive);
>> + let p = Path::new("cgroup.controllers");
>> + let mut buf = self.read_control_file(fd.as_fd(), p)?;
>> + let mut subtree = vec![];
>> + if buf.ends_with(b"\n") {
>> + buf.pop();
>> + }
>> + for controller in buf.split(|&b| b == b' ').filter(|e| !e.is_empty()) {
>> + for &c in controller {
>> + if c <= b' ' || c >= 0x7F {
>> + return Err(format!("Bad byte {c} in cgroup.controllers"));
>> + }
>> + }
>> + if !subtree.is_empty() {
>> + subtree.push(b' ');
>> + }
>> + subtree.push(b'+');
>> + subtree.extend_from_slice(controller);
>> + }
>> + if !subtree.is_empty() {
>> + self.write_cgroup_value("cgroup.subtree_control", str::from_utf8(&subtree).unwrap())?;
>> + }
>> + Ok(())
>> + }
>> +
>> + pub fn read_control_file(&self, fd: BorrowedFd, p: &Path) -> Result<Vec<u8>, String> {
>> + let mut buf = Vec::new();
>> + let err = |e: &dyn Display, p: &Path, msg: &str| {
>> + let path = self.path.join(p);
>> + format!("Cannot {msg} {path:?}: {e}")
>> + };
>> + File::from(open_subtree_raw(Path::new(p), fd.as_fd()).map_err(|e| err(&e, p, "open"))?)
>
> If we're using it for opening files, open_subtree_raw is probably misnamed.
Yup! Do you have a suggestion for improving it? Maybe open_child()?
>> + .read_to_end(&mut buf)
>> + .map_err(|e| err(&e, p, "read"))?;
>> + Ok(buf)
>> + }
>> +
>> + /// Open a single component as a sub-cgroup
>> + fn open_sub_cgroup_raw(&self, access: OFlags, component: &[u8]) -> Result<OwnedFd, Errno> {
>> + assert_single_component(component);
>> + rustix::fs::openat2(
>> + self.as_fd(),
>> + Path::new(OsStr::from_bytes(component)),
>> + OFlags::CLOEXEC | OFlags::NOFOLLOW | access,
>> + Mode::empty(),
>> + ResolveFlags::NO_SYMLINKS
>> + | ResolveFlags::NO_MAGICLINKS
>> + | ResolveFlags::BENEATH
>> + | ResolveFlags::NO_XDEV,
>
> This is also doing exactly the same thing as open_subtree_raw, except it
> allows changing the access mode, and also sets NO_MAGICLINKS. I don't
> think any of the other callers of open_subtree_raw would need to open
> magic links, so maybe this is evidence this should be unified with them?
It probably should.
>> + )
>> + }
>> +
>> + pub fn open_sub_cgroup(
>> + &mut self,
>> + path: &std::path::Path,
>> + exclusive: bool,
>> + allow_missing: bool,
>> + ) -> Result<bool, String> {
>> + let mut iter = path.components().peekable();
>> + while let Some(component) = iter.next() {
>
> Perhaps would be nicer:
>
> let mut components = path.components().peekable();
> for component in components {
That results in a borrowcheck error. The for loop takes ownership
of the iterator, but .peek() is called inside the loop.
>> + let component = match component {
>> + Component::Normal(component) => component,
>> + _ => unreachable!(),
>> + };
>> + let sub_fd = match self
>> + .open_sub_cgroup_raw(OFlags::DIRECTORY | OFlags::RDONLY, component.as_bytes())
>> + {
>> + Ok(sub_fd) => {
>> + self.path.push(component);
>
> I would really like to not try to store self.path. It seems very
> complicated to track. It's also very unclear to me from the name (and
> the code) what it is. Is it the path to the cgroup itself, or to its
> parent? It looks to me like it should be the cgroup itself, but then
> what's going on in purge?
It's the path to the cgroup itself, relative to /sys/fs/cgroup. Its only
purpose is for logging.
> We could actually improve readability of this quite complicated function
> even further if you find it acceptable to just use Errno for the error
> type. In that case, we'd just return Result<(), Errno>, and callers
> would check for Errno::NOENT if they wanted to allow missing. Then we
> could just completely drop that argument. In my opinion it would be
> worth it to move complexity out of here.
I can do this, but it would result in much worse error messages: the
error would only reference the file name, not the full cgroup path.
Which would you prefer?
>> + sub_fd
>> + }
>> + Err(Errno::NOENT) if allow_missing => return Ok(false),
>> + Err(e) => {
>> + return Err(format!(
>> + "Cannot open sub-cgroup {component:?} of {:?}: {e}",
>> + self.path
>> + ));
>> + }
>> + };
>> + let exclusive = exclusive && iter.peek().is_none();
>> + let lock_operation = if exclusive {
>> + FlockOperation::LockExclusive
>> + } else {
>> + FlockOperation::LockShared
>> + };
>> + rustix::fs::flock(sub_fd.as_fd(), lock_operation).map_err(|e| {
>> + let msg = format!("Cannot lock sub-cgroup {:?}: {e}", self.path);
>> + self.path.pop();
>> + msg
>> + })?;
>> + self.fd.push((sub_fd, exclusive));
>> + }
>> + Ok(true)
>> + }
>> +
>> + pub fn open_subtree(&self, path: &std::path::Path) -> Result<OwnedFd, Errno> {
>> + let dirfd = self.as_fd();
>> + open_subtree_raw(path, dirfd)
>> + }
>
> If open_subtree_raw just took &dyn AsFd, there'd be no need for this
> method.
Nice catch! Will change in v5.
>> +
>> + fn exclusive(&self) -> bool {
>> + self.fd.last().unwrap().1
>> + }
>> +
>> + pub fn joined_path(&self, p: &Path) -> PathBuf {
>> + let mut owned_p = self.path.clone();
>> + owned_p.push(p);
>> + owned_p
>> + }
>> +
>> + pub fn wait_for_empty(&self) -> std::io::Result<()> {
>> + assert!(self.exclusive());
>> + let wait_file = self.open_subtree(std::path::Path::new("cgroup.events"))?;
>> + let poll_fd = wait_file.as_raw_fd();
>> + let mut wait_fd = File::from(wait_file);
>> + let mut fds = libc::pollfd {
>> + fd: poll_fd,
>> + events: libc::POLLPRI | libc::POLLERR,
>> + revents: 0,
>> + };
>> + let mut v = vec![];
>> + loop {
>> + v.clear();
>> + wait_fd
>> + .seek(std::io::SeekFrom::Start(0))
>> + .expect("Seek on control group file should succeed");
>> + wait_fd
>> + .read_to_end(&mut v)
>> + .expect("reading from control group should work");
>> + if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
>> + break;
>> + }
>> + // SAFETY: FFI call, valid arguments, fds contains 1 element
>> + if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
>> + panic!("poll failed");
>> + }
>> + }
>
> Are you 100% confident that this doesn't race? I don't understand why
> poll would be triggered in this scenario:
>
> 1. "1" is written to cgroup.kill
> 2. Every process in the cgroup exits and is reaped.
> 3. cgroup.events is opened, with the cgroup already empty.
>
> Are you not relying on 2 happening after 3? Presumably if you open
> cgroup.events for a cgroup that's already empty, you're not going to get
> a poll event to tell you it's empty.
In that case, cgroup.events will include a "populated 0"
line, so poll will not be called.
>> + Ok(())
>> + }
>> +
>> + pub(crate) fn make_child(&mut self, path: &Path) -> Result<(), Errno> {
>> + assert!(self.exclusive());
>> + let component = path.as_os_str().as_bytes();
>> + assert_single_component(component);
>> + match rustix::fs::mkdirat(
>> + self.as_fd(),
>> + path,
>> + Mode::RUSR
>> + | Mode::WUSR
>> + | Mode::XUSR
>> + | Mode::RGRP
>> + | Mode::XGRP
>> + | Mode::ROTH
>> + | Mode::XOTH,
>> + ) {
>> + Ok(()) | Err(Errno::EXIST) => {}
>> + bad => return bad,
>> + }
>> + let p = self.open_sub_cgroup_raw(OFlags::RDONLY | OFlags::DIRECTORY, component)?;
>> + // exclusive lock on parent acts as exclusive lock on child
>> + self.fd.push((p, true));
>> + self.path.push(path);
>> + Ok(())
>> + }
>> +
>> + pub(super) fn purge(&mut self, path: &Path) -> Result<(), String> {
>
> I guess we have to call purge on the parent, rather than on the cgroup
> itself, because of the unlink? Maybe we could call it purge_child? It
> confused me for a while.
Correct. Will rename in v5.
The way to understand this code is that Cgroup has two stacks: one
for file descriptors and one for path components. All operations
operate at a specified depth from the top of the stack. 1 refers to
the top of the stack, 2 to one level below that, and so on.
This function is really confusing because it performs multiple pushes
and pops on the internal file descriptor stack. The specific algorithm is:
1. Start with an exclusive lock.
2. Try to delete the child directly.
3. If deletion succeeds, or if it fails with ENOENT, return success.
4. If deletion fails with anything other than EBUSY, return an error.
5. Open the child cgroup and take an exclusive lock on it. This pushes
the child cgroup's FD onto the stack. The open_subtree() method
also pushes the child path onto the stack.
6. Take a *shared* lock on the FD that is directly below the top
of the stack. This is the file descriptor that was initially
on the top of the stack.
This releases the exclusive lock, allowing other operations on
different children to proceed. Different operations on the cgroup
being purged will be blocked by the exclusive lock taken in step 5.
7. Kill all programs in the cgroup by writing 1 to cgroup.kill.
8. Open cgroup.events.
9. Read from the FD opened in step 8. If the file contains the line
"populated 0", go to step 11.
10. Call poll() on the FD opened in step 8 to wait for POLLERR or
POLLPRI to happen. Then go back to step 9.
This is race-free because the kernel will set the "this is ready"
flag after every change that affects what would be read from
the file.
11. Pop the file descriptor to the being-purged cgroup from the stack.
12. Use the just-popped file descriptor to remove all subdirectories
recursively. Files must not be deleted, as the kernel doesn't
allow it. Then close the file descriptor, releasing the exclusive
lock held on it.
13. Take an exclusive lock on the *parent* of the cgroup that was just purged.
This must be done after the file descriptor to the cgroup being
purged has been closed. Otherwise, there is the potential for
an ABBA deadlock: another program might hold a shared lock on
the parent, and be waiting to get an exclusive lock on the child.
14. Delete the being-purged cgroup. Treat EBUSY and ENOENT as success:
the first means that a concurrently-running program re-created the
cgroup, while the second means that a concurrently-running program
deleted it. The name of the cgroup being purged is currently at
the top of the path stack.
15. Pop the name of the cgroup being purged off of the stack.
At the end, self is in the same state it was before the operation.
If you are complaining that this is about as readable as Forth,
then I agree with you :).
>> + assert!(self.exclusive());>> + match rustix::fs::unlinkat(self.as_fd(), Path::new(path), AtFlags::REMOVEDIR) {
>> + // Trying to purge a deleted cgroup is not an error.
>> + Ok(()) | Err(Errno::NOENT) => return Ok(()),
>> + Err(Errno::BUSY) => {}
>
> This could use a comment.
Will add in v5.
>> + Err(e) => return Err(format!("Cannot purge {:?}: {e}", self.joined_path(path))),
>> + }
>> + if !self.open_sub_cgroup(path, true, true)? {
>> + return Ok(());
>> + }
>> +
>> + rustix::fs::flock(
>> + self.fd[self.fd.len() - 2].0.as_fd(),
>> + FlockOperation::LockShared,
>
> We already must have at least a shared lock on this at this point, no?
> I don't think we need another one.
We actually have an exclusive lock. If it succeeds,
Cgroup::open_sub_cgroup() pushes a file descriptor onto self.fd.
Therefore, the fd being locked here is the one that was initially on the
top of the stack. We assert that an exclusive lock is held on that FD.
Waiting for the control group to become empty is a blocking operation,
so this downgrades the lock to a shared one. Otherwise, an in-progress
purge of /a/b would prevent /a/c from being created.
>> + )
>> + .map_err(|e| format!("Cannot relock {:?}: {e}", self.path.parent()))?;
>> + self.write_cgroup_value("cgroup.kill", "1")?;
>> + self.wait_for_empty()
>> + .map_err(|e| format!("Cannot wait for cgroup to become empty: {e}"))?;
>> + let fd = self.fd.pop().unwrap().0;
>> + let v = (|| {
>> + remove_recursively(fd, 1000)
>> + .map_err(|e| format!("Cannot remove {:?}: {e}", self.path))?;
>> + rustix::fs::flock(self.as_fd(), FlockOperation::LockExclusive)
>> + .map_err(|e| format!("Cannot lock {:?}: {e}", self.path))?;
>
> We already checked self.exclusive above, meaning we already have this
> lock on self?
self.fd.pop() removes the FD from the stack, and remove_recursively closes it.
self.fd() returns the FD whose lock was downgraded to a shared one above.
>> + match rustix::fs::unlinkat(
>> + self.as_fd(),
>> + Path::new(self.path.file_name().unwrap()),
>
> I am too confused about what self.path is to know what to make of this.
> self.as_fd() should be the fd of the cgroup directory, and self.path
> sounds like it should be the path to this cgroup, so how can this cgroup
> directory have self.path.file_name() within it? This needs clearer
> names or a refactor or something.
self.open_sub_cgroup() pushes both the path and the FD, but
self.fd.pop() only pops the FD. This means that self.fd() is currently
the path to the *parent* of the cgroup.
>> + AtFlags::REMOVEDIR,
>> + ) {
>> + // something might have re-created the cgroup in the meantime, which is okay
>> + Ok(()) | Err(Errno::BUSY) => Ok(()),
>> + Err(e) => Err(format!("Cannot lock {:?}: {e}", self.path)),
>> + }
>> + })();
>> + assert!(self.path.pop());
>
> I don't get this. Where was it pushed? (But would prefer to just not
> attempt to track this, as mentioned above.)
It was pushed in self.open_sub_cgroup().
>> + v
>> + }
>> +
>> + pub(crate) fn write_cgroup_value(&self, name: &str, value: &str) -> Result<(), String> {
>> + let path = Path::new(name);
>> + let fd = rustix::fs::openat2(
>> + self.as_fd(),
>> + path,
>> + OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::WRONLY,
>> + Mode::empty(),
>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>> + )
>> + .map_err(|e| format!("Cannot open {:?}: {}", self.joined_path(Path::new(name)), e))?;
>
> Could also use ResolveFlags::MAGIC_LINKS and be unified with the other
> openat2 invocations maybe? I don't understand why this is NOATIME but
> others aren't.
Yes, it indeed should be.
>> + File::from(fd).write_all(value.as_bytes()).map_err(|e| {
>> + format!(
>> + "Cannot write {:?} to {:?}: {}",
>> + value,
>> + self.joined_path(Path::new(name)),
>> + e
>> + )
>> + })
>> + }
>> +}
>> +
>> +fn open_subtree_raw(path: &Path, dirfd: BorrowedFd<'_>) -> Result<OwnedFd, Errno> {
>> + rustix::fs::openat2(
>> + dirfd,
>> + path,
>> + OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY,
>> + Mode::empty(),
>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>> + )
>> +}
>> +
>> +fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> {
>> + if remaining_depth < 1 {
>> + panic!("control groups too deeply nested");
>> + }
>> + let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating");
>> + while let Some(element) = d.next() {
>> + let element = element.expect("Iterating through a cgroup directory failed?");
>> + if element.file_type() != rustix::fs::FileType::Directory {
>> + continue;
>> + }
>> +
>> + let remaining_depth = remaining_depth - 1;
>> + let d: &rustix::fs::Dir = &d;
>> + let dirfd = d.fd().unwrap();
>> + let path = element.file_name();
>> + remove_all(remaining_depth, dirfd, path)?;
>> + }
>> + drop(d);
>> + Ok(())
>> +}
>> +
>> +fn remove_all(
>> + remaining_depth: usize,
>> + dirfd: BorrowedFd<'_>,
>> + path: &std::ffi::CStr,
>> +) -> Result<(), Errno> {
>> + if path == c"." || path == c".." {
>> + return Ok(());
>> + }
>> + if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() {
>> + return Ok(());
>> + }
>> + let fd = rustix::fs::openat2(
>> + dirfd,
>> + path,
>> + OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY,
>> + Mode::empty(),
>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>> + )?;
>> + remove_recursively(fd, remaining_depth)?;
>> + rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?;
>> + Ok(())
>> +}
>
> Could we save a lot of code by calling std::fs::remove_dir_all with a
> /proc/self/fd path? It's already documented to ignore symlinks.
I tried, but that tries to delete files too, and that isn't allowed.
>> diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..2e7a4e25213a4aa2449b292f8005e1da23cbb1f9
>> --- /dev/null
>> +++ b/tools/cgroup-setup/src/main.rs
>> @@ -0,0 +1,347 @@
>> +// SPDX-License-Identifier: EUPL-1.2+
>> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>> +
>> +use std::{
>> + ffi::{OsStr, OsString},
>> + os::unix::prelude::*,
>> + path::{Path, PathBuf},
>> +};
>> +
>> +use crate::cgroup::Cgroup;
>> +
>> +mod cgroup;
>> +
>> +fn check_path(path: &OsStr) -> Result<(), String> {
>> + if path.is_empty() {
>> + return Ok(());
>> + }
>> +
>> + for component in path.as_bytes().split(|&b| b == b'/') {
>
> Would be a lot nicer to take &Path and use Path::components.
Will change in v5. I'm way more used to doing this in C.
>> + match component {
>> + b"" | b"." | b".." => {
>> + return Err(format!("Path {path:?} has empty, ., or .. component"));
>> + }
>> + // Cannot happen: command line arguments have no NUL byte,
>> + // and /proc/self/cgroup having a NUL byte is a kernel bug.
>> + _ if component.contains(&b'\0') => panic!("Path {path:?} has NUL byte"),
>
> It is in fact an invariant of both the Path and OsStr types (on Unix)
> that there are no NUL bytes, so there is no need to check for it at all.
It actually isn't. OsStr::from_bytes("\0") doesn't panic.
>> + _ if component.len() > 255 => {
>> + return Err(format!(
>> + "Path {path:?} has component {:?} that is longer than 255 bytes",
>> + OsStr::from_bytes(component)
>> + ));
>> + }
>
> Why on earth would we need to check for this? This limit is totally up
> to the kernel. Our code won't break with an excessively long path
> component.
It provides better error messages than ENAMETOOLONG, but I will remove it in v5.
>> + _ => {}
>> + }
>> + }
>> +
>> + Ok(())
>> +}
>> +
>> +/// Get the path of the cgroup for the provided command-line argument.
>> +/// Returns an empty path if the path is "/", or if it is "." and the
>> +/// current cgroup is "/".
>
> Please standardize terminology between "current cgroup" and "local
> cgroup", or if those are not the same thing use clearer phrasing.
>
> I hope all the different modes of this function are actually needed…
> (I assume they are but haven't reviewed the patches that make use of it
> yet.)
All of them are indeed used.
>> +///
>> +/// # Errors
>> +///
>> +/// Fails if the provided path is invalid or empty, or if it is relative
>> +/// and the local cgroup cannot be determined.
>> +fn get_cgroup(cgroup_path: OsString) -> Result<PathBuf, String> {
>
> Please call this function something clearer. Also using OsString where
> PathBuf would be more appropriate again…
Fair! Will fix in v5.
>> + if cgroup_path.as_bytes().starts_with(b"/") {
>
> … which would allow using Path::is_absolute if fixed.
Will fix in v5.
>> + let mut cgroup_path = cgroup_path.into_vec();
>> + cgroup_path.remove(0);
>
> Please use Path functions rather than byte manipulation like this.
Will change in v5.
>> + if cgroup_path.is_empty() {
>> + return Err("cgroup path cannot be /".to_owned());
>> + }
>> + let cgroup_path = OsString::from_vec(cgroup_path);
>> + check_path(&cgroup_path)?;
>> + Ok(cgroup_path.into())
>> + } else if cgroup_path.is_empty() {
>> + Err("cgroup path cannot be empty".to_owned())
>> + } else {
>> + check_path(&cgroup_path)?;
>> + let mut local_cgroup = local_cgroup()?;
>
> Can you reorder all these function definitions to be something more
> sensible? get_cgroup is here, near the start of main.rs, but
> local_cgroup, which it is the only direct caller of, is all the way at
> the other end. (Personally I like to define utility functions directly
> above their only caller — I think that creates the most natural reading
> flow.)
Will fix in v5.
>> + local_cgroup.push(cgroup_path);
>> + Ok(local_cgroup)
>> + }
>> +}
>> +
>> +/// Open the cgroup corresponding to the provided path.
>> +/// It must have already been made relative to `/sys/fs/cgroup`.
>> +///
>> +/// # Errors
>> +///
>> +/// Fails if the cgroup operation fails.
>
> This "Errors" section is just stating the obvious.
Will remove in v5.
>> +fn open_cgroup(path: &Path, exclusive: bool) -> Result<Cgroup, String> {
>> + if path.as_os_str().is_empty() {
>> + Cgroup::new(exclusive)
>> + } else {
>> + let mut cgroup = Cgroup::new(false)?;
>> + cgroup.open_sub_cgroup(path, exclusive, false)?;
>> + Ok(cgroup)
>> + }
>> +}
>
> I feel like it would be more natural if Cgroup::new just took two
> parameters and worked this way.
Will change in v5.
>> +
>> +/// Open the cgroup corresponding to the provided path's parent.
>> +/// It is made relative to the process's own cgroup if needed.
>> +///
>> +/// # Errors
>> +///
>> +/// Fails if the cgroup operation fails.
>> +fn open_relative_cgroup(arg: OsString) -> Result<(PathBuf, Cgroup), String> {
>> + let path = get_cgroup(arg)?;
>> + let cgroup = open_cgroup(path.parent().expect("always has a parent"), true)?;
>> + Ok((path, cgroup))
>> +}
>
> This is a deeply confusing function. Why is opening a cgroup for the
> path's _parent_ an operation that we should have a dedicated function
> for? Why not have the caller pass in the actual path of the cgroup it
> wants to open? This really looks like a function that's doing as many
> different things at it has lines, and should just be inlined.
Will inline in v5.
>> +
>> +fn main() {
>> + let mut args = std::env::args_os();
>> + let Some(prog_name) = args.next() else {
>> + eprintln!("No command line arguments (argv[0] is NULL)");
>> + std::process::exit(1);
>> + };
>> + match main_(&prog_name, args) {
>> + Ok(()) => {}
>> + Err(e) => {
>> + eprintln!("{prog_name:?}: {}", e);
>> + std::process::exit(1);
>> + }
>> + }
>> +}
>> +
>> +fn main_(prog_name: &OsStr, mut args: std::env::ArgsOs) -> Result<(), String> {
>
> main_ is a bit confusingly similar to name. I like "run" for this sort
> of function, and we use that elsewhere.
Will fix in v5.
>> + match prog_name
>> + .as_bytes()
>> + .split(|&b| b == b'/')
>> + .next_back()
>> + .unwrap()
>
> Another place that should use Path.
Will fix in v5.
>> + {
>> + b"cgroup-s6-finish" => {
>> + return s6_finish(&mut args);
>> + }
>> + b"cgroup-setup" => {}
>
> It is inconsistent for this not to also be its own function. Then the
> match could just be an expression that returned the result of the
> appropriate function.
Will fix in v5.
>> + b"cgroup-purge" => {
>> + if args.len() != 1 {
>> + return Err(format!(
>> + "cgroup-purge takes one argument, got {}",
>> + args.len()
>> + ));
>> + }
>> + let cgroup_path = args.next().unwrap();
>> + let (path, mut cgroup) = open_relative_cgroup(cgroup_path)?;
>> + let cgroup_target = Path::new(path.file_name().unwrap());
>> + return cgroup.purge(cgroup_target);
>> + }
>> + _ => {
>> + return Err(format!(
>> + "must be invoked as \"cgroup-setup\" \
>> + \"cgroup-purge\", or \"cgroup-s6-finish\", \
>> + got {prog_name:?}",
>> + ));
>> + }
>> + };
>> + let mut leaf = false;
>> + let mut cgroup_path;
>> + let mut delegate = false;
>> + let mut init_subtree = false;
>> + let mut child_name: Option<&'static OsStr> = None;
>> + let mut wait = true;
>> + loop {
>> + cgroup_path = args.next();
>> + let Some(ref arg_) = cgroup_path else {
>> + break;
>> + };
>> + let arg_ = arg_.as_bytes();
>> + if arg_ == b"--" {
>> + cgroup_path = args.next();
>> + break;
>> + }
>> + if !arg_.starts_with(b"-") {
>> + break;
>> + }
>> +
>> + if !arg_.starts_with(b"--") {
>> + return Err("takes no short options".to_owned());
>> + }
>> +
>> + match &arg_[2..] {
>> + b"leaf" => leaf = true,
>> + b"delegate" => delegate = true,
>> + b"init-subtree" => init_subtree = true,
>> + b"wait" => wait = true,
>> + b"no-wait" => wait = false,
>> + b"child-name" if child_name.is_none() => match args.next() {
>> + Some(arg) => child_name = Some(arg.leak()),
>> + None => return Err("--child-name: missing argument".to_owned()),
>> + },
>> + b"child-name" => return Err("--child-name: cannot be used twice".to_owned()),
>> + arg => match str::from_utf8(arg) {
>> + Ok(e) => return Err(format!("unknown long option {e:?}")),
>> + Err(_) => return Err("long option isn't UTF-8".to_owned()),
>> + },
>> + }
>> + }
>> +
>> + let default_child_name = OsStr::from_bytes(b"$inner.service");
>> +
>> + let child_name = Path::new(child_name.unwrap_or(default_child_name));
>> +
>> + let Some(mut cgroup_path) = cgroup_path else {
>> + return Err("have no positional arguments, expected at least 1".to_owned());
>> + };
>> +
>> + // Allow --init-subtree .
>> + if cgroup_path.as_bytes() == b"." && init_subtree && !leaf {
>> + cgroup_path = child_name.to_owned().into();
>> + leaf = true;
>> + }
>> +
>> + let (path, mut cgroup) = open_relative_cgroup(cgroup_path)?;
>> + let cgroup_target = Path::new(path.file_name().unwrap());
>> + cgroup
>> + .make_child(cgroup_target)
>> + .map_err(|e| format!("Cannot make child cgroup: {e}"))?;
>> + if wait {
>> + cgroup
>> + .wait_for_empty()
>> + .map_err(|e| format!("Cannot wait for {path:?} to be empty: {e}"))?;
>> + }
>> + let pid = std::process::id().to_string();
>> + if leaf {
>> + if args.len() != 0 {
>> + // If we aren't delegating any cgroups, don't create a sub-cgroup.
>> + cgroup
>> + .write_cgroup_value("cgroup.procs", &pid)
>> + .map_err(|e| format!("Cannot write to {path:?}/cgroup.procs: {e}"))?;
>> + }
>> + } else {
>> + // If the child process will need to manage cgroups itself, it will need
>> + // to set up a sub-cgroup due to the "no internal processes" rule. It's
>> + // simplest to just do it automatically.
>> + cgroup.make_child(Path::new(child_name)).map_err(|e| {
>> + format!(
>> + "Cannot create child cgroup {}/{}: {e}",
>> + path.display(),
>> + child_name.display()
>> + )
>> + })?;
>> + if args.len() != 0 {
>> + cgroup
>> + .write_cgroup_value("cgroup.procs", &pid)
>> + .map_err(|e| {
>> + format!(
>> + "Cannot write to {}/{}/cgroup.procs: {e}",
>> + path.display(),
>> + child_name.display()
>> + )
>> + })?;
>> + }
>> + }
>> + if !leaf {
>> + cgroup.enable_subtree_control(2)?;
>> + }
>> + if init_subtree {
>> + cgroup.enable_subtree_control(1)?;
>> + }
>> + if delegate {
>> + cgroup
>> + .enable_delegation(1)
>> + .map_err(|e| format!("Cannot enable cgroup delegation in {path:?}: {e}"))?;
>> + }
>> + let Some(program_name) = args.next() else {
>> + return Ok(());
>> + };
>> + let e = std::process::Command::new(&program_name).args(args).exec();
>> + Err(format!("Cannot spawn child {:?}: {}", program_name, e))
>> +}
>> +
>> +fn s6_finish(args: &mut std::env::ArgsOs) -> Result<(), String> {
>> + if args.len() < 3 {
>> + return Err(format!(
>> + "s6 finish scripts take at least 3 arguments, got {}",
>> + args.len()
>> + ));
>> + }
>> + let status = parse_digit_string(&args.next().unwrap(), "exit status")?;
>> + let signal = args.next().unwrap();
>> + let signal = if status == 256 {
>> + Some(parse_digit_string(&signal, "signal number")?)
>> + } else {
>> + None
>> + };
>> + let service = args.next().unwrap();
>> +
>> + let (path, mut cgroup) = open_relative_cgroup(service)?;
>> + let cgroup_target = Path::new(path.file_name().unwrap());
>> + let exit_125 = if let Some(signal) = signal {
>> + match signal as libc::c_int {
>> + libc::SIGBUS
>> + | libc::SIGFPE
>> + | libc::SIGABRT
>> + | libc::SIGTRAP
>> + | libc::SIGSEGV
>> + | libc::SIGILL => {
>> + // Process *crashed*, indicating a *possible exploit attempt*.
>> + // s6 should *not* restart it. This is distinct from a Rust panic,
>> + // which is much less likely to indicate memory corruption.
>> + true
>> + }
>
> This has absolutely nothing to do with cgroups. If you want to have
> some common finish behaviour, a program called cgroup-setup is not the
> place for it. I don't think there's any need for a separate
> cgroup-s6-finish mode (as opposed to cgroup-purge).
This program is a multi-call binary, so the various things it can do
aren't necessarily super tightly related. For instance, all of the
execline binaries can be built as one program, as can most if not
all busybox applets. When invoked as cgroup-setup or cgroup-purge,
it indeed only does cgroup-related tasks. cgroup-s6-finish not
only handles cgroups, but also other tasks related to being an s6
finish script.
That said, using this changes behavior in a way that isn't related
to cgroups, so if it is to be used at all it should be in a separate
patch series. I'll remove this from v5.
>> + _ => false,
>> + }
>> + } else {
>> + false
>> + };
>> + if exit_125 {
>> + // Ignore panics. Exit status is more important.
>> + // We already had a core dump.
>> + let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
>> + match cgroup.purge(cgroup_target) {
>> + Ok(()) => {}
>> + Err(e) => {
>> + eprintln!("cgroup-s6-finish: Failed to purge cgroup: {e}")
>> + }
>> + };
>> + }));
>> + std::process::exit(125)
>> + } else {
>> + cgroup.purge(cgroup_target)
>> + }
>> +}
>> +
>> +fn parse_digit_string(digits: &OsStr, msg: &str) -> Result<u16, String> {
>> + let checked = match str::from_utf8(digits.as_bytes()) {
>> + Ok(s) => s,
>> + Err(e) => return Err(format!("{msg} is not UTF-8: {e}")),
>> + };
>> + let r = checked
>> + .parse::<u16>()
>> + .map_err(|e| format!("{msg} {digits:?} is a bad 16-bit number: {e}"))?;
>> + match checked.as_bytes() {
>> + b"0" | [b'1'..=b'9', ..] => Ok(r),
>> + [b'0', ..] => Err(format!("{msg} {} has a leading 0", digits.display())),
>> + _ => Err(format!("{msg} {} starts with +", digits.display())),
>> + }
>
> Surely we trust s6 to turn a number into a string. These checks add nothing.
Yes, we can. This is part of cgroup-s6-finish, which will be removed in v5.
>> +}
>> +
>> +fn local_cgroup() -> Result<PathBuf, String> {
>> + let mut local_cgroup: Vec<u8> = std::fs::read("/proc/thread-self/cgroup")
>> + .map_err(|e| format!("cannot read /proc/thread-self/cgroup: {e}"))?;
>> + let local_cgroup_len = local_cgroup.len();
>> + if local_cgroup_len < 5
>> + || local_cgroup[..4] != *b"0::/"
>> + || local_cgroup[local_cgroup_len - 1] != b'\n'
>> + || local_cgroup[4..local_cgroup_len - 1].contains(&b'\n')
>
> Last time I suggested a clearer way of doing this, but it has instead
> got even less clear.
>
> (I'm not sure why we'd care if there's a newline specifically, as
> opposed to any other control character.)
If cgroups v1 is in use, the file can contain multiple lines, one for
each cgroup the program is in. I also am not sure if starting with
"0::/" is an invariant in that case.
Using this program with cgroups v1 mounted is user error and will
never happen on Spectrum, but if this tool is used outside of Spectrum,
it could happen.
>> + {
>> + // It's possible to get here if the cgroup path contains a newline,
>> + // but that never happens in Spectrum.
>> + return Err(format!(
>> + "Invalid contents {local_cgroup:?} of /proc/thread-self/cgroup - \
>> + do you have cgroups v1 mounted instead of cgroups v2?"
>> + ));
>> + }
>> +
>> + local_cgroup.copy_within(4..local_cgroup_len - 1, 0);
>> + local_cgroup.truncate(local_cgroup_len - 5);
>> + let local_cgroup = OsString::from_vec(local_cgroup);
>> + check_path(&local_cgroup).unwrap();
>
> Why do we need to do this? You're worried the kernel is going to start
> including .. components in /proc/thread-self/cgroup?
Originally, I was going to create a wrapper around `Path` that
guaranteed no `.` or `..` components were present. Its constructor
would have checked this invariant. However, this turned out to be
more work due to the amount of wrapper functions required.
>> + Ok(PathBuf::from(local_cgroup))
>> +}
>>
>> --
>> 2.55.0
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 03/20] Documentation: Mention control groups
2026-07-22 1:59 ` [PATCH v4 03/20] Documentation: Mention control groups Demi Marie Obenour
@ 2026-07-27 11:22 ` Alyssa Ross
2026-07-28 10:41 ` Valentin Gagarin
1 sibling, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-07-27 11:22 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development, Valentin Gagarin
[-- Attachment #1: Type: text/plain, Size: 5379 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> Documentation/doc/development/control-groups.adoc | 88 +++++++++++++++++++++++
> 1 file changed, 88 insertions(+)
CCing Valentin for documentation oversight. I found this very clearly
written and easy to read. :)
> diff --git a/Documentation/doc/development/control-groups.adoc b/Documentation/doc/development/control-groups.adoc
> new file mode 100644
> index 0000000000000000000000000000000000000000..6ce33f21a230d012a690fc5deb2ba597d1d01ef1
> --- /dev/null
> +++ b/Documentation/doc/development/control-groups.adoc
> @@ -0,0 +1,88 @@
> += Control groups in Spectrum
> +
> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0
> +
> +Linux control groups (cgroups) can be used for several purposes:
> +
> +1. They allow waiting for a group of processes to exit.
> +2. They allow terminating a group of processes.
> +3. They allow limiting a group of processes' access to resources.
> +
> +Spectrum currently uses the first two. The third is not yet used,
> +but will be in the future.
> +
> +== Control Group Hierarchy
> +
> +Spectrum uses the following cgroup hierarchy:
> +
> +1. There is a `/vm-services.slice` cgroup that contains all the per-VM
> + services on the system.
> +2. The per-VM services for each VM are under `/vm-services.slice/vm-${VM}.slice`,
> + where `${VM}` is replaced by the VM's ID.
> +3. Each per-VM service is under `/vm-services.slice/vm-${VM}.slice/${SERVICE_NAME}`,
> + where `${VM}` is replaced by the VM's ID and `${SERVICE_NAME}` is replaced by
> + the name of the service.
> +4. The VMM runs under `/vm-services.slice/vm-${VM}.slice/vmm`.
> +
> +If a cgroup contains child cgroups, it likely contains a `$inner.service`
> +cgroup. This is where programs that would otherwise run in the cgroup itself
> +are placed. Generally, these programs are instances of `s6-svscan` and/or
> +`s6-supervise`.
This section doesn't mention where non-per-VM services go. It might
also be nice to explicitly mention the "no internal processes" rule as
the reason for $inner.service, in case people are unfamiliar with it.
> +== Using Control Groups
> +
> +When adding a new s6 service, one should carefully consider whether it
> +should be placed in a control group. Most services should be placed in
> +a control group, with only a few exceptions:
> +
> +1. Services, such as `getty`, that spawn background processes.
> +2. Loggers.
> +3. Trivial services that don't do anything.
We shouldn't have any services that don't do anything! Maybe we could
be more specific? Or just say "trivial services where cgroup are
unnecessary" or something.
> +
> +Generally, it's best to set the control group up as the first thing
> +the service does. To do that, use `cgroup-setup --leaf -- $1 COMMAND_LINE`,
The " -- " here gets turned into an en-dash when rendered, so must need
to be escaped somehow.
> +where `$1` should be the service name and `COMMAND_LINE` is the program
> +to run in a cgroup.
> +
> +If you use execline for your run script, this is as simple as:
> +
> +.run
> +....
> +#!/bin/execlineb -WS1
> +
> +cgroup-setup --leaf -- $1
> +# rest of script comes here
> +....
> +
> +If the service exits, it's usually best to terminate any programs left
> +behind with SIGKILL and remove the control group. In Spectrum, this is
> +called "purging" the cgroup. To purge the cgroup when a service exits,
> +make the `finish` script invoke `/usr/bin/cgroup-s6-finish`. The first
> +two command line arguments must be the first two arguments passed to the
> +`finish` script. The third argument must be the path to the cgroup to
> +be purged relative to the cgroup the program itself is in. This is
> +usually, but not always, the third argument to the `finish` script.
> +
> +When invoked as `cgroup-s6-finish`, `cgroup-setup` checks if
> +the service exited due to a signal that caused it to dump core. If it
> +did, `cgroup-s6-finish` exits with status 125, ensuring that
> +`s6-supervise` will *not* restart it. This is intentional: if a service
> +crashes due to a fatal signal, this is possibly a sign of memory
> +corruption. Restarting the service in this case can turn an unreliable
> +memory corruption exploit into a reliable one. Rust panics do not cause
> +core dumps, so the service will be restarted afterwards.
(Just noting that in the review of cgroup-setup itself, I said that to
me this does not seem likely something that should be part of
cgroup-setup itself. If that does change, this will need to be updated.)
> +One can also use `cgroup-purge` to purge a cgroup explicitly. This is
> +used to stop the VMM and all per-VM services when a VM is shut down.
> +
> +== Future plans
> +
> +Control groups are designed around a single writer process controlling each
> +of them. Many Linux distros use systemd for this, but Spectrum doesn't use
> +systemd. The only persistent per-service process is s6-supervise, but that
> +doesn't have control group support.
> +
> +Instead, the plan is to have a database containing this information.
> +Whether this will be in the `data/` subdirectory of the service directory
> +or a separate system-wide database has not yet been determined.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 04/20] Mount cgroup2 filesystem at /sys/fs/cgroup
2026-07-22 1:59 ` [PATCH v4 04/20] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
@ 2026-07-27 11:23 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-07-27 11:23 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 843 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Needed for cgroups to work at all.
>
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> host/rootfs/image/etc/fstab | 1 +
> 1 file changed, 1 insertion(+)
Reviewed-by: Alyssa Ross <hi@alyssa.is>
> diff --git a/host/rootfs/image/etc/fstab b/host/rootfs/image/etc/fstab
> index 4d52a1da8c6d673cbbcfe2e9c0ec93085f06c40f..cab97cf9c49f864845492a1e524d43b7d546317f 100644
> --- a/host/rootfs/image/etc/fstab
> +++ b/host/rootfs/image/etc/fstab
> @@ -6,3 +6,4 @@ tmpfs /dev/shm tmpfs nosuid,nodev 0 0
> tmpfs /media tmpfs nosuid,nodev,noexec,nosymfollow,mode=755 0 0
> sysfs /sys sysfs nosuid,nodev,noexec 0 0
> tmpfs /tmp tmpfs nosuid,nodev 0 0
> +cgroup2 /sys/fs/cgroup cgroup2 nosuid,nodev,noexec,nosymfollow 0 0
>
> --
> 2.55.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 05/20] host/rootfs: Add helper program for per-VM services
2026-07-22 1:59 ` [PATCH v4 05/20] host/rootfs: Add helper program for per-VM services Demi Marie Obenour
@ 2026-07-27 11:27 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-07-27 11:27 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 2871 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> The program handles cgroups and common substitutions. This is better
> than repeating the boilerplate multiple times.
>
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> host/rootfs/file-list.mk | 1 +
> host/rootfs/image/usr/bin/vm-service-run | 34 ++++++++++++++++++++++++++++++++
> 2 files changed, 35 insertions(+)
>
> diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
> index e1280ab56d8797e40b9b1c584ab0daef3cda41d7..c707c01179238231f70164ffda5b7c940b34192a 100644
> --- a/host/rootfs/file-list.mk
> +++ b/host/rootfs/file-list.mk
> @@ -65,6 +65,7 @@ FILES = \
> image/usr/bin/spectrum-update \
> image/usr/bin/vm-console \
> image/usr/bin/vm-import \
> + image/usr/bin/vm-service-run \
> image/usr/bin/vm-start \
> image/usr/bin/vm-stop \
> image/usr/bin/xdg-open \
> diff --git a/host/rootfs/image/usr/bin/vm-service-run b/host/rootfs/image/usr/bin/vm-service-run
> new file mode 100755
> index 0000000000000000000000000000000000000000..c788574cf95ec7fd0d93b656557f85fa80b31690
> --- /dev/null
> +++ b/host/rootfs/image/usr/bin/vm-service-run
> @@ -0,0 +1,34 @@
> +#!/usr/bin/execlineb -WS1
> +# SPDX-License-Identifier: EUPL-1.2+
> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +
> +# Substitute all of the needed environment variables
> +# into both this script and its arguments
> +# (the subsequent command to run). This substitutes
> +# the *caller*'s arguments.
> +multisubstitute {
> + importas -iS VM
> + importas -iS "#"
> + importas -iS "1"
> + importas -iS WAYLAND_DISPLAY
> +}
> +
> +# Check that the VM name and service are reasonable.
> +# Then run the cgroup-setup program and the provided
> +# command line. Avoid premature substitution by
> +# escaping ${#} and ${1} with backslashes. Otherwise,
> +# they would be values for this script, not for the caller.
> +case -- "\\${#}@${VM}@\\${1}" {
> + # s6-supervise might start passing extra arguments in the future, but
> + # 0 arguments is not okay.
> + "[1-9][0-9]*@[A-Za-z0-9_][A-Za-z0-9_.-]*@[A-Za-z_][A-Za-z0-9_-]*" {
> + # The caller has been migrated from using -WS1 to using -Wp.
> + # Pop the environment to preserve the original behavior.
> + emptyenv -P cgroup-setup --leaf -- "\\${1}" $@
> + }
> +}
> +fdmove -c 1 2
> +if { printf "Wrong VM name %s, \
> +parent argument count %s, or parent argument %s\n"
> + $VM "\\$#" "\\${1}" }
> +exit 100
Not sure I get this. This validation looks quite complicated and
prescriptive, and why are we substituting e.g. WAYLAND_DISPLAY in here
when not every service is going to need it? I fear that this is the
sort of deduplication and validation that makes things harder to
understand and maintain…
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 02/20] tools: Add control group manager
2026-07-23 23:07 ` Demi Marie Obenour
@ 2026-07-27 12:10 ` Alyssa Ross
2026-07-30 0:40 ` Demi Marie Obenour
0 siblings, 1 reply; 132+ messages in thread
From: Alyssa Ross @ 2026-07-27 12:10 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 21022 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> On 7/22/26 12:01, Alyssa Ross wrote:
>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>> +#[derive(Debug)]
>>> +pub(crate) struct Cgroup {
>>> + path: PathBuf,
>>> + fd: Vec<(OwnedFd, bool)>,
>>
>> There's no point storing all these exclusivity bools, is there? I think
>> only the last one is ever checked, so we could make things tighter and
>> clearer like this, where we only track the exclusivity of the last fd:
>
> Cgroup::enable_subtree_control() checks the exclusivity
> of the caller-provided depth. Line 232 of main.rs calls
> enable_subtree_control(2).
>
> These bools are only used in assertions, so they could be removed.
> I will leave that up to you. The advantage of keeping them is that a
> panic is vastly easier to debug than a race condition due to improper
> locking.
Can you explain to me why we need to support enabling subtree control
for different depths? Intuitively, I'd expect this program to only ever
operate on the cgroup associated with the service that's invoking it.
>> fd: Vec<OwnedFd>,
>> exclusive: bool,
>>
>>> +}
>>> +fn assert_single_component(component: &[u8]) {
>>
>> Why not &Path, which is already guaranteed not to have a NUL byte?
>> Perhaps this whole thing could be simplified to
>> Some(component).as_os_str() == component.file_name()? Maybe that's too
>> clever, though…
>
> Path isn't actually guaranteed to not have a NUL byte.
> File::open(Path::new("\0")) fails with InvalidInput rather than
> panicking.
Oh, my mistake. You are right.
> I'm very used to writing this kind of code in C, so I went with a C-like
> style instead of using Rust stdlib APIs. I don't like having extra
> abstractions in this kind of code, as it obscures what is going on
> under the hood. That is less important here, but it's very important
> in programs like mount-flatpak.
I did say myself that my suggestion here might be too clever, because it
uses the standard library in a way that's unintuitive. In general,
though, we can expect readers of Rust code to be more familiar with the
standard library than with byte-by-byte stringy patch processing. Would
still be nicer to take &Path here, I think, even if you then walk
through it byte by byte, just to make it slightly clearer what this does.
>>> + pub fn read_control_file(&self, fd: BorrowedFd, p: &Path) -> Result<Vec<u8>, String> {
>>> + let mut buf = Vec::new();
>>> + let err = |e: &dyn Display, p: &Path, msg: &str| {
>>> + let path = self.path.join(p);
>>> + format!("Cannot {msg} {path:?}: {e}")
>>> + };
>>> + File::from(open_subtree_raw(Path::new(p), fd.as_fd()).map_err(|e| err(&e, p, "open"))?)
>>
>> If we're using it for opening files, open_subtree_raw is probably misnamed.
>
> Yup! Do you have a suggestion for improving it? Maybe open_child()?
open_beneath?
>>> + pub fn open_sub_cgroup(
>>> + &mut self,
>>> + path: &std::path::Path,
>>> + exclusive: bool,
>>> + allow_missing: bool,
>>> + ) -> Result<bool, String> {
>>> + let mut iter = path.components().peekable();
>>> + while let Some(component) = iter.next() {
>>
>> Perhaps would be nicer:
>>
>> let mut components = path.components().peekable();
>> for component in components {
>
> That results in a borrowcheck error. The for loop takes ownership
> of the iterator, but .peek() is called inside the loop.
Ah, okay. This is fine then.
>>> + let component = match component {
>>> + Component::Normal(component) => component,
>>> + _ => unreachable!(),
>>> + };
>>> + let sub_fd = match self
>>> + .open_sub_cgroup_raw(OFlags::DIRECTORY | OFlags::RDONLY, component.as_bytes())
>>> + {
>>> + Ok(sub_fd) => {
>>> + self.path.push(component);
>>
>> I would really like to not try to store self.path. It seems very
>> complicated to track. It's also very unclear to me from the name (and
>> the code) what it is. Is it the path to the cgroup itself, or to its
>> parent? It looks to me like it should be the cgroup itself, but then
>> what's going on in purge?
>
> It's the path to the cgroup itself, relative to /sys/fs/cgroup. Its only
> purpose is for logging.
>
>> We could actually improve readability of this quite complicated function
>> even further if you find it acceptable to just use Errno for the error
>> type. In that case, we'd just return Result<(), Errno>, and callers
>> would check for Errno::NOENT if they wanted to allow missing. Then we
>> could just completely drop that argument. In my opinion it would be
>> worth it to move complexity out of here.
>
> I can do this, but it would result in much worse error messages: the
> error would only reference the file name, not the full cgroup path.
> Which would you prefer?
I would much prefer code I can easily understand. There's always strace
for getting the full paths when debugging.
>>> + pub fn wait_for_empty(&self) -> std::io::Result<()> {
>>> + assert!(self.exclusive());
>>> + let wait_file = self.open_subtree(std::path::Path::new("cgroup.events"))?;
>>> + let poll_fd = wait_file.as_raw_fd();
>>> + let mut wait_fd = File::from(wait_file);
>>> + let mut fds = libc::pollfd {
>>> + fd: poll_fd,
>>> + events: libc::POLLPRI | libc::POLLERR,
>>> + revents: 0,
>>> + };
>>> + let mut v = vec![];
>>> + loop {
>>> + v.clear();
>>> + wait_fd
>>> + .seek(std::io::SeekFrom::Start(0))
>>> + .expect("Seek on control group file should succeed");
>>> + wait_fd
>>> + .read_to_end(&mut v)
>>> + .expect("reading from control group should work");
>>> + if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
>>> + break;
>>> + }
>>> + // SAFETY: FFI call, valid arguments, fds contains 1 element
>>> + if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
>>> + panic!("poll failed");
>>> + }
>>> + }
>>
>> Are you 100% confident that this doesn't race? I don't understand why
>> poll would be triggered in this scenario:
>>
>> 1. "1" is written to cgroup.kill
>> 2. Every process in the cgroup exits and is reaped.
>> 3. cgroup.events is opened, with the cgroup already empty.
>>
>> Are you not relying on 2 happening after 3? Presumably if you open
>> cgroup.events for a cgroup that's already empty, you're not going to get
>> a poll event to tell you it's empty.
>
> In that case, cgroup.events will include a "populated 0"
> line, so poll will not be called.
You are correct. :)
Perhaps could be written more clearly with a while, something like this:
while !v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
// SAFETY: FFI call, valid arguments, fds contains 1 element
if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
panic!("poll failed");
}
v.clear();
wait_fd
.seek(std::io::SeekFrom::Start(0))
.expect("Seek on control group file should succeed");
wait_fd
.read_to_end(&mut v)
.expect("reading from control group should work");
}
But if you prefer the current way (which avoids a redundant initial
check of an empty Vec, although that should be cheap) that's also fine.
>>> + pub(crate) fn make_child(&mut self, path: &Path) -> Result<(), Errno> {
>>> + assert!(self.exclusive());
>>> + let component = path.as_os_str().as_bytes();
>>> + assert_single_component(component);
>>> + match rustix::fs::mkdirat(
>>> + self.as_fd(),
>>> + path,
>>> + Mode::RUSR
>>> + | Mode::WUSR
>>> + | Mode::XUSR
>>> + | Mode::RGRP
>>> + | Mode::XGRP
>>> + | Mode::ROTH
>>> + | Mode::XOTH,
>>> + ) {
>>> + Ok(()) | Err(Errno::EXIST) => {}
>>> + bad => return bad,
>>> + }
>>> + let p = self.open_sub_cgroup_raw(OFlags::RDONLY | OFlags::DIRECTORY, component)?;
>>> + // exclusive lock on parent acts as exclusive lock on child
>>> + self.fd.push((p, true));
>>> + self.path.push(path);
>>> + Ok(())
>>> + }
>>> +
>>> + pub(super) fn purge(&mut self, path: &Path) -> Result<(), String> {
>>
>> I guess we have to call purge on the parent, rather than on the cgroup
>> itself, because of the unlink? Maybe we could call it purge_child? It
>> confused me for a while.
>
> Correct. Will rename in v5.
>
> The way to understand this code is that Cgroup has two stacks: one
> for file descriptors and one for path components. All operations
> operate at a specified depth from the top of the stack. 1 refers to
> the top of the stack, 2 to one level below that, and so on.
1-indexing is a little unintuitive for Rust, no?
>
> This function is really confusing because it performs multiple pushes
> and pops on the internal file descriptor stack. The specific algorithm is:
>
> 1. Start with an exclusive lock.
>
> 2. Try to delete the child directly.
>
> 3. If deletion succeeds, or if it fails with ENOENT, return success.
>
> 4. If deletion fails with anything other than EBUSY, return an error.
>
> 5. Open the child cgroup and take an exclusive lock on it. This pushes
> the child cgroup's FD onto the stack. The open_subtree() method
> also pushes the child path onto the stack.
>
> 6. Take a *shared* lock on the FD that is directly below the top
> of the stack. This is the file descriptor that was initially
> on the top of the stack.
>
> This releases the exclusive lock, allowing other operations on
> different children to proceed. Different operations on the cgroup
> being purged will be blocked by the exclusive lock taken in step 5.
>
> 7. Kill all programs in the cgroup by writing 1 to cgroup.kill.
>
> 8. Open cgroup.events.
>
> 9. Read from the FD opened in step 8. If the file contains the line
> "populated 0", go to step 11.
>
> 10. Call poll() on the FD opened in step 8 to wait for POLLERR or
> POLLPRI to happen. Then go back to step 9.
>
> This is race-free because the kernel will set the "this is ready"
> flag after every change that affects what would be read from
> the file.
>
> 11. Pop the file descriptor to the being-purged cgroup from the stack.
>
> 12. Use the just-popped file descriptor to remove all subdirectories
> recursively. Files must not be deleted, as the kernel doesn't
> allow it. Then close the file descriptor, releasing the exclusive
> lock held on it.
>
> 13. Take an exclusive lock on the *parent* of the cgroup that was just purged.
>
> This must be done after the file descriptor to the cgroup being
> purged has been closed. Otherwise, there is the potential for
> an ABBA deadlock: another program might hold a shared lock on
> the parent, and be waiting to get an exclusive lock on the child.
>
> 14. Delete the being-purged cgroup. Treat EBUSY and ENOENT as success:
> the first means that a concurrently-running program re-created the
> cgroup, while the second means that a concurrently-running program
> deleted it. The name of the cgroup being purged is currently at
> the top of the path stack.
>
> 15. Pop the name of the cgroup being purged off of the stack.
>
> At the end, self is in the same state it was before the operation.
>
> If you are complaining that this is about as readable as Forth,
> then I agree with you :).
Could this be made clearer by not mutating self, and either just storing
the child stuff in local variables or another Cgroup object? I think a
big part of the confusion here is that this function temporarily changes
which cgroup the Cgroup object it's called on refers to while it's
running. That's extremely difficult to reason about.
>>> + Err(e) => return Err(format!("Cannot purge {:?}: {e}", self.joined_path(path))),
>>> + }
>>> + if !self.open_sub_cgroup(path, true, true)? {
>>> + return Ok(());
>>> + }
>>> +
>>> + rustix::fs::flock(
>>> + self.fd[self.fd.len() - 2].0.as_fd(),
>>> + FlockOperation::LockShared,
>>
>> We already must have at least a shared lock on this at this point, no?
>> I don't think we need another one.
>
> We actually have an exclusive lock. If it succeeds,
> Cgroup::open_sub_cgroup() pushes a file descriptor onto self.fd.
> Therefore, the fd being locked here is the one that was initially on the
> top of the stack. We assert that an exclusive lock is held on that FD.
>
> Waiting for the control group to become empty is a blocking operation,
> so this downgrades the lock to a shared one. Otherwise, an in-progress
> purge of /a/b would prevent /a/c from being created.
Ah, didn't realise it would downgrade. Makes sense.
>>> +fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> {
>>> + if remaining_depth < 1 {
>>> + panic!("control groups too deeply nested");
>>> + }
>>> + let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating");
>>> + while let Some(element) = d.next() {
>>> + let element = element.expect("Iterating through a cgroup directory failed?");
>>> + if element.file_type() != rustix::fs::FileType::Directory {
>>> + continue;
>>> + }
>>> +
>>> + let remaining_depth = remaining_depth - 1;
>>> + let d: &rustix::fs::Dir = &d;
>>> + let dirfd = d.fd().unwrap();
>>> + let path = element.file_name();
>>> + remove_all(remaining_depth, dirfd, path)?;
I'd inline these variables into the function call the extent possible.
Otherwise I have to follow a lot of shuffling around.
>>> + }
>>> + drop(d);
>>> + Ok(())
>>> +}
>>> +
>>> +fn remove_all(
>>> + remaining_depth: usize,
>>> + dirfd: BorrowedFd<'_>,
>>> + path: &std::ffi::CStr,
>>> +) -> Result<(), Errno> {
>>> + if path == c"." || path == c".." {
>>> + return Ok(());
>>> + }
It's a bit weird that calling remove_all on . or .. does not fail.
Maybe would be clearer to move this check to the call site?
>>> + if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() {
>>> + return Ok(());
>>> + }
We could drop this, right? A few extra syscalls, but less to wrap my
head around.
>>> + let fd = rustix::fs::openat2(
>>> + dirfd,
>>> + path,
>>> + OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY,
>>> + Mode::empty(),
>>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>>> + )?;
>>> + remove_recursively(fd, remaining_depth)?;
>>> + rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?;
>>> + Ok(())
>>> +}
>>
>> Could we save a lot of code by calling std::fs::remove_dir_all with a
>> /proc/self/fd path? It's already documented to ignore symlinks.
>
> I tried, but that tries to delete files too, and that isn't allowed.
Ah, alright. That could use an explanatory comment.
How can we be confident we have enough stack for this? Is there a way
it could be done non-recursively, with state on the heap?
>>> + let status = parse_digit_string(&args.next().unwrap(), "exit status")?;
>>> + let signal = args.next().unwrap();
>>> + let signal = if status == 256 {
>>> + Some(parse_digit_string(&signal, "signal number")?)
>>> + } else {
>>> + None
>>> + };
>>> + let service = args.next().unwrap();
>>> +
>>> + let (path, mut cgroup) = open_relative_cgroup(service)?;
>>> + let cgroup_target = Path::new(path.file_name().unwrap());
>>> + let exit_125 = if let Some(signal) = signal {
>>> + match signal as libc::c_int {
>>> + libc::SIGBUS
>>> + | libc::SIGFPE
>>> + | libc::SIGABRT
>>> + | libc::SIGTRAP
>>> + | libc::SIGSEGV
>>> + | libc::SIGILL => {
>>> + // Process *crashed*, indicating a *possible exploit attempt*.
>>> + // s6 should *not* restart it. This is distinct from a Rust panic,
>>> + // which is much less likely to indicate memory corruption.
>>> + true
>>> + }
>>
>> This has absolutely nothing to do with cgroups. If you want to have
>> some common finish behaviour, a program called cgroup-setup is not the
>> place for it. I don't think there's any need for a separate
>> cgroup-s6-finish mode (as opposed to cgroup-purge).
>
> This program is a multi-call binary, so the various things it can do
> aren't necessarily super tightly related. For instance, all of the
> execline binaries can be built as one program, as can most if not
> all busybox applets. When invoked as cgroup-setup or cgroup-purge,
> it indeed only does cgroup-related tasks. cgroup-s6-finish not
> only handles cgroups, but also other tasks related to being an s6
> finish script.
>
> That said, using this changes behavior in a way that isn't related
> to cgroups, so if it is to be used at all it should be in a separate
> patch series. I'll remove this from v5.
Thank you. If we want to have a big multi-call binary that does lots of
different things, only some of which are cgroup-specific, cgroup-setup
is not the name for that program.
>>> +fn local_cgroup() -> Result<PathBuf, String> {
>>> + let mut local_cgroup: Vec<u8> = std::fs::read("/proc/thread-self/cgroup")
>>> + .map_err(|e| format!("cannot read /proc/thread-self/cgroup: {e}"))?;
>>> + let local_cgroup_len = local_cgroup.len();
>>> + if local_cgroup_len < 5
>>> + || local_cgroup[..4] != *b"0::/"
>>> + || local_cgroup[local_cgroup_len - 1] != b'\n'
>>> + || local_cgroup[4..local_cgroup_len - 1].contains(&b'\n')
>>
>> Last time I suggested a clearer way of doing this, but it has instead
>> got even less clear.
>>
>> (I'm not sure why we'd care if there's a newline specifically, as
>> opposed to any other control character.)
>
> If cgroups v1 is in use, the file can contain multiple lines, one for
> each cgroup the program is in. I also am not sure if starting with
> "0::/" is an invariant in that case.
>
> Using this program with cgroups v1 mounted is user error and will
> never happen on Spectrum, but if this tool is used outside of Spectrum,
> it could happen.
I see. It seems like with cgroups v1, it _could_ start with 0::/, but
probably wouldn't. I think it may not be possible to tell from this
file whether cgroups v1 is in use.
So I suppose it depends what you want to happen if cgroups v1 is in use.
If it looks enough like cgroups v2, do you continue, or do you
explicitly check for cgroups v1? If the latter (sounds more sensible to
me), you need to explicitly check for cgroups v1 somehow I think. Can
cgroups v1 and v2 be in use at the same time? If so, checking might be
complicated, but if not, you can just check what type of filesystem is
mounted at /sys/fs/cgroup, or see if it has a
/sys/fs/cgroup/cgroup.controllers file.
>>> + {
>>> + // It's possible to get here if the cgroup path contains a newline,
>>> + // but that never happens in Spectrum.
>>> + return Err(format!(
>>> + "Invalid contents {local_cgroup:?} of /proc/thread-self/cgroup - \
>>> + do you have cgroups v1 mounted instead of cgroups v2?"
>>> + ));
>>> + }
>>> +
>>> + local_cgroup.copy_within(4..local_cgroup_len - 1, 0);
>>> + local_cgroup.truncate(local_cgroup_len - 5);
>>> + let local_cgroup = OsString::from_vec(local_cgroup);
>>> + check_path(&local_cgroup).unwrap();
>>
>> Why do we need to do this? You're worried the kernel is going to start
>> including .. components in /proc/thread-self/cgroup?
>
> Originally, I was going to create a wrapper around `Path` that
> guaranteed no `.` or `..` components were present. Its constructor
> would have checked this invariant. However, this turned out to be
> more work due to the amount of wrapper functions required.
Wise not to proceed with that, I think. But I don't think we need this
particular instance of the check, given it comes from the kernel.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 06/20] host/rootfs: Enable controllers in sub-cgroups
2026-07-22 1:59 ` [PATCH v4 06/20] host/rootfs: Enable controllers in sub-cgroups Demi Marie Obenour
@ 2026-07-27 12:11 ` Alyssa Ross
2026-07-28 2:19 ` Demi Marie Obenour
0 siblings, 1 reply; 132+ messages in thread
From: Alyssa Ross @ 2026-07-27 12:11 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 945 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Actually use the cgroup manager for the first time.
>
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> host/rootfs/image/etc/init | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
> index 8ec7859ab00e24007a93d4000c8f34e353de50ae..5d31a23bae2f29f35bfeced68242dededf6ae0c7 100755
> --- a/host/rootfs/image/etc/init
> +++ b/host/rootfs/image/etc/init
> @@ -12,4 +12,8 @@ if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
> if { /bin/mount --make-shared / }
> if { /bin/mount -a --mkdir }
>
> +# Enable subtree control of all cgroups and move
> +# process to a child cgroup.
> +/usr/bin/cgroup-setup --init-subtree .
> +
> /bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
>
Why do we need this? Isn't the root cgroup an exception to the "no
internal processes" rule?
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 07/20] host/rootfs: Add comments where cgroups are intentionally not used
2026-07-22 1:59 ` [PATCH v4 07/20] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
@ 2026-07-27 12:12 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-07-27 12:12 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 906 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Most services should use cgroups, but some do not.
>
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run | 1 +
> host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run | 1 +
> host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run | 1 +
> host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run | 1 +
> .../etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run | 1 +
> host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run | 1 +
> host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run | 1 +
> .../image/etc/s6-linux-init/run-image/service/serial-getty/template/run | 1 +
> 8 files changed, 8 insertions(+)
Reviewed-by: Alyssa Ross <hi@alyssa.is>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 08/20] host/rootfs: serial-getty-generator: Use cgroups
2026-07-22 1:59 ` [PATCH v4 08/20] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
@ 2026-07-27 12:14 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-07-27 12:14 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 2848 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Avoids concurrent execution and leaked child processes.
Those things shouldn't happen anyway, but I guess it's cheap to make
sure of it. With cgroup-s6-finish changed to something that only does
cgroups:
Reviewed-by: Alysa Ross <hi@alyssa.is>
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> host/rootfs/file-list.mk | 1 +
> .../s6-linux-init/run-image/service/serial-getty-generator/finish | 5 +++++
> .../etc/s6-linux-init/run-image/service/serial-getty-generator/run | 4 +++-
> 3 files changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
> index c707c01179238231f70164ffda5b7c940b34192a..b899b4356d73d73ad0fc4ecd3837b7a3525d6f5a 100644
> --- a/host/rootfs/file-list.mk
> +++ b/host/rootfs/file-list.mk
> @@ -21,6 +21,7 @@ FILES = \
> image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run \
> image/etc/s6-linux-init/run-image/service/s6-svscan-log/notification-fd \
> image/etc/s6-linux-init/run-image/service/s6-svscan-log/run \
> + image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish \
> image/etc/s6-linux-init/run-image/service/serial-getty-generator/run \
> image/etc/s6-linux-init/run-image/service/serial-getty/notification-fd \
> image/etc/s6-linux-init/run-image/service/serial-getty/run \
> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
> new file mode 100755
> index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
> --- /dev/null
> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
> @@ -0,0 +1,5 @@
> +#!/usr/bin/execlineb -WS3
> +# SPDX-License-Identifier: EUPL-1.2+
> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +
> +cgroup-s6-finish $@
> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
> index bf66ab0878fc6d8e77ca71a4a89e50c139a6bb11..a1846a29cf55892127feb7ecfc0adcd12f0497bd 100755
> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
> @@ -1,7 +1,9 @@
> -#!/bin/execlineb -WP
> +#!/bin/execlineb -WS1
> # SPDX-License-Identifier: EUPL-1.2+
> # SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
>
> +# avoid concurrent operation
> +cgroup-setup --leaf -- $1
> piperw 3 4
> background {
> fdclose 3
>
> --
> 2.55.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 09/20] host/rootfs: Set up parent cgroup for all per-VM services
2026-07-22 1:59 ` [PATCH v4 09/20] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
@ 2026-07-27 12:16 ` Alyssa Ross
2026-07-28 3:01 ` Demi Marie Obenour
0 siblings, 1 reply; 132+ messages in thread
From: Alyssa Ross @ 2026-07-27 12:16 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 1149 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
> index 78f794202bf174f3c036f3e20755ac087a988277..430065b1efae8ce1fe7c7a07b67920daa88bf98c 100755
> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
> @@ -1,5 +1,8 @@
> #!/bin/execlineb -WP
> # SPDX-License-Identifier: EUPL-1.2+
> # SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>
> +# Set up the parent cgroup of all VMMs and VM services
> +cgroup-setup --no-wait -- /vm-services.slice
> s6-svscan -d3 instance
We wouldn't want to wait if vm-services was restarted? (I'm not sure
either way, would just like to read the reasoning.)
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 12/20] host/rootfs: run-appimage: Purge the per-VM cgroup
2026-07-22 1:59 ` [PATCH v4 12/20] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
@ 2026-07-27 12:18 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-07-27 12:18 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 1088 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> host/rootfs/image/usr/bin/run-appimage | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/host/rootfs/image/usr/bin/run-appimage b/host/rootfs/image/usr/bin/run-appimage
> index 672ec22d4b815fb9c2940e13ae08ed4093baee4e..03020c89b02f295844f5ed814034ba66be741f4b 100755
> --- a/host/rootfs/image/usr/bin/run-appimage
> +++ b/host/rootfs/image/usr/bin/run-appimage
> @@ -89,7 +89,12 @@ fdclose 4
> foreground { run-vmm $id }
> fdclose 3
>
> -if { s6-instance-delete /run/service/vm-services $id }
> +if {
> + # Even if s6-instance-delete fails, cgroup-purge is enough
> + # to ensure that the VMM and all per-VM services are killed.
> + foreground { s6-instance-delete /run/service/vm-services $id }
> + cgroup-purge /vm-services.slice/vm-${id}.slice
> +}
>
> if { umount -R /run/vm/by-id/${id}/ns }
> rm -r -- $diskdir /run/vm/by-id/${id} /run/configs/${id}
>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 13/20] host/rootfs: run-flatpak: Purge the per-VM cgroup
2026-07-22 1:59 ` [PATCH v4 13/20] host/rootfs: run-flatpak: " Demi Marie Obenour
@ 2026-07-27 12:19 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-07-27 12:19 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 1088 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> host/rootfs/image/usr/bin/run-flatpak | 7 +++++--
> 1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/host/rootfs/image/usr/bin/run-flatpak b/host/rootfs/image/usr/bin/run-flatpak
> index fafbc82ff6966d8be9c8083a17618f1fb192bd07..cdd13fb52d97751f92864d34a5221367df6d6f0d 100755
> --- a/host/rootfs/image/usr/bin/run-flatpak
> +++ b/host/rootfs/image/usr/bin/run-flatpak
> @@ -87,9 +87,12 @@ if {
> fdclose 4
>
> foreground { run-vmm $id }
> -}
>
> -if { s6-instance-delete -- /run/service/vm-services $id }
> + # Even if s6-instance-delete fails, cgroup-purge is enough
> + # to ensure that the VMM and all per-VM services are killed.
> + foreground { s6-instance-delete /run/service/vm-services $id }
> + cgroup-purge /vm-services.slice/vm-${id}.slice
> +}
>
> if { umount -R /run/vm/by-id/${id}/ns }
> rm -r -- $diskdir /run/vm/by-id/${id} /run/configs/${id}
>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 18/20] host/rootfs: systemd-udevd: Run in cgroup
2026-07-22 1:59 ` [PATCH v4 18/20] host/rootfs: systemd-udevd: " Demi Marie Obenour
@ 2026-07-27 12:20 ` Alyssa Ross
2026-07-28 3:11 ` Demi Marie Obenour
0 siblings, 1 reply; 132+ messages in thread
From: Alyssa Ross @ 2026-07-27 12:20 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 2328 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> host/rootfs/file-list.mk | 1 +
> host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +++++
> host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +++--
> 3 files changed, 9 insertions(+), 2 deletions(-)
>
> diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
> index e3411d40014342e67071a48f9e1c7bc7b3954bcf..065571ef3a5c61fa79a1d2ed878052a5f391be96 100644
> --- a/host/rootfs/file-list.mk
> +++ b/host/rootfs/file-list.mk
> @@ -106,6 +106,7 @@ S6_RC_FILES = \
> image/etc/s6-rc/systemd-udevd-coldplug/dependencies.d/systemd-udevd \
> image/etc/s6-rc/systemd-udevd-coldplug/type \
> image/etc/s6-rc/systemd-udevd-coldplug/up \
> + image/etc/s6-rc/systemd-udevd/finish \
> image/etc/s6-rc/systemd-udevd/notification-fd \
> image/etc/s6-rc/systemd-udevd/run \
> image/etc/s6-rc/systemd-udevd/type \
> diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/finish b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
> new file mode 100755
> index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
> --- /dev/null
> +++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
> @@ -0,0 +1,5 @@
> +#!/usr/bin/execlineb -WS3
> +# SPDX-License-Identifier: EUPL-1.2+
> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +
> +cgroup-s6-finish $@
> diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/run b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
> old mode 100644
> new mode 100755
> index aec6444e951503eae988e666b77fda8f2ae33d72..901f3e6667a8de04a6af39945406a1757455f089
> --- a/host/rootfs/image/etc/s6-rc/systemd-udevd/run
> +++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
> @@ -1,7 +1,8 @@
> -#!/bin/execlineb -WP
> +#!/bin/execlineb -WS1
> # SPDX-License-Identifier: EUPL-1.2+
> # SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
> -s6-setlock /run/sd-notify-wrapper/systemd-udevd.lock
> +
> +cgroup-setup --delegate --child-name udev -- $1
> s6-ipcserver-socketbinder -b0 -m -a 0600 /run/sd-notify-wrapper/systemd-udevd.sock
> background -d {
> fdmove 1 3
>
Is there a reason not to always enable delegation? Is there a reason we
need to set the name here?
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 19/20] host/rootfs: weston: Run in cgroup
2026-07-22 1:59 ` [PATCH v4 19/20] host/rootfs: weston: " Demi Marie Obenour
@ 2026-07-27 12:23 ` Alyssa Ross
2026-07-28 3:14 ` Demi Marie Obenour
0 siblings, 1 reply; 132+ messages in thread
From: Alyssa Ross @ 2026-07-27 12:23 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 2241 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> host/rootfs/file-list.mk | 1 +
> host/rootfs/image/etc/s6-rc/weston/finish | 5 +++++
> host/rootfs/image/etc/s6-rc/weston/run | 9 +++++++--
> 3 files changed, 13 insertions(+), 2 deletions(-)
>
> diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
> index 065571ef3a5c61fa79a1d2ed878052a5f391be96..e890e6c211ff011e494cbdd64a1c26ff6360d94a 100644
> --- a/host/rootfs/file-list.mk
> +++ b/host/rootfs/file-list.mk
> @@ -119,6 +119,7 @@ S6_RC_FILES = \
> image/etc/s6-rc/vmm-env/contents.d/systemd-udevd-coldplug \
> image/etc/s6-rc/vmm-env/type \
> image/etc/s6-rc/weston/dependencies.d/systemd-udevd-coldplug \
> + image/etc/s6-rc/weston/finish \
> image/etc/s6-rc/weston/notification-fd \
> image/etc/s6-rc/weston/run \
> image/etc/s6-rc/weston/type
> diff --git a/host/rootfs/image/etc/s6-rc/weston/finish b/host/rootfs/image/etc/s6-rc/weston/finish
> new file mode 100755
> index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
> --- /dev/null
> +++ b/host/rootfs/image/etc/s6-rc/weston/finish
> @@ -0,0 +1,5 @@
> +#!/usr/bin/execlineb -WS3
> +# SPDX-License-Identifier: EUPL-1.2+
> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +
> +cgroup-s6-finish $@
> diff --git a/host/rootfs/image/etc/s6-rc/weston/run b/host/rootfs/image/etc/s6-rc/weston/run
> old mode 100644
> new mode 100755
> index 8b065ee9a535d0a08e9255900c73b88a0c376d4b..a306d9903da6da920efa55eccdfb4f3d14698a79
> --- a/host/rootfs/image/etc/s6-rc/weston/run
> +++ b/host/rootfs/image/etc/s6-rc/weston/run
> @@ -1,8 +1,13 @@
> -#!/bin/execlineb -WP
> +#!/bin/execlineb -Wp
> # SPDX-License-Identifier: EUPL-1.2+
> # SPDX-FileCopyrightText: 2021, 2025 Alyssa Ross <hi@alyssa.is>
>
> -importas -Siu WAYLAND_DISPLAY
> +multisubstitute {
> + importas -Siu WAYLAND_DISPLAY
> + importas -Si 1
> +}
> +
> +cgroup-setup --leaf --no-wait -- $1
>
> piperw 4 3
> background {
>
It would also have been fine to just write "weston" here. ;)
(But if you prefer this way I don't mind.)
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 06/20] host/rootfs: Enable controllers in sub-cgroups
2026-07-27 12:11 ` Alyssa Ross
@ 2026-07-28 2:19 ` Demi Marie Obenour
2026-07-29 14:13 ` Alyssa Ross
0 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-28 2:19 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1: Type: text/plain, Size: 1324 bytes --]
On 7/27/26 08:11, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> Actually use the cgroup manager for the first time.
>>
>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>> ---
>> host/rootfs/image/etc/init | 4 ++++
>> 1 file changed, 4 insertions(+)
>>
>> diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
>> index 8ec7859ab00e24007a93d4000c8f34e353de50ae..5d31a23bae2f29f35bfeced68242dededf6ae0c7 100755
>> --- a/host/rootfs/image/etc/init
>> +++ b/host/rootfs/image/etc/init
>> @@ -12,4 +12,8 @@ if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
>> if { /bin/mount --make-shared / }
>> if { /bin/mount -a --mkdir }
>>
>> +# Enable subtree control of all cgroups and move
>> +# process to a child cgroup.
>> +/usr/bin/cgroup-setup --init-subtree .
>> +
>> /bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
>>
>
> Why do we need this? Isn't the root cgroup an exception to the "no
> internal processes" rule?
It is indeed exempt, though as systemd does create a sub-cgroup for
itself (init.scope) I don't know if the corresponding code paths in the
kernel are well-tested. However, enabling controllers *is* necessary.
If you'd prefer, I can use sed for that.
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 09/20] host/rootfs: Set up parent cgroup for all per-VM services
2026-07-27 12:16 ` Alyssa Ross
@ 2026-07-28 3:01 ` Demi Marie Obenour
2026-07-29 14:29 ` Alyssa Ross
0 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-28 3:01 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1: Type: text/plain, Size: 2029 bytes --]
On 7/27/26 08:16, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>> ---
>> host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run | 3 +++
>> 1 file changed, 3 insertions(+)
>>
>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
>> index 78f794202bf174f3c036f3e20755ac087a988277..430065b1efae8ce1fe7c7a07b67920daa88bf98c 100755
>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
>> @@ -1,5 +1,8 @@
>> #!/bin/execlineb -WP
>> # SPDX-License-Identifier: EUPL-1.2+
>> # SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
>> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>>
>> +# Set up the parent cgroup of all VMMs and VM services
>> +cgroup-setup --no-wait -- /vm-services.slice
>> s6-svscan -d3 instance
>
> We wouldn't want to wait if vm-services was restarted? (I'm not sure
> either way, would just like to read the reasoning.)
cgroup-setup waits for the cgroup *and all of its transitive child
cgroups* to become empty. Therefore, waiting would block until all
VMMs had exited and all of their services had stopped.
Even if it only waited on /vm-service.slice/$inner.service to become
empty, it would still wait for all of the s6-supervise processes in
that cgroup to exit. That's unlikely to ever happen.
Running more than one instance of s6-svscan (or s6-supervise) is
harmless: the program immediately exits with status 100. That will
cause it to be restarted indefinitely, consuming CPU. However, I
don't know a good alternative unless s6 gets support for bind-mounted
pidfds, allowing s6-svscan and s6-supervise to keep track of child
processes across restarts.
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 18/20] host/rootfs: systemd-udevd: Run in cgroup
2026-07-27 12:20 ` Alyssa Ross
@ 2026-07-28 3:11 ` Demi Marie Obenour
2026-07-29 14:15 ` Alyssa Ross
0 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-28 3:11 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1: Type: text/plain, Size: 2864 bytes --]
On 7/27/26 08:20, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>> ---
>> host/rootfs/file-list.mk | 1 +
>> host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +++++
>> host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +++--
>> 3 files changed, 9 insertions(+), 2 deletions(-)
>>
>> diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
>> index e3411d40014342e67071a48f9e1c7bc7b3954bcf..065571ef3a5c61fa79a1d2ed878052a5f391be96 100644
>> --- a/host/rootfs/file-list.mk
>> +++ b/host/rootfs/file-list.mk
>> @@ -106,6 +106,7 @@ S6_RC_FILES = \
>> image/etc/s6-rc/systemd-udevd-coldplug/dependencies.d/systemd-udevd \
>> image/etc/s6-rc/systemd-udevd-coldplug/type \
>> image/etc/s6-rc/systemd-udevd-coldplug/up \
>> + image/etc/s6-rc/systemd-udevd/finish \
>> image/etc/s6-rc/systemd-udevd/notification-fd \
>> image/etc/s6-rc/systemd-udevd/run \
>> image/etc/s6-rc/systemd-udevd/type \
>> diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/finish b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
>> new file mode 100755
>> index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
>> --- /dev/null
>> +++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
>> @@ -0,0 +1,5 @@
>> +#!/usr/bin/execlineb -WS3
>> +# SPDX-License-Identifier: EUPL-1.2+
>> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>> +
>> +cgroup-s6-finish $@
>> diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/run b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
>> old mode 100644
>> new mode 100755
>> index aec6444e951503eae988e666b77fda8f2ae33d72..901f3e6667a8de04a6af39945406a1757455f089
>> --- a/host/rootfs/image/etc/s6-rc/systemd-udevd/run
>> +++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
>> @@ -1,7 +1,8 @@
>> -#!/bin/execlineb -WP
>> +#!/bin/execlineb -WS1
>> # SPDX-License-Identifier: EUPL-1.2+
>> # SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
>> -s6-setlock /run/sd-notify-wrapper/systemd-udevd.lock
>> +
>> +cgroup-setup --delegate --child-name udev -- $1
>> s6-ipcserver-socketbinder -b0 -m -a 0600 /run/sd-notify-wrapper/systemd-udevd.sock
>> background -d {
>> fdmove 1 3
>>
>
> Is there a reason not to always enable delegation? Is there a reason we
> need to set the name here?
--delegate only sets the user.delegate=1 xattr used by systemd.
Programs that are systemd-aware and manage their own cgroups check this
xattr to see if a cgroup has been delegated by systemd. The kernel
and other programs do not care.
systemd-udevd.service uses the "udev" name, so I decided to also use it
for consistency.
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 19/20] host/rootfs: weston: Run in cgroup
2026-07-27 12:23 ` Alyssa Ross
@ 2026-07-28 3:14 ` Demi Marie Obenour
0 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-28 3:14 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1: Type: text/plain, Size: 2430 bytes --]
On 7/27/26 08:23, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>> ---
>> host/rootfs/file-list.mk | 1 +
>> host/rootfs/image/etc/s6-rc/weston/finish | 5 +++++
>> host/rootfs/image/etc/s6-rc/weston/run | 9 +++++++--
>> 3 files changed, 13 insertions(+), 2 deletions(-)
>>
>> diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
>> index 065571ef3a5c61fa79a1d2ed878052a5f391be96..e890e6c211ff011e494cbdd64a1c26ff6360d94a 100644
>> --- a/host/rootfs/file-list.mk
>> +++ b/host/rootfs/file-list.mk
>> @@ -119,6 +119,7 @@ S6_RC_FILES = \
>> image/etc/s6-rc/vmm-env/contents.d/systemd-udevd-coldplug \
>> image/etc/s6-rc/vmm-env/type \
>> image/etc/s6-rc/weston/dependencies.d/systemd-udevd-coldplug \
>> + image/etc/s6-rc/weston/finish \
>> image/etc/s6-rc/weston/notification-fd \
>> image/etc/s6-rc/weston/run \
>> image/etc/s6-rc/weston/type
>> diff --git a/host/rootfs/image/etc/s6-rc/weston/finish b/host/rootfs/image/etc/s6-rc/weston/finish
>> new file mode 100755
>> index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
>> --- /dev/null
>> +++ b/host/rootfs/image/etc/s6-rc/weston/finish
>> @@ -0,0 +1,5 @@
>> +#!/usr/bin/execlineb -WS3
>> +# SPDX-License-Identifier: EUPL-1.2+
>> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>> +
>> +cgroup-s6-finish $@
>> diff --git a/host/rootfs/image/etc/s6-rc/weston/run b/host/rootfs/image/etc/s6-rc/weston/run
>> old mode 100644
>> new mode 100755
>> index 8b065ee9a535d0a08e9255900c73b88a0c376d4b..a306d9903da6da920efa55eccdfb4f3d14698a79
>> --- a/host/rootfs/image/etc/s6-rc/weston/run
>> +++ b/host/rootfs/image/etc/s6-rc/weston/run
>> @@ -1,8 +1,13 @@
>> -#!/bin/execlineb -WP
>> +#!/bin/execlineb -Wp
>> # SPDX-License-Identifier: EUPL-1.2+
>> # SPDX-FileCopyrightText: 2021, 2025 Alyssa Ross <hi@alyssa.is>
>>
>> -importas -Siu WAYLAND_DISPLAY
>> +multisubstitute {
>> + importas -Siu WAYLAND_DISPLAY
>> + importas -Si 1
>> +}
>> +
>> +cgroup-setup --leaf --no-wait -- $1
>>
>> piperw 4 3
>> background {
>>
>
> It would also have been fine to just write "weston" here. ;)
> (But if you prefer this way I don't mind.)
I like your way a bit better :).
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 03/20] Documentation: Mention control groups
2026-07-22 1:59 ` [PATCH v4 03/20] Documentation: Mention control groups Demi Marie Obenour
2026-07-27 11:22 ` Alyssa Ross
@ 2026-07-28 10:41 ` Valentin Gagarin
1 sibling, 0 replies; 132+ messages in thread
From: Valentin Gagarin @ 2026-07-28 10:41 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development, Alyssa Ross
On Tue, 21 Jul 2026 21:59:08 -0400, Demi Marie Obenour <demiobenour@gmail.com> wrote:
> diff --git a/Documentation/doc/development/control-groups.adoc b/Documentation/doc/development/control-groups.adoc
> new file mode 100644
> index 0000000..6ce33f2
> --- /dev/null
> +++ b/Documentation/doc/development/control-groups.adoc
> @@ -0,0 +1,88 @@
> += Control groups in Spectrum
> +
> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0
> +
> +Linux control groups (cgroups) can be used for several purposes:
> +
> +1. They allow waiting for a group of processes to exit.
> +2. They allow terminating a group of processes.
> +3. They allow limiting a group of processes' access to resources.
Not repeating prefixes helps readability. And a link to upstream
documentation is always good.
https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html[Linux control groups] (cgroups) can be used for several purposes:
1. Waiting for a group of processes to exit.
2. Terminating a group of processes.
3. Limiting a group of processes' access to resources.
> [ ... skip 10 lines ... ]
> +2. The per-VM services for each VM are under `/vm-services.slice/vm-${VM}.slice`,
> + where `${VM}` is replaced by the VM's ID.
> +3. Each per-VM service is under `/vm-services.slice/vm-${VM}.slice/${SERVICE_NAME}`,
> + where `${VM}` is replaced by the VM's ID and `${SERVICE_NAME}` is replaced by
> + the name of the service.
> +4. The VMM runs under `/vm-services.slice/vm-${VM}.slice/vmm`.
I don't think we introduce VMM anywhere else in the documentation, so
let's write it out.
4. The virtual machine manager (VMM) runs under `/vm-services.slice/vm-${VM}.slice/vmm`.
> +
> +If a cgroup contains child cgroups, it likely contains a `$inner.service`
> +cgroup. This is where programs that would otherwise run in the cgroup itself
> +are placed. Generally, these programs are instances of `s6-svscan` and/or
> +`s6-supervise`.
Since we're not discussing s6 anywhere, please link to what these things
mean. Also I think it's fine to imply logical OR when using "or" with
our audience, and reduce the visual noise (the general recommendation is
to avoid slashes in prose).
are placed. Generally, these programs are instances of
https://skarnet.org/software/s6/s6-svscan.html[`s6-svscan`] or
https://skarnet.org/software/s6/s6-supervise.html[`s6-supervise`].
> +
> +== Using Control Groups
> +
> +When adding a new s6 service, one should carefully consider whether it
> +should be placed in a control group. Most services should be placed in
> +a control group, with only a few exceptions:
The first sentence reads as if one now has to make serious decisions,
but the second one amounts to "mostly no". This was confusing to read,
and took attention away from the relevant parts, which seem to revolve
around "how to do that". I recommend removing the first sentence.
Most services should be placed in
a control group, with only a few exceptions:
> +
> +1. Services, such as `getty`, that spawn background processes.
> +2. Loggers.
> +3. Trivial services that don't do anything.
> +
> +Generally, it's best to set the control group up as the first thing
> +the service does. To do that, use `cgroup-setup --leaf -- $1 COMMAND_LINE`,
> +where `$1` should be the service name and `COMMAND_LINE` is the program
> +to run in a cgroup.
It took me a while to figure out that this is a new tool that comes
with Spectrum. Not sure how to handle that best, but here's an attempt:
Generally, it's best to set the control group up as the first thing
the service does, with Spectrum's https://spectrum-os.org/git/spectrum/tree/tools/cgroup-setup[`cgroup-setup`].
Call `cgroup-setup --leaf -- $1 COMMAND_LINE`,
where `$1` should be the service name and `COMMAND_LINE` is the program
to run in a cgroup.
> +
> +If you use execline for your run script, this is as simple as:
For example, using https://skarnet.org/software/execline/[execline] for your run script:
> +
> +.run
> +....
But I'd prefer "manually" to make that unambiguous.
> +#!/bin/execlineb -WS1
That renders as a separate paragraph only containing "run". Let's
remove that, it doesn't make sense visuallyr
> +
> +cgroup-setup --leaf -- $1
> +# rest of script comes here
> +....
> +
> +If the service exits, it's usually best to terminate any programs left
> +behind with SIGKILL and remove the control group. In Spectrum, this is
To keep the convention of displaying code items in monospace. Also
shouldn't it be "when a service exits", since it's usually a matter of
time and holds for any service?
When a service exits, it's usually best to terminate any programs left
behind with `SIGKILL` and remove the control group. In Spectrum, this is
This new (sub-)section could use a heading, such as "Purging control
groups". A separate section also removes the implication that we're
still talking about "the" same service as before.
> +called "purging" the cgroup. To purge the cgroup when a service exits,
> +make the `finish` script invoke `/usr/bin/cgroup-s6-finish`. The first
> +two command line arguments must be the first two arguments passed to the
> +`finish` script. The third argument must be the path to the cgroup to
To purge the cgroup when a service exits,
make the `finish` script in the https://skarnet.org/software/s6/servicedir.html[s6 service directory]
invoke Spectrum's `/usr/bin/cgroup-s6-finish`.
This also needs an example. It took me a couple of times to visualise
that it's just `cgroup-s6-finish $1 $2 $3`.
And is it necessary to say `/usr/bin/...`? Ideally we'd already
indicate here that `cgroup-s6-finish` is a symlink to `cgroup-setup`.
> +be purged relative to the cgroup the program itself is in. This is
> +usually, but not always, the third argument to the `finish` script.
This needs more information. When is it the third argument, when is it
not? I'd be helpless at this point.
> +
> +When invoked as `cgroup-s6-finish`, `cgroup-setup` checks if
> +the service exited due to a signal that caused it to dump core. If it
> +did, `cgroup-s6-finish` exits with status 125, ensuring that
> +`s6-supervise` will *not* restart it. This is intentional: if a service
+crashes due to a fatal signal, this is possibly a sign of memory
> +crashes due to a fatal signal, this is possibly a sign of memory
> +corruption. Restarting the service in this case can turn an unreliable
This should probably go into reference documentation for that program,
but let's keep it here until we have a place to list it and a way to do
that automatically.
> +memory corruption exploit into a reliable one. Rust panics do not cause
> +core dumps, so the service will be restarted afterwards.
I don't understand what that last sentence means for me as a user. This
seems detached from the previous explanations.
> +
> +One can also use `cgroup-purge` to purge a cgroup explicitly. This is
> +used to stop the VMM and all per-VM services when a VM is shut down.
Do you mean that command is intended to be used manually? I'm not sure
here, because "explicitly" doesn't intuitively convey that to me. If
you want to keep the terminology, an example with a bit of introduction
would help, such as:
> +
> +== Future plans
> +
> +Control groups are designed around a single writer process controlling each
> +of them. Many Linux distros use systemd for this, but Spectrum doesn't use
> +systemd. The only persistent per-service process is s6-supervise, but that
> +doesn't have control group support.
s/distros/distributions/ and maybe highlight command names with
monospace.
Control groups are designed around a single writer process controlling each
of them. Many Linux distributions use `systemd` for this, but Spectrum doesn't use
`systemd`. The only persistent per-service process is `s6-supervise`, but that
doesn't have control group support.
Generally I strongly recommend one sentence per line for documentation,
a recommendation that apparently goes back to early Unix[1]. That makes
review a lot easier. As you can observe, otherwise one has to cut
random unrelated pieces from the diff to discuss one word or phrase from
a sentence. We don't have to introduce it now, but should start this as
a convention.
[1]: https://rhodesmill.org/brandon/2012/one-sentence-per-line/
Reviewed-by: Valentin Gagarin <valentin@gagarin.work>
--
Valentin Gagarin <valentin@gagarin.work>
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 06/20] host/rootfs: Enable controllers in sub-cgroups
2026-07-28 2:19 ` Demi Marie Obenour
@ 2026-07-29 14:13 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-07-29 14:13 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 1681 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> On 7/27/26 08:11, Alyssa Ross wrote:
>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>
>>> Actually use the cgroup manager for the first time.
>>>
>>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>>> ---
>>> host/rootfs/image/etc/init | 4 ++++
>>> 1 file changed, 4 insertions(+)
>>>
>>> diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
>>> index 8ec7859ab00e24007a93d4000c8f34e353de50ae..5d31a23bae2f29f35bfeced68242dededf6ae0c7 100755
>>> --- a/host/rootfs/image/etc/init
>>> +++ b/host/rootfs/image/etc/init
>>> @@ -12,4 +12,8 @@ if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
>>> if { /bin/mount --make-shared / }
>>> if { /bin/mount -a --mkdir }
>>>
>>> +# Enable subtree control of all cgroups and move
>>> +# process to a child cgroup.
>>> +/usr/bin/cgroup-setup --init-subtree .
>>> +
>>> /bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
>>>
>>
>> Why do we need this? Isn't the root cgroup an exception to the "no
>> internal processes" rule?
>
> It is indeed exempt, though as systemd does create a sub-cgroup for
> itself (init.scope) I don't know if the corresponding code paths in the
> kernel are well-tested. However, enabling controllers *is* necessary.
> If you'd prefer, I can use sed for that.
I'd prefer to leave things as they come unless we have a reason not to.
I wonder if there's a technical reason for systemd to do that, or if it
just fit into their design better? If you can write a comment
explaining why we need a cgroup, then this is fine; otherwise let's
stick to what's strictly necessary.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 18/20] host/rootfs: systemd-udevd: Run in cgroup
2026-07-28 3:11 ` Demi Marie Obenour
@ 2026-07-29 14:15 ` Alyssa Ross
2026-07-29 20:39 ` Demi Marie Obenour
0 siblings, 1 reply; 132+ messages in thread
From: Alyssa Ross @ 2026-07-29 14:15 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 3237 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> On 7/27/26 08:20, Alyssa Ross wrote:
>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>
>>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>>> ---
>>> host/rootfs/file-list.mk | 1 +
>>> host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +++++
>>> host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +++--
>>> 3 files changed, 9 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
>>> index e3411d40014342e67071a48f9e1c7bc7b3954bcf..065571ef3a5c61fa79a1d2ed878052a5f391be96 100644
>>> --- a/host/rootfs/file-list.mk
>>> +++ b/host/rootfs/file-list.mk
>>> @@ -106,6 +106,7 @@ S6_RC_FILES = \
>>> image/etc/s6-rc/systemd-udevd-coldplug/dependencies.d/systemd-udevd \
>>> image/etc/s6-rc/systemd-udevd-coldplug/type \
>>> image/etc/s6-rc/systemd-udevd-coldplug/up \
>>> + image/etc/s6-rc/systemd-udevd/finish \
>>> image/etc/s6-rc/systemd-udevd/notification-fd \
>>> image/etc/s6-rc/systemd-udevd/run \
>>> image/etc/s6-rc/systemd-udevd/type \
>>> diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/finish b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
>>> new file mode 100755
>>> index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
>>> --- /dev/null
>>> +++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
>>> @@ -0,0 +1,5 @@
>>> +#!/usr/bin/execlineb -WS3
>>> +# SPDX-License-Identifier: EUPL-1.2+
>>> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>>> +
>>> +cgroup-s6-finish $@
>>> diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/run b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
>>> old mode 100644
>>> new mode 100755
>>> index aec6444e951503eae988e666b77fda8f2ae33d72..901f3e6667a8de04a6af39945406a1757455f089
>>> --- a/host/rootfs/image/etc/s6-rc/systemd-udevd/run
>>> +++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
>>> @@ -1,7 +1,8 @@
>>> -#!/bin/execlineb -WP
>>> +#!/bin/execlineb -WS1
>>> # SPDX-License-Identifier: EUPL-1.2+
>>> # SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
>>> -s6-setlock /run/sd-notify-wrapper/systemd-udevd.lock
>>> +
>>> +cgroup-setup --delegate --child-name udev -- $1
>>> s6-ipcserver-socketbinder -b0 -m -a 0600 /run/sd-notify-wrapper/systemd-udevd.sock
>>> background -d {
>>> fdmove 1 3
>>>
>>
>> Is there a reason not to always enable delegation? Is there a reason we
>> need to set the name here?
>
> --delegate only sets the user.delegate=1 xattr used by systemd.
> Programs that are systemd-aware and manage their own cgroups check this
> xattr to see if a cgroup has been delegated by systemd. The kernel
> and other programs do not care.
Right, but why can't we just always set user.delegate=1? Would we ever
have a program that checked it that we didn't want to manage its own
cgroups?
> systemd-udevd.service uses the "udev" name, so I decided to also use it
> for consistency.
Is this the only reason we even need a --child-name feature? If so, it
doesn't seem well justified to me unless it actually affects something.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 09/20] host/rootfs: Set up parent cgroup for all per-VM services
2026-07-28 3:01 ` Demi Marie Obenour
@ 2026-07-29 14:29 ` Alyssa Ross
2026-07-29 20:20 ` Demi Marie Obenour
0 siblings, 1 reply; 132+ messages in thread
From: Alyssa Ross @ 2026-07-29 14:29 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 2200 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> On 7/27/26 08:16, Alyssa Ross wrote:
>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>
>>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>>> ---
>>> host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run | 3 +++
>>> 1 file changed, 3 insertions(+)
>>>
>>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
>>> index 78f794202bf174f3c036f3e20755ac087a988277..430065b1efae8ce1fe7c7a07b67920daa88bf98c 100755
>>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
>>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
>>> @@ -1,5 +1,8 @@
>>> #!/bin/execlineb -WP
>>> # SPDX-License-Identifier: EUPL-1.2+
>>> # SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
>>> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>>>
>>> +# Set up the parent cgroup of all VMMs and VM services
>>> +cgroup-setup --no-wait -- /vm-services.slice
>>> s6-svscan -d3 instance
>>
>> We wouldn't want to wait if vm-services was restarted? (I'm not sure
>> either way, would just like to read the reasoning.)
>
> cgroup-setup waits for the cgroup *and all of its transitive child
> cgroups* to become empty. Therefore, waiting would block until all
> VMMs had exited and all of their services had stopped.
>
> Even if it only waited on /vm-service.slice/$inner.service to become
> empty, it would still wait for all of the s6-supervise processes in
> that cgroup to exit. That's unlikely to ever happen.
>
> Running more than one instance of s6-svscan (or s6-supervise) is
> harmless: the program immediately exits with status 100. That will
> cause it to be restarted indefinitely, consuming CPU. However, I
> don't know a good alternative unless s6 gets support for bind-mounted
> pidfds, allowing s6-svscan and s6-supervise to keep track of child
> processes across restarts.
Why is this different to the top-level vmm service? It also just runs
s6-svscan, but it does not use --no-wait.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 09/20] host/rootfs: Set up parent cgroup for all per-VM services
2026-07-29 14:29 ` Alyssa Ross
@ 2026-07-29 20:20 ` Demi Marie Obenour
0 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-29 20:20 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1: Type: text/plain, Size: 2411 bytes --]
On 7/29/26 10:29, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> On 7/27/26 08:16, Alyssa Ross wrote:
>>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>>
>>>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>>>> ---
>>>> host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run | 3 +++
>>>> 1 file changed, 3 insertions(+)
>>>>
>>>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
>>>> index 78f794202bf174f3c036f3e20755ac087a988277..430065b1efae8ce1fe7c7a07b67920daa88bf98c 100755
>>>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
>>>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
>>>> @@ -1,5 +1,8 @@
>>>> #!/bin/execlineb -WP
>>>> # SPDX-License-Identifier: EUPL-1.2+
>>>> # SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
>>>> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>>>>
>>>> +# Set up the parent cgroup of all VMMs and VM services
>>>> +cgroup-setup --no-wait -- /vm-services.slice
>>>> s6-svscan -d3 instance
>>>
>>> We wouldn't want to wait if vm-services was restarted? (I'm not sure
>>> either way, would just like to read the reasoning.)
>>
>> cgroup-setup waits for the cgroup *and all of its transitive child
>> cgroups* to become empty. Therefore, waiting would block until all
>> VMMs had exited and all of their services had stopped.
>>
>> Even if it only waited on /vm-service.slice/$inner.service to become
>> empty, it would still wait for all of the s6-supervise processes in
>> that cgroup to exit. That's unlikely to ever happen.
>>
>> Running more than one instance of s6-svscan (or s6-supervise) is
>> harmless: the program immediately exits with status 100. That will
>> cause it to be restarted indefinitely, consuming CPU. However, I
>> don't know a good alternative unless s6 gets support for bind-mounted
>> pidfds, allowing s6-svscan and s6-supervise to keep track of child
>> processes across restarts.
>
> Why is this different to the top-level vmm service? It also just runs
> s6-svscan, but it does not use --no-wait.
It is different because this one is correct and the other is wrong :).
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 18/20] host/rootfs: systemd-udevd: Run in cgroup
2026-07-29 14:15 ` Alyssa Ross
@ 2026-07-29 20:39 ` Demi Marie Obenour
2026-07-30 14:55 ` Alyssa Ross
0 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-29 20:39 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1: Type: text/plain, Size: 3601 bytes --]
On 7/29/26 10:15, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> On 7/27/26 08:20, Alyssa Ross wrote:
>>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>>
>>>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>>>> ---
>>>> host/rootfs/file-list.mk | 1 +
>>>> host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +++++
>>>> host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +++--
>>>> 3 files changed, 9 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
>>>> index e3411d40014342e67071a48f9e1c7bc7b3954bcf..065571ef3a5c61fa79a1d2ed878052a5f391be96 100644
>>>> --- a/host/rootfs/file-list.mk
>>>> +++ b/host/rootfs/file-list.mk
>>>> @@ -106,6 +106,7 @@ S6_RC_FILES = \
>>>> image/etc/s6-rc/systemd-udevd-coldplug/dependencies.d/systemd-udevd \
>>>> image/etc/s6-rc/systemd-udevd-coldplug/type \
>>>> image/etc/s6-rc/systemd-udevd-coldplug/up \
>>>> + image/etc/s6-rc/systemd-udevd/finish \
>>>> image/etc/s6-rc/systemd-udevd/notification-fd \
>>>> image/etc/s6-rc/systemd-udevd/run \
>>>> image/etc/s6-rc/systemd-udevd/type \
>>>> diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/finish b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
>>>> new file mode 100755
>>>> index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
>>>> --- /dev/null
>>>> +++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
>>>> @@ -0,0 +1,5 @@
>>>> +#!/usr/bin/execlineb -WS3
>>>> +# SPDX-License-Identifier: EUPL-1.2+
>>>> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>>>> +
>>>> +cgroup-s6-finish $@
>>>> diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/run b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
>>>> old mode 100644
>>>> new mode 100755
>>>> index aec6444e951503eae988e666b77fda8f2ae33d72..901f3e6667a8de04a6af39945406a1757455f089
>>>> --- a/host/rootfs/image/etc/s6-rc/systemd-udevd/run
>>>> +++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
>>>> @@ -1,7 +1,8 @@
>>>> -#!/bin/execlineb -WP
>>>> +#!/bin/execlineb -WS1
>>>> # SPDX-License-Identifier: EUPL-1.2+
>>>> # SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
>>>> -s6-setlock /run/sd-notify-wrapper/systemd-udevd.lock
>>>> +
>>>> +cgroup-setup --delegate --child-name udev -- $1
>>>> s6-ipcserver-socketbinder -b0 -m -a 0600 /run/sd-notify-wrapper/systemd-udevd.sock
>>>> background -d {
>>>> fdmove 1 3
>>>>
>>>
>>> Is there a reason not to always enable delegation? Is there a reason we
>>> need to set the name here?
>>
>> --delegate only sets the user.delegate=1 xattr used by systemd.
>> Programs that are systemd-aware and manage their own cgroups check this
>> xattr to see if a cgroup has been delegated by systemd. The kernel
>> and other programs do not care.
>
> Right, but why can't we just always set user.delegate=1? Would we ever
> have a program that checked it that we didn't want to manage its own
> cgroups?
It's slightly wasteful (extra syscall) but harmless.
>> systemd-udevd.service uses the "udev" name, so I decided to also use it
>> for consistency.
>
> Is this the only reason we even need a --child-name feature? If so, it
> doesn't seem well justified to me unless it actually affects something.
I'd prefer to stick with what systemd uses, as that is what is tested.
Varying from what systemd uses increases the risk of regressions.
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 02/20] tools: Add control group manager
2026-07-27 12:10 ` Alyssa Ross
@ 2026-07-30 0:40 ` Demi Marie Obenour
2026-07-30 14:53 ` Alyssa Ross
0 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-30 0:40 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1: Type: text/plain, Size: 22042 bytes --]
On 7/27/26 08:10, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> On 7/22/26 12:01, Alyssa Ross wrote:
>>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>>>> +#[derive(Debug)]
>>>> +pub(crate) struct Cgroup {
>>>> + path: PathBuf,
>>>> + fd: Vec<(OwnedFd, bool)>,
>>>
>>> There's no point storing all these exclusivity bools, is there? I think
>>> only the last one is ever checked, so we could make things tighter and
>>> clearer like this, where we only track the exclusivity of the last fd:
>>
>> Cgroup::enable_subtree_control() checks the exclusivity
>> of the caller-provided depth. Line 232 of main.rs calls
>> enable_subtree_control(2).
>>
>> These bools are only used in assertions, so they could be removed.
>> I will leave that up to you. The advantage of keeping them is that a
>> panic is vastly easier to debug than a race condition due to improper
>> locking.
>
> Can you explain to me why we need to support enabling subtree control
> for different depths? Intuitively, I'd expect this program to only ever
> operate on the cgroup associated with the service that's invoking it.
Init runs "cgroup-setup --init-subtree .", which moves PID 1 to
/$inner.service and enables all controllers in /. I'll just use sed
to enable the controllers, which avoids special cases around paths
with no file name.
>> I'm very used to writing this kind of code in C, so I went with a C-like
>> style instead of using Rust stdlib APIs. I don't like having extra
>> abstractions in this kind of code, as it obscures what is going on
>> under the hood. That is less important here, but it's very important
>> in programs like mount-flatpak.
>
> I did say myself that my suggestion here might be too clever, because it
> uses the standard library in a way that's unintuitive. In general,
> though, we can expect readers of Rust code to be more familiar with the
> standard library than with byte-by-byte stringy patch processing. Would
> still be nicer to take &Path here, I think, even if you then walk
> through it byte by byte, just to make it slightly clearer what this does.
Will fix in v5.
>>>> + pub fn read_control_file(&self, fd: BorrowedFd, p: &Path) -> Result<Vec<u8>, String> {
>>>> + let mut buf = Vec::new();
>>>> + let err = |e: &dyn Display, p: &Path, msg: &str| {
>>>> + let path = self.path.join(p);
>>>> + format!("Cannot {msg} {path:?}: {e}")
>>>> + };
>>>> + File::from(open_subtree_raw(Path::new(p), fd.as_fd()).map_err(|e| err(&e, p, "open"))?)
>>>
>>> If we're using it for opening files, open_subtree_raw is probably misnamed.
>>
>> Yup! Do you have a suggestion for improving it? Maybe open_child()?
>
> open_beneath?
Will use in v5.
>>>> + pub fn open_sub_cgroup(
>>>> + &mut self,
>>>> + path: &std::path::Path,
>>>> + exclusive: bool,
>>>> + allow_missing: bool,
>>>> + ) -> Result<bool, String> {
>>>> + let mut iter = path.components().peekable();
>>>> + while let Some(component) = iter.next() {
>>>
>>> Perhaps would be nicer:
>>>
>>> let mut components = path.components().peekable();
>>> for component in components {
>>
>> That results in a borrowcheck error. The for loop takes ownership
>> of the iterator, but .peek() is called inside the loop.
>
> Ah, okay. This is fine then.
Need is obviated in v5 by a refactor.
>>>> + let component = match component {
>>>> + Component::Normal(component) => component,
>>>> + _ => unreachable!(),
>>>> + };
>>>> + let sub_fd = match self
>>>> + .open_sub_cgroup_raw(OFlags::DIRECTORY | OFlags::RDONLY, component.as_bytes())
>>>> + {
>>>> + Ok(sub_fd) => {
>>>> + self.path.push(component);
>>>
>>> I would really like to not try to store self.path. It seems very
>>> complicated to track. It's also very unclear to me from the name (and
>>> the code) what it is. Is it the path to the cgroup itself, or to its
>>> parent? It looks to me like it should be the cgroup itself, but then
>>> what's going on in purge?
>>
>> It's the path to the cgroup itself, relative to /sys/fs/cgroup. Its only
>> purpose is for logging.
>>
>>> We could actually improve readability of this quite complicated function
>>> even further if you find it acceptable to just use Errno for the error
>>> type. In that case, we'd just return Result<(), Errno>, and callers
>>> would check for Errno::NOENT if they wanted to allow missing. Then we
>>> could just completely drop that argument. In my opinion it would be
>>> worth it to move complexity out of here.
>>
>> I can do this, but it would result in much worse error messages: the
>> error would only reference the file name, not the full cgroup path.
>> Which would you prefer?
>
> I would much prefer code I can easily understand. There's always strace
> for getting the full paths when debugging.
This code is dropped in v5.
>>>> + pub fn wait_for_empty(&self) -> std::io::Result<()> {
>>>> + assert!(self.exclusive());
>>>> + let wait_file = self.open_subtree(std::path::Path::new("cgroup.events"))?;
>>>> + let poll_fd = wait_file.as_raw_fd();
>>>> + let mut wait_fd = File::from(wait_file);
>>>> + let mut fds = libc::pollfd {
>>>> + fd: poll_fd,
>>>> + events: libc::POLLPRI | libc::POLLERR,
>>>> + revents: 0,
>>>> + };
>>>> + let mut v = vec![];
>>>> + loop {
>>>> + v.clear();
>>>> + wait_fd
>>>> + .seek(std::io::SeekFrom::Start(0))
>>>> + .expect("Seek on control group file should succeed");
>>>> + wait_fd
>>>> + .read_to_end(&mut v)
>>>> + .expect("reading from control group should work");
>>>> + if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
>>>> + break;
>>>> + }
>>>> + // SAFETY: FFI call, valid arguments, fds contains 1 element
>>>> + if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
>>>> + panic!("poll failed");
>>>> + }
>>>> + }
>>>
>>> Are you 100% confident that this doesn't race? I don't understand why
>>> poll would be triggered in this scenario:
>>>
>>> 1. "1" is written to cgroup.kill
>>> 2. Every process in the cgroup exits and is reaped.
>>> 3. cgroup.events is opened, with the cgroup already empty.
>>>
>>> Are you not relying on 2 happening after 3? Presumably if you open
>>> cgroup.events for a cgroup that's already empty, you're not going to get
>>> a poll event to tell you it's empty.
>>
>> In that case, cgroup.events will include a "populated 0"
>> line, so poll will not be called.
>
> You are correct. :)
>
> Perhaps could be written more clearly with a while, something like this:
>
> while !v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
> // SAFETY: FFI call, valid arguments, fds contains 1 element
> if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
> panic!("poll failed");
> }
> v.clear();
> wait_fd
> .seek(std::io::SeekFrom::Start(0))
> .expect("Seek on control group file should succeed");
> wait_fd
> .read_to_end(&mut v)
> .expect("reading from control group should work");
> }
>
> But if you prefer the current way (which avoids a redundant initial
> check of an empty Vec, although that should be cheap) that's also fine.
That version would call poll() before checking that the cgroup is empty,
and that *can* deadlock :).
>>>> + pub(crate) fn make_child(&mut self, path: &Path) -> Result<(), Errno> {
>>>> + assert!(self.exclusive());
>>>> + let component = path.as_os_str().as_bytes();
>>>> + assert_single_component(component);
>>>> + match rustix::fs::mkdirat(
>>>> + self.as_fd(),
>>>> + path,
>>>> + Mode::RUSR
>>>> + | Mode::WUSR
>>>> + | Mode::XUSR
>>>> + | Mode::RGRP
>>>> + | Mode::XGRP
>>>> + | Mode::ROTH
>>>> + | Mode::XOTH,
>>>> + ) {
>>>> + Ok(()) | Err(Errno::EXIST) => {}
>>>> + bad => return bad,
>>>> + }
>>>> + let p = self.open_sub_cgroup_raw(OFlags::RDONLY | OFlags::DIRECTORY, component)?;
>>>> + // exclusive lock on parent acts as exclusive lock on child
>>>> + self.fd.push((p, true));
>>>> + self.path.push(path);
>>>> + Ok(())
>>>> + }
>>>> +
>>>> + pub(super) fn purge(&mut self, path: &Path) -> Result<(), String> {
>>>
>>> I guess we have to call purge on the parent, rather than on the cgroup
>>> itself, because of the unlink? Maybe we could call it purge_child? It
>>> confused me for a while.
>>
>> Correct. Will rename in v5.
>>
>> The way to understand this code is that Cgroup has two stacks: one
>> for file descriptors and one for path components. All operations
>> operate at a specified depth from the top of the stack. 1 refers to
>> the top of the stack, 2 to one level below that, and so on.
>
> 1-indexing is a little unintuitive for Rust, no?
>
>>
>> This function is really confusing because it performs multiple pushes
>> and pops on the internal file descriptor stack. The specific algorithm is:
>>
>> 1. Start with an exclusive lock.
>>
>> 2. Try to delete the child directly.
>>
>> 3. If deletion succeeds, or if it fails with ENOENT, return success.
>>
>> 4. If deletion fails with anything other than EBUSY, return an error.
>>
>> 5. Open the child cgroup and take an exclusive lock on it. This pushes
>> the child cgroup's FD onto the stack. The open_subtree() method
>> also pushes the child path onto the stack.
>>
>> 6. Take a *shared* lock on the FD that is directly below the top
>> of the stack. This is the file descriptor that was initially
>> on the top of the stack.
>>
>> This releases the exclusive lock, allowing other operations on
>> different children to proceed. Different operations on the cgroup
>> being purged will be blocked by the exclusive lock taken in step 5.
>>
>> 7. Kill all programs in the cgroup by writing 1 to cgroup.kill.
>>
>> 8. Open cgroup.events.
>>
>> 9. Read from the FD opened in step 8. If the file contains the line
>> "populated 0", go to step 11.
>>
>> 10. Call poll() on the FD opened in step 8 to wait for POLLERR or
>> POLLPRI to happen. Then go back to step 9.
>>
>> This is race-free because the kernel will set the "this is ready"
>> flag after every change that affects what would be read from
>> the file.
>>
>> 11. Pop the file descriptor to the being-purged cgroup from the stack.
>>
>> 12. Use the just-popped file descriptor to remove all subdirectories
>> recursively. Files must not be deleted, as the kernel doesn't
>> allow it. Then close the file descriptor, releasing the exclusive
>> lock held on it.
>>
>> 13. Take an exclusive lock on the *parent* of the cgroup that was just purged.
>>
>> This must be done after the file descriptor to the cgroup being
>> purged has been closed. Otherwise, there is the potential for
>> an ABBA deadlock: another program might hold a shared lock on
>> the parent, and be waiting to get an exclusive lock on the child.
>>
>> 14. Delete the being-purged cgroup. Treat EBUSY and ENOENT as success:
>> the first means that a concurrently-running program re-created the
>> cgroup, while the second means that a concurrently-running program
>> deleted it. The name of the cgroup being purged is currently at
>> the top of the path stack.
>>
>> 15. Pop the name of the cgroup being purged off of the stack.
>>
>> At the end, self is in the same state it was before the operation.
>>
>> If you are complaining that this is about as readable as Forth,
>> then I agree with you :).
>
> Could this be made clearer by not mutating self, and either just storing
> the child stuff in local variables or another Cgroup object? I think a
> big part of the confusion here is that this function temporarily changes
> which cgroup the Cgroup object it's called on refers to while it's
> running. That's extremely difficult to reason about.
That makes sense. I'll try to move stuff into helper objects. You're
correct that the
>>>> + Err(e) => return Err(format!("Cannot purge {:?}: {e}", self.joined_path(path))),
>>>> + }
>>>> + if !self.open_sub_cgroup(path, true, true)? {
>>>> + return Ok(());
>>>> + }
>>>> +
>>>> + rustix::fs::flock(
>>>> + self.fd[self.fd.len() - 2].0.as_fd(),
>>>> + FlockOperation::LockShared,
>>>
>>> We already must have at least a shared lock on this at this point, no?
>>> I don't think we need another one.
>>
>> We actually have an exclusive lock. If it succeeds,
>> Cgroup::open_sub_cgroup() pushes a file descriptor onto self.fd.
>> Therefore, the fd being locked here is the one that was initially on the
>> top of the stack. We assert that an exclusive lock is held on that FD.
>>
>> Waiting for the control group to become empty is a blocking operation,
>> so this downgrades the lock to a shared one. Otherwise, an in-progress
>> purge of /a/b would prevent /a/c from being created.
>
> Ah, didn't realise it would downgrade. Makes sense.
>>>> +fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> {
>>>> + if remaining_depth < 1 {
>>>> + panic!("control groups too deeply nested");
>>>> + }
>>>> + let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating");
>>>> + while let Some(element) = d.next() {
>>>> + let element = element.expect("Iterating through a cgroup directory failed?");
>>>> + if element.file_type() != rustix::fs::FileType::Directory {
>>>> + continue;
>>>> + }
>>>> +
>>>> + let remaining_depth = remaining_depth - 1;
>>>> + let d: &rustix::fs::Dir = &d;
>>>> + let dirfd = d.fd().unwrap();
>>>> + let path = element.file_name();
>>>> + remove_all(remaining_depth, dirfd, path)?;
>
> I'd inline these variables into the function call the extent possible.
> Otherwise I have to follow a lot of shuffling around.
Will fix in v5.
>>>> + }
>>>> + drop(d);
>>>> + Ok(())
>>>> +}
>>>> +
>>>> +fn remove_all(
>>>> + remaining_depth: usize,
>>>> + dirfd: BorrowedFd<'_>,
>>>> + path: &std::ffi::CStr,
>>>> +) -> Result<(), Errno> {
>>>> + if path == c"." || path == c".." {
>>>> + return Ok(());
>>>> + }
>
> It's a bit weird that calling remove_all on . or .. does not fail.
> Maybe would be clearer to move this check to the call site?
Will fix in v5, making the code simpler.
>>>> + if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() {
>>>> + return Ok(());
>>>> + }
>
> We could drop this, right? A few extra syscalls, but less to wrap my
> head around.
Correct.
>>>> + let fd = rustix::fs::openat2(
>>>> + dirfd,
>>>> + path,
>>>> + OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY,
>>>> + Mode::empty(),
>>>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>>>> + )?;
>>>> + remove_recursively(fd, remaining_depth)?;
>>>> + rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?;
>>>> + Ok(())
>>>> +}
>>>
>>> Could we save a lot of code by calling std::fs::remove_dir_all with a
>>> /proc/self/fd path? It's already documented to ignore symlinks.
>>
>> I tried, but that tries to delete files too, and that isn't allowed.
>
> Ah, alright. That could use an explanatory comment.
>
> How can we be confident we have enough stack for this? Is there a way
> it could be done non-recursively, with state on the heap?
In Spectrum, the depth is currently limited to a small constant.
>>>> + let status = parse_digit_string(&args.next().unwrap(), "exit status")?;
>>>> + let signal = args.next().unwrap();
>>>> + let signal = if status == 256 {
>>>> + Some(parse_digit_string(&signal, "signal number")?)
>>>> + } else {
>>>> + None
>>>> + };
>>>> + let service = args.next().unwrap();
>>>> +
>>>> + let (path, mut cgroup) = open_relative_cgroup(service)?;
>>>> + let cgroup_target = Path::new(path.file_name().unwrap());
>>>> + let exit_125 = if let Some(signal) = signal {
>>>> + match signal as libc::c_int {
>>>> + libc::SIGBUS
>>>> + | libc::SIGFPE
>>>> + | libc::SIGABRT
>>>> + | libc::SIGTRAP
>>>> + | libc::SIGSEGV
>>>> + | libc::SIGILL => {
>>>> + // Process *crashed*, indicating a *possible exploit attempt*.
>>>> + // s6 should *not* restart it. This is distinct from a Rust panic,
>>>> + // which is much less likely to indicate memory corruption.
>>>> + true
>>>> + }
>>>
>>> This has absolutely nothing to do with cgroups. If you want to have
>>> some common finish behaviour, a program called cgroup-setup is not the
>>> place for it. I don't think there's any need for a separate
>>> cgroup-s6-finish mode (as opposed to cgroup-purge).
>>
>> This program is a multi-call binary, so the various things it can do
>> aren't necessarily super tightly related. For instance, all of the
>> execline binaries can be built as one program, as can most if not
>> all busybox applets. When invoked as cgroup-setup or cgroup-purge,
>> it indeed only does cgroup-related tasks. cgroup-s6-finish not
>> only handles cgroups, but also other tasks related to being an s6
>> finish script.
>>
>> That said, using this changes behavior in a way that isn't related
>> to cgroups, so if it is to be used at all it should be in a separate
>> patch series. I'll remove this from v5.
>
> Thank you. If we want to have a big multi-call binary that does lots of
> different things, only some of which are cgroup-specific, cgroup-setup
> is not the name for that program.
No argument there!
>>>> +fn local_cgroup() -> Result<PathBuf, String> {
>>>> + let mut local_cgroup: Vec<u8> = std::fs::read("/proc/thread-self/cgroup")
>>>> + .map_err(|e| format!("cannot read /proc/thread-self/cgroup: {e}"))?;
>>>> + let local_cgroup_len = local_cgroup.len();
>>>> + if local_cgroup_len < 5
>>>> + || local_cgroup[..4] != *b"0::/"
>>>> + || local_cgroup[local_cgroup_len - 1] != b'\n'
>>>> + || local_cgroup[4..local_cgroup_len - 1].contains(&b'\n')
>>>
>>> Last time I suggested a clearer way of doing this, but it has instead
>>> got even less clear.
>>>
>>> (I'm not sure why we'd care if there's a newline specifically, as
>>> opposed to any other control character.)
>>
>> If cgroups v1 is in use, the file can contain multiple lines, one for
>> each cgroup the program is in. I also am not sure if starting with
>> "0::/" is an invariant in that case.
>>
>> Using this program with cgroups v1 mounted is user error and will
>> never happen on Spectrum, but if this tool is used outside of Spectrum,
>> it could happen.
>
> I see. It seems like with cgroups v1, it _could_ start with 0::/, but
> probably wouldn't. I think it may not be possible to tell from this
> file whether cgroups v1 is in use.
>
> So I suppose it depends what you want to happen if cgroups v1 is in use.
> If it looks enough like cgroups v2, do you continue, or do you
> explicitly check for cgroups v1? If the latter (sounds more sensible to
> me), you need to explicitly check for cgroups v1 somehow I think. Can
> cgroups v1 and v2 be in use at the same time? If so, checking might be
> complicated, but if not, you can just check what type of filesystem is
> mounted at /sys/fs/cgroup, or see if it has a
> /sys/fs/cgroup/cgroup.controllers file.
In the case of Spectrum, I think it's okay to just panic if cgroups v2
isn't mounted or isn't working properly. It's a bug in either
cgroup-setup or the kernel, almost certainly the former.
>>>> + {
>>>> + // It's possible to get here if the cgroup path contains a newline,
>>>> + // but that never happens in Spectrum.
>>>> + return Err(format!(
>>>> + "Invalid contents {local_cgroup:?} of /proc/thread-self/cgroup - \
>>>> + do you have cgroups v1 mounted instead of cgroups v2?"
>>>> + ));
>>>> + }
>>>> +
>>>> + local_cgroup.copy_within(4..local_cgroup_len - 1, 0);
>>>> + local_cgroup.truncate(local_cgroup_len - 5);
>>>> + let local_cgroup = OsString::from_vec(local_cgroup);
>>>> + check_path(&local_cgroup).unwrap();
>>>
>>> Why do we need to do this? You're worried the kernel is going to start
>>> including .. components in /proc/thread-self/cgroup?
>>
>> Originally, I was going to create a wrapper around `Path` that
>> guaranteed no `.` or `..` components were present. Its constructor
>> would have checked this invariant. However, this turned out to be
>> more work due to the amount of wrapper functions required.
>
> Wise not to proceed with that, I think. But I don't think we need this
> particular instance of the check, given it comes from the kernel.
Dropped in v5.
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 02/20] tools: Add control group manager
2026-07-30 0:40 ` Demi Marie Obenour
@ 2026-07-30 14:53 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-07-30 14:53 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 4859 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> On 7/27/26 08:10, Alyssa Ross wrote:
>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>
>>> On 7/22/26 12:01, Alyssa Ross wrote:
>>>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>
>>>>> +fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> {
>>>>> + if remaining_depth < 1 {
>>>>> + panic!("control groups too deeply nested");
>>>>> + }
>>>>> + let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating");
>>>>> + while let Some(element) = d.next() {
>>>>> + let element = element.expect("Iterating through a cgroup directory failed?");
>>>>> + if element.file_type() != rustix::fs::FileType::Directory {
>>>>> + continue;
>>>>> + }
>>>>> +
>>>>> + let remaining_depth = remaining_depth - 1;
>>>>> + let d: &rustix::fs::Dir = &d;
>>>>> + let dirfd = d.fd().unwrap();
>>>>> + let path = element.file_name();
>>>>> + remove_all(remaining_depth, dirfd, path)?;
>>
>> I'd inline these variables into the function call the extent possible.
>> Otherwise I have to follow a lot of shuffling around.
>
> Will fix in v5.
>
>>>>> + }
>>>>> + drop(d);
>>>>> + Ok(())
>>>>> +}
>>>>> +
>>>>> +fn remove_all(
>>>>> + remaining_depth: usize,
>>>>> + dirfd: BorrowedFd<'_>,
>>>>> + path: &std::ffi::CStr,
>>>>> +) -> Result<(), Errno> {
>>>>> + if path == c"." || path == c".." {
>>>>> + return Ok(());
>>>>> + }
>>
>> It's a bit weird that calling remove_all on . or .. does not fail.
>> Maybe would be clearer to move this check to the call site?
>
> Will fix in v5, making the code simpler.
>
>>>>> + if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() {
>>>>> + return Ok(());
>>>>> + }
>>
>> We could drop this, right? A few extra syscalls, but less to wrap my
>> head around.
>
> Correct.
>
>>>>> + let fd = rustix::fs::openat2(
>>>>> + dirfd,
>>>>> + path,
>>>>> + OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY,
>>>>> + Mode::empty(),
>>>>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV,
>>>>> + )?;
>>>>> + remove_recursively(fd, remaining_depth)?;
>>>>> + rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?;
>>>>> + Ok(())
>>>>> +}
>>>>
>>>> Could we save a lot of code by calling std::fs::remove_dir_all with a
>>>> /proc/self/fd path? It's already documented to ignore symlinks.
>>>
>>> I tried, but that tries to delete files too, and that isn't allowed.
>>
>> Ah, alright. That could use an explanatory comment.
>>
>> How can we be confident we have enough stack for this? Is there a way
>> it could be done non-recursively, with state on the heap?
>
> In Spectrum, the depth is currently limited to a small constant.
Right, but that's really papering over the problem, and our stack is
also limited to a small constant size. We could avoid the problem
entirely by instead running paths to operate on through a VecDeque
rather than recursing, and then we wouldn't need any arbitrary limit.
>>>> Last time I suggested a clearer way of doing this, but it has instead
>>>> got even less clear.
>>>>
>>>> (I'm not sure why we'd care if there's a newline specifically, as
>>>> opposed to any other control character.)
>>>
>>> If cgroups v1 is in use, the file can contain multiple lines, one for
>>> each cgroup the program is in. I also am not sure if starting with
>>> "0::/" is an invariant in that case.
>>>
>>> Using this program with cgroups v1 mounted is user error and will
>>> never happen on Spectrum, but if this tool is used outside of Spectrum,
>>> it could happen.
>>
>> I see. It seems like with cgroups v1, it _could_ start with 0::/, but
>> probably wouldn't. I think it may not be possible to tell from this
>> file whether cgroups v1 is in use.
>>
>> So I suppose it depends what you want to happen if cgroups v1 is in use.
>> If it looks enough like cgroups v2, do you continue, or do you
>> explicitly check for cgroups v1? If the latter (sounds more sensible to
>> me), you need to explicitly check for cgroups v1 somehow I think. Can
>> cgroups v1 and v2 be in use at the same time? If so, checking might be
>> complicated, but if not, you can just check what type of filesystem is
>> mounted at /sys/fs/cgroup, or see if it has a
>> /sys/fs/cgroup/cgroup.controllers file.
>
> In the case of Spectrum, I think it's okay to just panic if cgroups v2
> isn't mounted or isn't working properly. It's a bug in either
> cgroup-setup or the kernel, almost certainly the former.
Yeah I agree it's completely fine for new stuff to requrie cgroups v2.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v4 18/20] host/rootfs: systemd-udevd: Run in cgroup
2026-07-29 20:39 ` Demi Marie Obenour
@ 2026-07-30 14:55 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-07-30 14:55 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 3831 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> On 7/29/26 10:15, Alyssa Ross wrote:
>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>
>>> On 7/27/26 08:20, Alyssa Ross wrote:
>>>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>>>
>>>>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>>>>> ---
>>>>> host/rootfs/file-list.mk | 1 +
>>>>> host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +++++
>>>>> host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +++--
>>>>> 3 files changed, 9 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
>>>>> index e3411d40014342e67071a48f9e1c7bc7b3954bcf..065571ef3a5c61fa79a1d2ed878052a5f391be96 100644
>>>>> --- a/host/rootfs/file-list.mk
>>>>> +++ b/host/rootfs/file-list.mk
>>>>> @@ -106,6 +106,7 @@ S6_RC_FILES = \
>>>>> image/etc/s6-rc/systemd-udevd-coldplug/dependencies.d/systemd-udevd \
>>>>> image/etc/s6-rc/systemd-udevd-coldplug/type \
>>>>> image/etc/s6-rc/systemd-udevd-coldplug/up \
>>>>> + image/etc/s6-rc/systemd-udevd/finish \
>>>>> image/etc/s6-rc/systemd-udevd/notification-fd \
>>>>> image/etc/s6-rc/systemd-udevd/run \
>>>>> image/etc/s6-rc/systemd-udevd/type \
>>>>> diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/finish b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
>>>>> new file mode 100755
>>>>> index 0000000000000000000000000000000000000000..4d5b454e97c3584d0644c2d738514f13b1c53957
>>>>> --- /dev/null
>>>>> +++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
>>>>> @@ -0,0 +1,5 @@
>>>>> +#!/usr/bin/execlineb -WS3
>>>>> +# SPDX-License-Identifier: EUPL-1.2+
>>>>> +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>>>>> +
>>>>> +cgroup-s6-finish $@
>>>>> diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/run b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
>>>>> old mode 100644
>>>>> new mode 100755
>>>>> index aec6444e951503eae988e666b77fda8f2ae33d72..901f3e6667a8de04a6af39945406a1757455f089
>>>>> --- a/host/rootfs/image/etc/s6-rc/systemd-udevd/run
>>>>> +++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
>>>>> @@ -1,7 +1,8 @@
>>>>> -#!/bin/execlineb -WP
>>>>> +#!/bin/execlineb -WS1
>>>>> # SPDX-License-Identifier: EUPL-1.2+
>>>>> # SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
>>>>> -s6-setlock /run/sd-notify-wrapper/systemd-udevd.lock
>>>>> +
>>>>> +cgroup-setup --delegate --child-name udev -- $1
>>>>> s6-ipcserver-socketbinder -b0 -m -a 0600 /run/sd-notify-wrapper/systemd-udevd.sock
>>>>> background -d {
>>>>> fdmove 1 3
>>>>>
>>>>
>>>> Is there a reason not to always enable delegation? Is there a reason we
>>>> need to set the name here?
>>>
>>> --delegate only sets the user.delegate=1 xattr used by systemd.
>>> Programs that are systemd-aware and manage their own cgroups check this
>>> xattr to see if a cgroup has been delegated by systemd. The kernel
>>> and other programs do not care.
>>
>> Right, but why can't we just always set user.delegate=1? Would we ever
>> have a program that checked it that we didn't want to manage its own
>> cgroups?
>
> It's slightly wasteful (extra syscall) but harmless.
>
>>> systemd-udevd.service uses the "udev" name, so I decided to also use it
>>> for consistency.
>>
>> Is this the only reason we even need a --child-name feature? If so, it
>> doesn't seem well justified to me unless it actually affects something.
>
> I'd prefer to stick with what systemd uses, as that is what is tested.
> Varying from what systemd uses increases the risk of regressions.
I'd be very surprised if the name of the cgroup is a practical
regression risk. I don't think it justifies extra complexity in
cgroup-setup.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* [PATCH v5 00/19] Control group support
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
` (19 preceding siblings ...)
2026-07-22 1:59 ` [PATCH v4 20/20] host/rootfs: spectrum-router: " Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 01/19] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
` (19 more replies)
20 siblings, 20 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Changes in v5:
- Rename open_subtree_raw() to open_beneath().
- Use consistent file modes.
- Avoid using O_NOFOLLOW when RESOLVE_NO_SYMLINKS is also used.
- Drop tracking of specific cgroup paths (at the cost of worse error messages).
- Have functions take &dyn AsFd where that makes sense.
- Drop exclusive vs shared lock tracking.
- Remove vm-service-run wrapper script.
- Use explicit cgroup names in VM service run and finish scripts.
- Use sed to write to cgroup.subtree_control in the root cgroup,
avoiding a special case in cgroup-setup.
- Avoid mutating the Cgroup struct when creating child cgroups.
- Avoid mutating the Cgroup struct when purging cgroups.
- Improve documentation.
- Drop cgroup-s6-finish and call cgroup-purge directly.
- Remove support for operating on . or / in cgroup-setup.
- Fix comments.
- Link to v4: https://spectrum-os.org/lists/archives/spectrum-devel/20260721-cgroups-v4-0-46b2e5fff7b6@gmail.com
Changes in v4:
- Implement proper locking to make concurrent operations safe.
- Purge VMM cgroup in vmm service finish script.
- Delete /run/vsock/${VM}/vsock before running Cloud Hypervisor.
- Massively refactor cgroup-setup tool.
- Link to v3: https://spectrum-os.org/lists/archives/spectrum-devel/20260711-cgroups-v3-1-5cba61a20cba@gmail.com
Changes in v3:
- Remove the implicit .service suffix on leaf control groups.
- Make cgroup-setup acts as an s6 finish script when called as "finish".
- Put the VMMs in the same cgroups as the per-VM services.
- Add common helper script for the per-VM services.
- Significantly refactor the Rust code.
- Link to v2: https://spectrum-os.org/lists/archives/spectrum-devel/20260620-cgroups-v2-1-ccae224b6c85@gmail.com
Changes in v2:
- Omit resource control support. It was completely broken, and the way
Spectrum run scripts work means that it is better to setup resource
controls just before the execve() into the final service process.
This will be done by a separate tool.
- Link to v1: https://spectrum-os.org/lists/archives/spectrum-devel/20260620-cgroups-v1-1-0e5abf35101b@gmail.com
---
Demi Marie Obenour (19):
host/rootfs: Mount filesystems before s6-rc-init
tools: Add control group manager
Documentation: Mention control groups
Mount cgroup2 filesystem at /sys/fs/cgroup
host/rootfs: Enable controllers in non-root cgroups
host/rootfs: Add comments where cgroups are intentionally not used
host/rootfs: serial-getty-generator: Use cgroups
host/rootfs: systemd-udevd: Run in cgroup
host/rootfs: weston: Run in cgroup
host/rootfs: Set up parent cgroup for all per-VM services
host/rootfs: Create per-VM cgroup for all of the VM's services
host/rootfs: run-vmm: Create per-VM cgroup
host/rootfs: run-appimage: Purge the per-VM cgroup
host/rootfs: run-flatpak: Purge the per-VM cgroup
host/rootfs: dbus: Run in cgroup
host/rootfs: vhost-user-fs: Run in cgroup
host/rootfs: vhost-user-gpu: Run in cgroup
host/rootfs: xdg-desktop-portal-spectrum-host: Run in cgroup
host/rootfs: spectrum-router: Run in cgroup
.codespellrc | 2 +-
Documentation/doc/development/control-groups.adoc | 95 +++++++
host/rootfs/default.nix | 6 +-
host/rootfs/file-list.mk | 10 +
host/rootfs/image/etc/fstab | 1 +
host/rootfs/image/etc/init | 14 +-
.../s6-linux-init/run-image/service/getty-tty2/run | 1 +
.../s6-linux-init/run-image/service/getty-tty3/run | 1 +
.../s6-linux-init/run-image/service/getty-tty4/run | 1 +
.../run-image/service/root-terminal/run | 1 +
.../run-image/service/s6-linux-init-shutdownd/run | 1 +
.../run-image/service/s6-svscan-log/run | 1 +
.../service/serial-getty-generator/finish | 5 +
.../run-image/service/serial-getty-generator/run | 4 +-
.../run-image/service/serial-getty/run | 1 +
.../run-image/service/serial-getty/template/run | 1 +
.../run-image/service/vm-services/run | 3 +
.../vm-services/template/data/service/dbus/finish | 5 +
.../vm-services/template/data/service/dbus/run | 2 +
.../template/data/service/spectrum-router/finish | 5 +
.../template/data/service/spectrum-router/run | 2 +
.../template/data/service/vhost-user-fs/finish | 5 +
.../template/data/service/vhost-user-fs/run | 5 +-
.../template/data/service/vhost-user-gpu/finish | 5 +
.../template/data/service/vhost-user-gpu/run | 2 +
.../xdg-desktop-portal-spectrum-host/finish | 5 +
.../service/xdg-desktop-portal-spectrum-host/run | 2 +
.../run-image/service/vm-services/template/finish | 5 +
.../run-image/service/vm-services/template/run | 4 +
.../run-image/service/vmm/template/finish | 5 +
.../rootfs/image/etc/s6-linux-init/scripts/rc.init | 7 -
host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +
host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +-
host/rootfs/image/etc/s6-rc/weston/finish | 5 +
host/rootfs/image/etc/s6-rc/weston/run | 2 +
host/rootfs/image/usr/bin/run-appimage | 7 +-
host/rootfs/image/usr/bin/run-flatpak | 7 +-
host/rootfs/image/usr/bin/run-vmm | 4 +
host/rootfs/image/usr/bin/vm-stop | 5 +-
pkgs/default.nix | 1 +
tools/cgroup-setup/Cargo.lock | 67 +++++
tools/cgroup-setup/Cargo.lock.license | 2 +
tools/cgroup-setup/Cargo.toml | 10 +
tools/cgroup-setup/default.nix | 22 ++
tools/cgroup-setup/src/cgroup.rs | 308 +++++++++++++++++++++
tools/cgroup-setup/src/main.rs | 186 +++++++++++++
46 files changed, 823 insertions(+), 20 deletions(-)
---
base-commit: 7e91788b3d630a4f4be8b318c9a3a436c37d39ec
change-id: 20260528-cgroups-d609e270e649
--
Sincerely,
Demi Marie Obenour (she/her/hers)
^ permalink raw reply [flat|nested] 132+ messages in thread
* [PATCH v5 01/19] host/rootfs: Mount filesystems before s6-rc-init
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 02/19] tools: Add control group manager Demi Marie Obenour
` (18 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
cgroup-setup, a tool which will be introduced in a later patch, is
written in Rust and uses the Rust standard library. It also relies on
cgroupfs being mounted at /sys/fs/cgroup. Furthermore, it runs very,
*very* early on, since it sets up the cgroups in which s6-svscan and
s6-supervise processes run. This allows setting up sub-cgroups without
having to worry about name collisions.
While it would be possible to only mount the bare minimum necessary,
it's simpler to just mount everything before starting any services.
These mounts are all pseudo-filesystems or tmpfs instances, and none
have any dependencies.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/image/etc/init | 11 ++++++++++-
host/rootfs/image/etc/s6-linux-init/scripts/rc.init | 7 -------
2 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
index db8a6d9c747e3d212c21b6c983c8f7e299822abe..8ec7859ab00e24007a93d4000c8f34e353de50ae 100755
--- a/host/rootfs/image/etc/init
+++ b/host/rootfs/image/etc/init
@@ -1,6 +1,15 @@
#!/bin/execlineb -Ws0
# SPDX-License-Identifier: EUPL-1.2+
-# SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2020-2022, 2024 Alyssa Ross <hi@alyssa.is>
/bin/setpriv --no-new-privs --
+
+if { /bin/ln -s /proc/self/fd /dev }
+if { /bin/ln -s /proc/self/fd/0 /dev/stdin }
+if { /bin/ln -s /proc/self/fd/1 /dev/stdout }
+if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
+
+if { /bin/mount --make-shared / }
+if { /bin/mount -a --mkdir }
+
/bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
diff --git a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
index f638e373589884acb959c868462fdf532380a851..db766ba0ca0a11982f5b55497c17eaf5e95afb2c 100755
--- a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
+++ b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
@@ -4,13 +4,6 @@
if { s6-rc-init -c /etc/s6-rc /run/service }
-if { ln -s /proc/self/fd /dev }
-if { ln -s /proc/self/fd/0 /dev/stdin }
-if { ln -s /proc/self/fd/1 /dev/stdout }
-if { ln -s /proc/self/fd/2 /dev/stderr }
-
-if { mount --make-shared / }
if { mount --make-shared /run }
-if { mount -a --mkdir }
s6-rc change ok-all
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 02/19] tools: Add control group manager
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 01/19] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-08-03 12:47 ` Alyssa Ross
2026-07-31 21:54 ` [PATCH v5 03/19] Documentation: Mention control groups Demi Marie Obenour
` (17 subsequent siblings)
19 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
The cgroup-setup Rust program can create and purge cgroups. It can also
wait for one to become empty, spawn a program in a cgroup, and more. In
the future, it will also support cgroup-based resource control. Locking
is used to ensure that concurrent invocations are safe.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
.codespellrc | 2 +-
host/rootfs/default.nix | 6 +-
pkgs/default.nix | 1 +
tools/cgroup-setup/Cargo.lock | 67 ++++++++
tools/cgroup-setup/Cargo.lock.license | 2 +
tools/cgroup-setup/Cargo.toml | 10 ++
tools/cgroup-setup/default.nix | 22 +++
tools/cgroup-setup/src/cgroup.rs | 308 ++++++++++++++++++++++++++++++++++
tools/cgroup-setup/src/main.rs | 186 ++++++++++++++++++++
9 files changed, 600 insertions(+), 4 deletions(-)
diff --git a/.codespellrc b/.codespellrc
index d8023afc64ec44e98a88c5e397c8d5c681dde063..ae20da8309530759ac729689825a4afc683afa09 100644
--- a/.codespellrc
+++ b/.codespellrc
@@ -2,4 +2,4 @@
# SPDX-License-Identifier: CC0-1.0
[codespell]
-ignore-words-list = crate,passt,rouge,ser
+ignore-words-list = crate,passt,rouge,ser,WRONLY
diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix
index 6bfeefbe0a5f76c1538ccb40e5eb8f291f5d3592..ccf626e2ec0f4bf96573dc5edf058c9375bb65f6 100644
--- a/host/rootfs/default.nix
+++ b/host/rootfs/default.nix
@@ -8,7 +8,7 @@ import ../../lib/call-package.nix (
}:
pkgsMusl.callPackage (
-{ spectrum-host-tools, spectrum-router
+{ spectrum-host-tools, spectrum-router, spectrum-cgroup-setup
, lib, stdenvNoCC, nixos, runCommand, writeClosure, erofs-utils, s6-rc
, btrfs-progs, bubblewrap, busybox, cloud-hypervisor, cosmic-files
, crosvm, cryptsetup, dejavu_fonts, dbus, execline, foot, fuse3
@@ -27,8 +27,8 @@ let
packages = [
btrfs-progs bubblewrap cloud-hypervisor cosmic-files crosvm cryptsetup dbus
execline fuse3 inotify-tools iproute2 jq kmod mdevd mount-flatpak s6
- s6-linux-init s6-rc shadow socat spectrum-host-tools spectrum-router
- virtiofsd xdg-desktop-portal-spectrum-host
+ s6-linux-init s6-rc shadow socat spectrum-cgroup-setup spectrum-host-tools
+ spectrum-router virtiofsd xdg-desktop-portal-spectrum-host
(foot.override { allowPgo = false; })
diff --git a/pkgs/default.nix b/pkgs/default.nix
index 44f7b5ff78cb6b9e755292a6a417d0b627ed3fb0..0a13393164ad5d7f752e630763f3f97166479af5 100644
--- a/pkgs/default.nix
+++ b/pkgs/default.nix
@@ -51,6 +51,7 @@ let
driverSupport = true;
};
spectrum-router = self.callSpectrumPackage ../tools/router {};
+ spectrum-cgroup-setup = self.callSpectrumPackage ../tools/cgroup-setup {};
xdg-desktop-portal-spectrum-host =
self.callSpectrumPackage ../tools/xdg-desktop-portal-spectrum-host {};
diff --git a/tools/cgroup-setup/Cargo.lock b/tools/cgroup-setup/Cargo.lock
new file mode 100644
index 0000000000000000000000000000000000000000..fe967b3aa02c296c87b6b36ac59253dbe0a32de9
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.lock
@@ -0,0 +1,67 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 4
+
+[[package]]
+name = "bitflags"
+version = "2.11.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3"
+
+[[package]]
+name = "cgroup-setup"
+version = "0.0.0"
+dependencies = [
+ "libc",
+ "rustix",
+]
+
+[[package]]
+name = "errno"
+version = "0.3.14"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
+dependencies = [
+ "libc",
+ "windows-sys",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.186"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
+
+[[package]]
+name = "linux-raw-sys"
+version = "0.12.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
+
+[[package]]
+name = "rustix"
+version = "1.1.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
+dependencies = [
+ "bitflags",
+ "errno",
+ "libc",
+ "linux-raw-sys",
+ "windows-sys",
+]
+
+[[package]]
+name = "windows-link"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
+
+[[package]]
+name = "windows-sys"
+version = "0.61.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
+dependencies = [
+ "windows-link",
+]
diff --git a/tools/cgroup-setup/Cargo.lock.license b/tools/cgroup-setup/Cargo.lock.license
new file mode 100644
index 0000000000000000000000000000000000000000..62497aa72873adb3e62fb4f186bd19b579b36734
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.lock.license
@@ -0,0 +1,2 @@
+SPDX-License-Identifier: CC0-1.0
+SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
diff --git a/tools/cgroup-setup/Cargo.toml b/tools/cgroup-setup/Cargo.toml
new file mode 100644
index 0000000000000000000000000000000000000000..bba3f55bcc6dda049f2808efd11c397131742500
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.toml
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: CC0-1.0
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+[package]
+name = "cgroup-setup"
+edition = "2024"
+
+[dependencies]
+libc = "0.2.177"
+rustix = { version = "1.1.2", features = ["fs"] }
diff --git a/tools/cgroup-setup/default.nix b/tools/cgroup-setup/default.nix
new file mode 100644
index 0000000000000000000000000000000000000000..fe3a8bcd4e6118f3cbf49c2fc0e3cda01570c484
--- /dev/null
+++ b/tools/cgroup-setup/default.nix
@@ -0,0 +1,22 @@
+# SPDX-FileCopyrightText: 2024 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
+# SPDX-License-Identifier: MIT
+
+import ../../lib/call-package.nix (
+{ src, lib, rustPlatform }:
+
+rustPlatform.buildRustPackage {
+ name = "spectrum-cgroup-setup";
+
+ src = lib.fileset.toSource {
+ root = ../..;
+ fileset = lib.fileset.intersection src ./.;
+ };
+ sourceRoot = "source/tools/cgroup-setup";
+
+ cargoLock.lockFile = ./Cargo.lock;
+
+ postInstall = ''
+ ln -s -- cgroup-setup "$out/bin/cgroup-purge"
+ '';
+}) (_: {})
diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs
new file mode 100644
index 0000000000000000000000000000000000000000..4ea89b5787d379e9d0c40405810ce5701167c639
--- /dev/null
+++ b/tools/cgroup-setup/src/cgroup.rs
@@ -0,0 +1,308 @@
+// SPDX-License-Identifier: EUPL-1.2+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+use std::cell::RefCell;
+use std::ffi::OsStr;
+use std::fs::File;
+use std::io::{Read as _, Seek as _, Write as _};
+use std::os::unix::prelude::*;
+
+use std::path::{Component, Path, PathBuf};
+use std::rc::Rc;
+
+use rustix::fs::{AtFlags, CWD, Dir, FlockOperation};
+use rustix::{
+ fs::{Mode, OFlags, ResolveFlags},
+ io::Errno,
+};
+
+#[derive(Debug)]
+pub(crate) struct Cgroup {
+ fd: Vec<OwnedFd>,
+}
+
+impl AsFd for Cgroup {
+ fn as_fd(&self) -> BorrowedFd<'_> {
+ self.fd.last().unwrap().as_fd()
+ }
+}
+
+fn assert_single_component(component: &Path) {
+ match component.as_os_str().as_bytes() {
+ b"" | b"." | b".." => panic!("bad component"),
+ c if c.contains(&b'\0') => panic!("NUL in component"),
+ c if c.contains(&b'/') => panic!("/ in component"),
+ _ => {}
+ }
+}
+
+// Wrapper around openat2() with better defaults.
+pub fn openat2_simple(fd: &dyn AsFd, path: &Path, flags: OFlags) -> Result<OwnedFd, Errno> {
+ rustix::fs::openat2(
+ fd.as_fd(),
+ path,
+ OFlags::CLOEXEC | OFlags::NOCTTY | flags,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_XDEV,
+ )
+}
+
+pub const DEFAULT_LEAF: &str = "$inner.service";
+
+pub fn check_path(path: &Path) -> Result<(), String> {
+ let bytes = path.as_os_str().as_bytes();
+ // Path::components() skips ., so use string manipulation instead.
+ for component in bytes[path.is_absolute() as usize..].split(|&b| b == b'/') {
+ if matches!(component, b"" | b"." | b"..") {
+ return Err(format!("cgroup path {path:?} isn't canonical"));
+ }
+ }
+ Ok(())
+}
+
+fn push_child_fds(fds: &mut Vec<(Rc<RefCell<Dir>>, PathBuf)>, fd: OwnedFd) {
+ // The rustix source code shows that Dir::new() never fails.
+ let child_fd = Rc::new(RefCell::new(Dir::new(fd).unwrap()));
+ while let Some(element) = child_fd.borrow_mut().next() {
+ let element = element.expect("Iterating through a cgroup directory failed?");
+ if element.file_type() != rustix::fs::FileType::Directory {
+ continue;
+ }
+ match element.file_name().to_bytes() {
+ b"." | b".." => {}
+ other => {
+ let other = Path::new(OsStr::from_bytes(other)).to_owned();
+ assert_single_component(&other);
+ fds.push((child_fd.clone(), other));
+ }
+ }
+ }
+}
+
+// Remove all subdirectories of the given directory recursively,
+// but not the directory itself. The directory file descriptor
+// is closed.
+//
+// This isn't the most efficient possible algorithm, but
+// simplicity is more important than performance in this
+// case. Also, it keeps open more file descriptors than
+// strictly necessary, but Spectrum runs with a very high
+// limit for the number of open file descriptors, and it
+// uses shallow control group hierarchies.
+fn remove_child_directories(dirfd: OwnedFd) -> Result<(), Errno> {
+ let mut fds = Vec::new();
+ // Push the children of this directory onto the stack.
+ push_child_fds(&mut fds, dirfd);
+ while let Some((d, path)) = fds.pop() {
+ assert_single_component(&path);
+ // Try to delete the directory. If that fails because there are child
+ // directories, push the child directories onto the stack, then push
+ // this directory again.
+ match rustix::fs::unlinkat(
+ // The rustix source code shows that Dir::fd() never fails.
+ d.borrow().fd().unwrap(),
+ &path,
+ AtFlags::REMOVEDIR,
+ ) {
+ Err(Errno::NOTEMPTY) => {}
+ Ok(()) => continue,
+ Err(bad) => return Err(bad),
+ }
+ let fd = openat2_simple(
+ // The rustix source code shows that Dir::fd() never fails.
+ &d.borrow().fd().unwrap(),
+ &path,
+ OFlags::DIRECTORY | OFlags::RDONLY,
+ )?;
+ // Process child directories first, then attempt to delete the
+ // directory again.
+ fds.push((d, path));
+ push_child_fds(&mut fds, fd);
+ }
+ Ok(())
+}
+
+// If the path is absolute, make it relative.
+// Otherwise, read the current cgroup from /proc/thread-self/cgroup
+// and prepend it to the path.
+fn prepend_current_cgroup_if_needed(path: &Path) -> PathBuf {
+ if let Ok(suffix) = path.strip_prefix("/") {
+ suffix.to_owned()
+ } else {
+ // /proc/thread-self is the same as /proc/self, except for the current
+ // thread instead of the initial thread. In this case, the two are
+ // identical, but using /proc/thread-self is better practice as it is
+ // correct in more cases. Reading /proc/thread-self/cgroup should
+ // never fail unless the system is seriously broken.
+ let current_cgroup = std::fs::read("/proc/thread-self/cgroup")
+ .expect("cannot read /proc/thread-self/cgroup");
+ // Using this on a system without cgroups v2 mounted is user error
+ // and not supported.
+ let current_cgroup = current_cgroup
+ .strip_prefix(b"0::/")
+ .and_then(|e| e.strip_suffix(b"\n"))
+ .expect("you don't have cgroups v2 mounted");
+ let mut current_cgroup = PathBuf::from(OsStr::from_bytes(current_cgroup));
+ // Strip the implied $inner.service suffix.
+ // This is used to satisfy the "no internal processes" rule.
+ if current_cgroup.ends_with(Path::new(DEFAULT_LEAF)) {
+ assert!(current_cgroup.pop());
+ }
+ current_cgroup.push(path);
+ current_cgroup
+ }
+}
+
+pub(crate) fn write_value(fd: &dyn AsFd, name: &Path, value: &[u8]) -> Result<(), String> {
+ let path = Path::new(name);
+ let fd = openat2_simple(fd, path, OFlags::WRONLY)
+ .map_err(|e| format!("Cannot open {name:?}: {e}"))?;
+ File::from(fd).write_all(value).map_err(|e| {
+ format!(
+ "Cannot write {:?} to {name:?}: {e}",
+ OsStr::from_bytes(value)
+ )
+ })
+}
+
+impl Cgroup {
+ pub fn open_beneath(&self, path: &Path, flags: OFlags) -> Result<OwnedFd, Errno> {
+ openat2_simple(self, path, flags)
+ }
+
+ pub fn new(path: &Path) -> Result<Self, String> {
+ let cgroup_root = rustix::fs::openat2(
+ CWD,
+ Path::new("/sys/fs/cgroup"),
+ OFlags::CLOEXEC | OFlags::DIRECTORY | OFlags::RDONLY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::NO_MAGICLINKS,
+ )
+ .map_err(|e| format!("Cannot open /sys/fs/cgroup: {e}"))?;
+ let mut cgroup = Self {
+ fd: vec![(cgroup_root)],
+ };
+
+ let path = prepend_current_cgroup_if_needed(path);
+ for component in path.components() {
+ let component = match component {
+ Component::Normal(component) => component,
+ _ => unreachable!(),
+ };
+ let sub_fd = cgroup
+ .open_beneath(Path::new(component), OFlags::RDONLY | OFlags::DIRECTORY)
+ .map_err(|e| format!("Cannot open sub-cgroup {component:?}: {e}"))?;
+ // Take a shared lock on the *previous* file descriptor.
+ rustix::fs::flock(&cgroup, FlockOperation::LockShared)
+ .map_err(|e| format!("Cannot lock sub-cgroup {component:?}: {e}"))?;
+ cgroup.fd.push(sub_fd);
+ }
+ // Take an exclusive lock on the final file descriptor.
+ rustix::fs::flock(&cgroup, FlockOperation::LockExclusive)
+ .map_err(|e| format!("Cannot lock {path:?}: {e}"))?;
+ Ok(cgroup)
+ }
+
+ pub fn wait_for_empty(fd: &dyn AsFd) -> std::io::Result<()> {
+ let wait_file = openat2_simple(fd, Path::new("cgroup.events"), OFlags::RDONLY)?;
+ let poll_fd = wait_file.as_raw_fd();
+ let mut wait_fd = File::from(wait_file);
+ let mut fds = libc::pollfd {
+ fd: poll_fd,
+ events: libc::POLLPRI | libc::POLLERR,
+ revents: 0,
+ };
+ let mut v = vec![];
+ loop {
+ v.clear();
+ wait_fd
+ .seek(std::io::SeekFrom::Start(0))
+ .expect("Seek on control group file should succeed");
+ wait_fd
+ .read_to_end(&mut v)
+ .expect("reading from control group should work");
+ // Check that the cgroup isn't already empty. If it was,
+ // the kernel would not send an event and poll() would wait
+ // forever.
+ if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
+ break;
+ }
+ // SAFETY: FFI call, valid arguments, fds contains 1 element
+ if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
+ panic!("poll failed");
+ }
+ }
+ Ok(())
+ }
+
+ pub fn purge_child(&mut self, path: &Path) -> Result<(), String> {
+ assert_single_component(path);
+ // See if we can just delete the child directly.
+ match rustix::fs::unlinkat(&self, Path::new(path), AtFlags::REMOVEDIR) {
+ // If the cgroup was successfully deleted, or if it
+ // has already been deleted, we are done.
+ Ok(()) | Err(Errno::NOENT) => return Ok(()),
+ // If this cgroup is in use, keep going.
+ Err(Errno::BUSY) => {}
+ Err(e) => return Err(format!("Cannot purge {path:?}: {e}")),
+ }
+
+ let sub_fd = match self.open_beneath(path, OFlags::RDONLY | OFlags::DIRECTORY) {
+ Ok(sub_fd) => sub_fd,
+ Err(Errno::NOENT) => return Ok(()),
+ Err(e) => {
+ return Err(format!("Cannot open sub-cgroup {path:?}: {e}",));
+ }
+ };
+
+ // Take an exclusive lock on the cgroup that is about to be
+ // removed. This avoids concurrent executions of this program
+ // operating on deleted sub-cgroups.
+ rustix::fs::flock(&sub_fd, FlockOperation::LockExclusive)
+ .map_err(|e| format!("Cannot lock sub-cgroup: {e}"))?;
+
+ // Drop the exclusive lock on the original cgroup,
+ // This avoids blocking concurrent operations on other
+ // child cgroups while the cgroup is being purged,
+ // or while waiting for programs to exit.
+ rustix::fs::flock(&self, FlockOperation::LockShared)
+ .map_err(|e| format!("Cannot relock: {e}"))?;
+
+ // Kill all processes in the child cgroup.
+ write_value(&sub_fd, Path::new("cgroup.kill"), b"1")?;
+
+ // Wait for the child cgroup to become empty.
+ Self::wait_for_empty(&sub_fd)
+ .map_err(|e| format!("Cannot wait for cgroup to become empty: {e}"))
+ .inspect_err(|_| {
+ self.fd.pop().unwrap();
+ })?;
+
+ // Remove the child cgroup and its contents recursively.
+ remove_child_directories(sub_fd).map_err(|e| format!("Cannot remove: {e}"))?;
+ // Re-take an exclusive lock on the parent of the cgroup being purged.
+ // Otherwise, a concurrent instance of cgroup-setup might create a cgroup
+ // only for this one to delete it. The other instance could then try to
+ // create a sub-cgroup of a deleted cgroup, which would fail. Waiting
+ // until nobody is using the parent cgroup ensures these problems can't
+ // happen.
+ //
+ // This must happen *after* the lock on the cgroup being purged is released.
+ // Another instance of the program might have a shared lock on the parent
+ // and be waiting for an exclusive lock on the child. Trying to take an
+ // exclusive lock on the parent while a lock is held on the child would
+ // result in an ABBA deadlock.
+ rustix::fs::flock(&self, FlockOperation::LockExclusive)
+ .map_err(|e| format!("Cannot re-lock exclusively: {e}"))?;
+ // Delete the cgroup. If it's been re-created in the meantime
+ // and is currently in use, this is not an error. Another
+ // process deleting the cgroup is also not an error. Both of
+ // these can happen because of the time period between
+ // remove_child_directories() closing the file descriptor
+ // (releasing its lock) and the above call to flock().
+ match rustix::fs::unlinkat(&self, path, AtFlags::REMOVEDIR) {
+ Ok(()) | Err(Errno::BUSY) | Err(Errno::NOENT) => Ok(()),
+ Err(e) => Err(format!("Cannot delete: {e}")),
+ }
+ }
+}
diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
new file mode 100644
index 0000000000000000000000000000000000000000..e8d9e9d7c2111857cc25b36433d8f33ddb28c27b
--- /dev/null
+++ b/tools/cgroup-setup/src/main.rs
@@ -0,0 +1,186 @@
+// SPDX-License-Identifier: EUPL-1.2+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+mod cgroup;
+
+use cgroup::{Cgroup, openat2_simple, write_value};
+use rustix::{
+ fs::{FlockOperation, Mode, OFlags, XattrFlags},
+ io::Errno,
+};
+use std::{
+ env::ArgsOs,
+ ffi::OsStr,
+ fs::File,
+ io::Read as _,
+ os::unix::prelude::*,
+ path::{Path, PathBuf},
+};
+
+// Check that the path is canonical,
+// then split it into basename and filename.
+fn split_path(path: &Path) -> Result<(&Path, &Path), String> {
+ cgroup::check_path(path)
+ .map(|()| (path.parent().unwrap(), Path::new(path.file_name().unwrap())))
+}
+
+fn read_control_file(fd: &dyn AsFd, p: &Path) -> Result<Vec<u8>, String> {
+ let mut buf = Vec::new();
+ File::from(
+ openat2_simple(&fd, Path::new(p), OFlags::RDONLY)
+ .map_err(|e| format!("Cannot open {p:?}: {e}"))?,
+ )
+ .read_to_end(&mut buf)
+ .map_err(|e| format!("Cannot read {p:?}: {e}"))?;
+ Ok(buf)
+}
+
+fn enable_subtree_control(fd: &dyn AsFd) -> Result<(), String> {
+ let p = Path::new("cgroup.controllers");
+ let buf = read_control_file(fd, p)?;
+ let mut subtree = vec![];
+ for controller in buf.split(|&b| b == b' ').filter(|e| !e.is_empty()) {
+ if !subtree.is_empty() {
+ subtree.push(b' ');
+ }
+ subtree.push(b'+');
+ subtree.extend_from_slice(controller);
+ }
+ if !subtree.is_empty() {
+ write_value(&fd, Path::new("cgroup.subtree_control"), &subtree)?;
+ }
+ Ok(())
+}
+
+fn cgroup_setup(mut args: ArgsOs) -> Result<(), String> {
+ let mut leaf = false;
+ let mut cgroup_path;
+ let mut systemd_compat = false;
+ let mut wait = true;
+ loop {
+ cgroup_path = args.next();
+ let Some(ref arg_) = cgroup_path else {
+ break;
+ };
+ let arg_ = arg_.as_bytes();
+ if arg_ == b"--" {
+ cgroup_path = args.next();
+ break;
+ }
+ if !arg_.starts_with(b"-") {
+ break;
+ }
+
+ if !arg_.starts_with(b"--") {
+ return Err("takes no short options".to_owned());
+ }
+
+ match &arg_[2..] {
+ b"leaf" => leaf = true,
+ b"wait" => wait = true,
+ b"no-wait" => wait = false,
+ b"systemd-compat" => systemd_compat = true,
+ arg => return Err(format!("unknown long option {:?}", OsStr::from_bytes(arg))),
+ }
+ }
+ let Some(cgroup_path) = cgroup_path.map(PathBuf::from) else {
+ return Err("have no positional arguments, expected at least 1".to_owned());
+ };
+
+ let (parent_cgroup_path, child_cgroup_path) = split_path(&cgroup_path)?;
+ let cgroup = Cgroup::new(parent_cgroup_path)?;
+ match rustix::fs::mkdirat(&cgroup, child_cgroup_path, Mode::from_raw_mode(0o755)) {
+ Ok(()) | Err(Errno::EXIST) => {}
+ Err(e) => return Err(format!("Cannot make child cgroup: {e}")),
+ }
+ let child = cgroup
+ .open_beneath(child_cgroup_path, OFlags::RDONLY | OFlags::DIRECTORY)
+ .map_err(|e| format!("Cannot make child cgroup: {e}"))?;
+ if wait {
+ // While waiting, only hold an exclusive lock on the child, not the parent.
+ rustix::fs::flock(&child, FlockOperation::LockExclusive)
+ .map_err(|e| format!("Cannot take an exclusive lock on child cgroup: {e}"))?;
+ rustix::fs::flock(&cgroup, FlockOperation::LockShared)
+ .map_err(|e| format!("Cannot downgrade lock on cgroup to a shared lock: {e}"))?;
+ Cgroup::wait_for_empty(&child)
+ .map_err(|e| format!("Cannot wait for {parent_cgroup_path:?} to be empty: {e}"))?;
+ }
+ let pid = std::process::id().to_string();
+ if leaf {
+ if args.len() != 0 {
+ // If we aren't delegating any cgroups, don't create a sub-cgroup.
+ write_value(&child, Path::new("cgroup.procs"), pid.as_bytes())
+ .map_err(|e| format!("Cannot move process to child cgroup: {e}"))?;
+ }
+ } else {
+ // If the child process will need to manage cgroups itself, it will need
+ // to set up a sub-cgroup due to the "no internal processes" rule. It's
+ // simplest to just do it automatically. If the cgroup already exists,
+ // that isn't an error.
+ match rustix::fs::mkdirat(&child, cgroup::DEFAULT_LEAF, Mode::from_raw_mode(0o755)) {
+ Ok(()) | Err(Errno::EXIST) => {}
+ Err(e) => return Err(format!("Cannot make child cgroup: {e}")),
+ }
+ if args.len() != 0 {
+ let child_proc_path = Path::new(cgroup::DEFAULT_LEAF).join(Path::new("cgroup.procs"));
+ write_value(&child, &child_proc_path, pid.as_bytes())
+ .map_err(|e| format!("Cannot move process to child cgroup: {e}"))?;
+ }
+ if systemd_compat {
+ // systemd-aware programs expect to have user.delegate=1
+ // and to set cgroup.subtree_control themselves
+ rustix::fs::fsetxattr(&child, c"user.delegate", b"1", XattrFlags::empty()).map_err(
+ |e| format!("Cannot enable cgroup delegation in {parent_cgroup_path:?}: {e}"),
+ )?
+ } else {
+ // Spectrum's programs do not check for user.delegate=1
+ // and expect the caller to set cgroup.subtree_control.
+ enable_subtree_control(&child)?;
+ }
+ }
+ let Some(program_name) = args.next() else {
+ return Ok(());
+ };
+ let e = std::process::Command::new(&program_name).args(args).exec();
+ Err(format!("Cannot spawn child {:?}: {}", program_name, e))
+}
+
+fn cgroup_purge(mut args: ArgsOs) -> Result<(), String> {
+ if args.len() != 1 {
+ return Err("usage: cgroup-purge CGROUP_TO_PURGE".to_owned());
+ }
+ let arg = args.next().unwrap();
+ let (parent, child) = split_path(Path::new(&arg))?;
+ Cgroup::new(parent)?.purge_child(child)
+}
+
+fn run(prog_name: &OsStr, args: ArgsOs) -> Result<(), String> {
+ match prog_name
+ .as_bytes()
+ .split(|&b| b == b'/')
+ .next_back()
+ .unwrap()
+ {
+ b"cgroup-setup" => cgroup_setup(args),
+ b"cgroup-purge" => cgroup_purge(args),
+ _ => Err(format!(
+ "must be invoked as \"cgroup-setup\" or \
+ \"cgroup-purge\", got {prog_name:?}",
+ )),
+ }
+}
+
+fn main() {
+ let mut args = std::env::args_os();
+ let Some(prog_name) = args.next() else {
+ eprintln!("No command line arguments (argv[0] is NULL)");
+ std::process::exit(1);
+ };
+ match run(&prog_name, args) {
+ Ok(()) => {}
+ Err(e) => {
+ eprintln!("{prog_name:?}: {}", e);
+ std::process::exit(1);
+ }
+ }
+}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 03/19] Documentation: Mention control groups
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 01/19] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 02/19] tools: Add control group manager Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-08-03 13:22 ` Alyssa Ross
2026-07-31 21:54 ` [PATCH v5 04/19] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
` (16 subsequent siblings)
19 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Documentation/doc/development/control-groups.adoc | 95 +++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/Documentation/doc/development/control-groups.adoc b/Documentation/doc/development/control-groups.adoc
new file mode 100644
index 0000000000000000000000000000000000000000..50bbdcbd79125fcacd229ce41f6337f3fcaa4190
--- /dev/null
+++ b/Documentation/doc/development/control-groups.adoc
@@ -0,0 +1,95 @@
+= Control groups in Spectrum
+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0
+
+https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html[Linux control groups] (cgroups) can be used for several purposes:
+
+1. Waiting for a group of processes to exit.
+2. Terminating a group of processes.
+3. Limiting a group of processes' access to resources.
+
+Spectrum currently uses the first two. The third is not yet used,
+but will be in the future.
+
+== Control Group Hierarchy
+
+Spectrum uses the following cgroup hierarchy:
+
+1. There is a `+/vm-services.slice+` cgroup that contains all the per-VM
+ services on the system.
+2. The per-VM services for each VM are under `+/vm-services.slice/vm-${VM}.slice+`,
+ where `+${VM}+` is replaced by the VM's ID.
+3. Each per-VM service is under `+/vm-services.slice/vm-${VM}.slice/${SERVICE_NAME}+`,
+ where `+${VM}+` is replaced by the VM's ID and `+${SERVICE_NAME}+` is replaced by
+ the name of the service.
+4. The virtual machine monitor (VMM) runs under `+/vm-services.slice/vm-${VM}.slice/vmm+`.
+
+If a cgroup contains child cgroups, it likely contains a `$inner.service`
+cgroup. This is where programs that would otherwise run in the cgroup itself
+are placed. Generally, these programs are instances of
+https://skarnet.org/software/s6/s6-svscan.html[`s6-svscan`] or
+https://skarnet.org/software/s6/s6-supervise.html[`s6-supervise`].
+
+== Using Control Groups
+
+Most services should be placed in a control group, with only a few
+exceptions:
+
+1. Services, such as `getty`, that spawn background processes.
+2. Loggers.
+3. Trivial services that do almost nothing.
+
+=== Setting Up Control Groups
+
+Generally, it's best to set the control group up as the first thing
+the service does using Spectrum's https://spectrum-os.org/git/spectrum/tree/tools/cgroup-setup[`cgroup-setup`] tool.
+To do that, use `+cgroup-setup --leaf -- $1 COMMAND_LINE+`,
+where `+$1+` should be the service name and `+COMMAND_LINE+` is the program
+to run in a cgroup.
+
+If the control group starts with `/`, it is interpreted as a path relative
+to `/sys/fs/cgroup`. Otherwise, it is relative to the current control
+group the program is in.
+
+For example, using https://skarnet.org/software/execline/[execline] for your run script:
+
+[source,execline]
+....
+#!/bin/execlineb -WS1
+
+# Often, your cgroup is just the parent cgroup
+# with the name of the service ($1) appended.
+cgroup-setup --leaf -- $1
+# The rest of the script goes here.
+....
+
+=== Purging Control Groups
+
+If the service exits, it's usually best to terminate any programs left
+behind with `SIGKILL` and remove the control group. To do this,
+make the `finish` script invoke `cgroup-purge`. Its sole command-line
+argument is the cgroup to remove.
+
+[source,execline]
+....
+#!/bin/execlineb -WS3
+
+# Use the same path you used in the run script.
+cgroup-purge $3
+# The rest of the script goes here.
+....
+
+One can also use `cgroup-purge` to purge a cgroup explicitly. This is
+used to stop the VMM and all per-VM services when a VM is shut down.
+
+== Future plans
+
+Control groups are designed around a single writer process controlling each
+of them. Many Linux distros use systemd for this, but Spectrum doesn't use
+systemd. The only persistent per-service process is s6-supervise, but that
+doesn't have control group support.
+
+Instead, the plan is to have a database containing this information.
+Whether this will be in the `data/` subdirectory of the service directory
+or a separate system-wide database has not yet been determined.
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 04/19] Mount cgroup2 filesystem at /sys/fs/cgroup
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (2 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 03/19] Documentation: Mention control groups Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 05/19] host/rootfs: Enable controllers in non-root cgroups Demi Marie Obenour
` (15 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Needed for cgroups to work at all.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/image/etc/fstab | 1 +
1 file changed, 1 insertion(+)
diff --git a/host/rootfs/image/etc/fstab b/host/rootfs/image/etc/fstab
index 4d52a1da8c6d673cbbcfe2e9c0ec93085f06c40f..cab97cf9c49f864845492a1e524d43b7d546317f 100644
--- a/host/rootfs/image/etc/fstab
+++ b/host/rootfs/image/etc/fstab
@@ -6,3 +6,4 @@ tmpfs /dev/shm tmpfs nosuid,nodev 0 0
tmpfs /media tmpfs nosuid,nodev,noexec,nosymfollow,mode=755 0 0
sysfs /sys sysfs nosuid,nodev,noexec 0 0
tmpfs /tmp tmpfs nosuid,nodev 0 0
+cgroup2 /sys/fs/cgroup cgroup2 nosuid,nodev,noexec,nosymfollow 0 0
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 05/19] host/rootfs: Enable controllers in non-root cgroups
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (3 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 04/19] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 06/19] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
` (14 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
This is done with sed and execline to avoid special-cases in the cgroup
manager.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/init | 3 +++
1 file changed, 3 insertions(+)
diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
index 8ec7859ab00e24007a93d4000c8f34e353de50ae..2ab09d272702c0ea9daf6cf8bc880645a32eb2f0 100755
--- a/host/rootfs/image/etc/init
+++ b/host/rootfs/image/etc/init
@@ -12,4 +12,7 @@ if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
if { /bin/mount --make-shared / }
if { /bin/mount -a --mkdir }
+# Enable subtree control of all cgroups.
+if { sed -nE "s/(^| )/&+/g; w /sys/fs/cgroup/cgroup.subtree_control" /sys/fs/cgroup/cgroup.controllers }
+
/bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 06/19] host/rootfs: Add comments where cgroups are intentionally not used
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (4 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 05/19] host/rootfs: Enable controllers in non-root cgroups Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 07/19] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
` (13 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Most services should use cgroups, but some do not.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run | 1 +
.../etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run | 1 +
.../image/etc/s6-linux-init/run-image/service/serial-getty/template/run | 1 +
8 files changed, 8 insertions(+)
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
index 4d2cd7172e887e90d9ce87489cdacf41e51f77e8..5b2421a3dedf7c282ea818f4064535d0fc29d511 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty2 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
index 06abf5422365ff4edc9379e5cbcab716e6830399..3ca382f76368174ee1f99ea880827958c57b602c 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty3 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
index 3dd235fc93d403fc8971e923ec62cb025684b67a..983bd82927e468bab6ae20f61411680d173e4924 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty4 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
index 86b9a1ef70300de52f33a84cbfac22a77e00df36..fb55327dff5f189f543fb3629702caad9ff247e2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
@@ -2,6 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# cgroups make no sense here
s6-ipcserver-socketbinder -a 0700 /run/root-terminal
if { chown wayland /run/root-terminal }
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
index 327f4ccc5056e7ef87c089e666090307845cae3a..a6733604e91692f8440fd68efeb1a6b7378be167 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# cgroups make no sense for shutdownd
s6-linux-init-shutdownd -Bc /etc/s6-linux-init
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
index 8f7bd889e50d76a49b98d8ba9e2c6172f287277c..cf3f5c7b499d997fd426062651eca64b3accc146 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
@@ -2,5 +2,6 @@
# SPDX-License-Identifier: ISC
# SPDX-FileCopyrightText: Copyright (c) 2015-2024 Laurent Bercot <ska-skaware@skarnet.org>
+# cgroups make no sense for a logger
redirfd -rnb 0 fifo
s6-log -bpd3 -- T /run/log
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
index 78f794202bf174f3c036f3e20755ac087a988277..8fb1b6779c0bb1e3cdf67d1fa14722caacc25994 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# No need for a cgroup here.
s6-svscan -d3 instance
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
index f3ed9a7f08e60206afd222f4fc02c5fb19527561..775b828c72b9d73bef8a3355fb3ae0536975f474 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021, 2024 Alyssa Ross <hi@alyssa.is>
+# Don't put this in a cgroup. We want nohup to work.
getty -i -n -l /etc/login 0,115200,57600,38400,9600 $1 dumb
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 07/19] host/rootfs: serial-getty-generator: Use cgroups
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (5 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 06/19] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 08/19] host/rootfs: systemd-udevd: Run in cgroup Demi Marie Obenour
` (12 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Avoids concurrent execution and leaked child processes.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/file-list.mk | 1 +
.../s6-linux-init/run-image/service/serial-getty-generator/finish | 5 +++++
.../etc/s6-linux-init/run-image/service/serial-getty-generator/run | 4 +++-
3 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 3899d620717fc97f42e669e5313c4100dcf5b1cd..deb4937fe380ff143fc2aafccee1b846fe15b974 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -21,6 +21,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run \
image/etc/s6-linux-init/run-image/service/s6-svscan-log/notification-fd \
image/etc/s6-linux-init/run-image/service/s6-svscan-log/run \
+ image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish \
image/etc/s6-linux-init/run-image/service/serial-getty-generator/run \
image/etc/s6-linux-init/run-image/service/serial-getty/notification-fd \
image/etc/s6-linux-init/run-image/service/serial-getty/run \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
new file mode 100755
index 0000000000000000000000000000000000000000..b1fb5799a949a5e69d5926a2145873da5f09e0a3
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-purge $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
index bf66ab0878fc6d8e77ca71a4a89e50c139a6bb11..a1846a29cf55892127feb7ecfc0adcd12f0497bd 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
@@ -1,7 +1,9 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+# avoid concurrent operation
+cgroup-setup --leaf -- $1
piperw 3 4
background {
fdclose 3
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 08/19] host/rootfs: systemd-udevd: Run in cgroup
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (6 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 07/19] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 09/19] host/rootfs: weston: " Demi Marie Obenour
` (11 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +++++
host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +++--
3 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index deb4937fe380ff143fc2aafccee1b846fe15b974..5403e464966e4946e2089841e9f2f529c0e856a5 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -97,6 +97,7 @@ S6_RC_FILES = \
image/etc/s6-rc/systemd-udevd-coldplug/dependencies.d/systemd-udevd \
image/etc/s6-rc/systemd-udevd-coldplug/type \
image/etc/s6-rc/systemd-udevd-coldplug/up \
+ image/etc/s6-rc/systemd-udevd/finish \
image/etc/s6-rc/systemd-udevd/notification-fd \
image/etc/s6-rc/systemd-udevd/run \
image/etc/s6-rc/systemd-udevd/type \
diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/finish b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
new file mode 100755
index 0000000000000000000000000000000000000000..b1fb5799a949a5e69d5926a2145873da5f09e0a3
--- /dev/null
+++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-purge $3
diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/run b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
old mode 100644
new mode 100755
index aec6444e951503eae988e666b77fda8f2ae33d72..c831cd22cad8bcf78dcfb78b3181d256af723d7d
--- a/host/rootfs/image/etc/s6-rc/systemd-udevd/run
+++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
@@ -1,7 +1,8 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
-s6-setlock /run/sd-notify-wrapper/systemd-udevd.lock
+
+cgroup-setup --systemd-compat -- $1
s6-ipcserver-socketbinder -b0 -m -a 0600 /run/sd-notify-wrapper/systemd-udevd.sock
background -d {
fdmove 1 3
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 09/19] host/rootfs: weston: Run in cgroup
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (7 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 08/19] host/rootfs: systemd-udevd: Run in cgroup Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 10/19] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
` (10 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
host/rootfs/image/etc/s6-rc/weston/finish | 5 +++++
host/rootfs/image/etc/s6-rc/weston/run | 2 ++
3 files changed, 8 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 5403e464966e4946e2089841e9f2f529c0e856a5..2b76b31dad7f73dd83c6a24f94f3ed946988bbea 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -110,6 +110,7 @@ S6_RC_FILES = \
image/etc/s6-rc/vmm-env/contents.d/systemd-udevd-coldplug \
image/etc/s6-rc/vmm-env/type \
image/etc/s6-rc/weston/dependencies.d/systemd-udevd-coldplug \
+ image/etc/s6-rc/weston/finish \
image/etc/s6-rc/weston/notification-fd \
image/etc/s6-rc/weston/run \
image/etc/s6-rc/weston/type
diff --git a/host/rootfs/image/etc/s6-rc/weston/finish b/host/rootfs/image/etc/s6-rc/weston/finish
new file mode 100755
index 0000000000000000000000000000000000000000..b1fb5799a949a5e69d5926a2145873da5f09e0a3
--- /dev/null
+++ b/host/rootfs/image/etc/s6-rc/weston/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-purge $3
diff --git a/host/rootfs/image/etc/s6-rc/weston/run b/host/rootfs/image/etc/s6-rc/weston/run
old mode 100644
new mode 100755
index 8b065ee9a535d0a08e9255900c73b88a0c376d4b..5ae6ca2f5527ca3ca4d58ce566dd58cbf1f79b5e
--- a/host/rootfs/image/etc/s6-rc/weston/run
+++ b/host/rootfs/image/etc/s6-rc/weston/run
@@ -4,6 +4,8 @@
importas -Siu WAYLAND_DISPLAY
+cgroup-setup --leaf --no-wait weston
+
piperw 4 3
background {
fdclose 3
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 10/19] host/rootfs: Set up parent cgroup for all per-VM services
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (8 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 09/19] host/rootfs: weston: " Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 11/19] host/rootfs: Create per-VM cgroup for all of the VM's services Demi Marie Obenour
` (9 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run | 3 +++
1 file changed, 3 insertions(+)
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
index 78f794202bf174f3c036f3e20755ac087a988277..430065b1efae8ce1fe7c7a07b67920daa88bf98c 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
@@ -1,5 +1,8 @@
#!/bin/execlineb -WP
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# Set up the parent cgroup of all VMMs and VM services
+cgroup-setup --no-wait -- /vm-services.slice
s6-svscan -d3 instance
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 11/19] host/rootfs: Create per-VM cgroup for all of the VM's services
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (9 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 10/19] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 12/19] host/rootfs: run-vmm: Create per-VM cgroup Demi Marie Obenour
` (8 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../etc/s6-linux-init/run-image/service/vm-services/template/finish | 5 +++++
.../etc/s6-linux-init/run-image/service/vm-services/template/run | 4 ++++
3 files changed, 10 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 2b76b31dad7f73dd83c6a24f94f3ed946988bbea..36f508962c0515b2132aa9ab30cde929e2de100e 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -39,6 +39,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/run \
image/etc/s6-linux-init/run-image/service/vmm/notification-fd \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
new file mode 100755
index 0000000000000000000000000000000000000000..2351ff7cfffaa8002ffda387a72be49033e715c7
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-purge /vm-services.slice/vm-${3}.slice
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
index bf90e99eef4106a9ad17dfba310b6edb3eaa64a3..d3a5f3490e447448929383a6b5d36dcef723712a 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
@@ -2,6 +2,10 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# Move into the cgroup with all per-VM services
+# Do not wait for the existing processes in the slice to finish.
+cgroup-setup --no-wait -- /vm-services.slice/vm-${1}.slice
+
export VM $1
s6-svscan -d3 data/service
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 12/19] host/rootfs: run-vmm: Create per-VM cgroup
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (10 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 11/19] host/rootfs: Create per-VM cgroup for all of the VM's services Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 13/19] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
` (7 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../image/etc/s6-linux-init/run-image/service/vmm/template/finish | 5 +++++
host/rootfs/image/usr/bin/run-vmm | 4 ++++
host/rootfs/image/usr/bin/vm-stop | 5 ++++-
4 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 36f508962c0515b2132aa9ab30cde929e2de100e..5f7b0c3fc1d7d7f8dc184ecc991126a793e316c6 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -44,6 +44,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/run \
image/etc/s6-linux-init/run-image/service/vmm/notification-fd \
image/etc/s6-linux-init/run-image/service/vmm/run \
+ image/etc/s6-linux-init/run-image/service/vmm/template/finish \
image/etc/s6-linux-init/run-image/service/vmm/template/notification-fd \
image/etc/s6-linux-init/scripts/rc.init \
image/etc/s6-linux-init/scripts/rc.shutdown \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/template/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/template/finish
new file mode 100755
index 0000000000000000000000000000000000000000..6f6e549f6d95915138e53b8d406fb246d2968ccc
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/template/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-purge /vm-services.slice/vm-${3}.slice/vmm
diff --git a/host/rootfs/image/usr/bin/run-vmm b/host/rootfs/image/usr/bin/run-vmm
index 6967dc5586335d999f8030371fe0f38502409919..12896c276fbffa024fa0f89f42a2967506b1cb74 100755
--- a/host/rootfs/image/usr/bin/run-vmm
+++ b/host/rootfs/image/usr/bin/run-vmm
@@ -5,6 +5,7 @@
if {
backtick -D "" mnt {
importas -Siu 1
+ if { cgroup-setup --no-wait /vm-services.slice/vm-${1}.slice }
nsenter --mount=/run/vm/by-id/${1}/ns/mnt
findmnt -no FSTYPE,SOURCE /run/fs/${1}/disk
}
@@ -33,6 +34,8 @@ if {
elgetpositionals
+if { rm -f /run/vsock/${1}/vsock }
+
s6-ipcserver-socketbinder -B /run/vm/by-id/${1}/vmm
getpid -E vmm_pid
@@ -86,6 +89,7 @@ redirfd -r 0 /dev/null
s6-softlimit -H -l 18446744073709551615
if { udevadm wait /dev/kvm }
+cgroup-setup --leaf /vm-services.slice/vm-${1}.slice/vmm
s6-envuidgid vmm-${1}
s6-applyuidgid -Uz
bwrap
diff --git a/host/rootfs/image/usr/bin/vm-stop b/host/rootfs/image/usr/bin/vm-stop
index 6fb6aaed46de684eb035243e4f7c79111e93bf8c..23a815c0c260452ff9b1bcce7a091519b7420196 100755
--- a/host/rootfs/image/usr/bin/vm-stop
+++ b/host/rootfs/image/usr/bin/vm-stop
@@ -2,4 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
-ch-remote --api-socket /run/vm/by-id/${1}/vmm shutdown
+foreground { ch-remote --api-socket /run/vm/by-id/${1}/vmm shutdown }
+if { cgroup-purge /vm-services.slice/vm-${1}.slice/vmm }
+importas -iS ?
+exit $?
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 13/19] host/rootfs: run-appimage: Purge the per-VM cgroup
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (11 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 12/19] host/rootfs: run-vmm: Create per-VM cgroup Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 14/19] host/rootfs: run-flatpak: " Demi Marie Obenour
` (6 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/image/usr/bin/run-appimage | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/image/usr/bin/run-appimage b/host/rootfs/image/usr/bin/run-appimage
index 672ec22d4b815fb9c2940e13ae08ed4093baee4e..03020c89b02f295844f5ed814034ba66be741f4b 100755
--- a/host/rootfs/image/usr/bin/run-appimage
+++ b/host/rootfs/image/usr/bin/run-appimage
@@ -89,7 +89,12 @@ fdclose 4
foreground { run-vmm $id }
fdclose 3
-if { s6-instance-delete /run/service/vm-services $id }
+if {
+ # Even if s6-instance-delete fails, cgroup-purge is enough
+ # to ensure that the VMM and all per-VM services are killed.
+ foreground { s6-instance-delete /run/service/vm-services $id }
+ cgroup-purge /vm-services.slice/vm-${id}.slice
+}
if { umount -R /run/vm/by-id/${id}/ns }
rm -r -- $diskdir /run/vm/by-id/${id} /run/configs/${id}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 14/19] host/rootfs: run-flatpak: Purge the per-VM cgroup
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (12 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 13/19] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 15/19] host/rootfs: dbus: Run in cgroup Demi Marie Obenour
` (5 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/image/usr/bin/run-flatpak | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/image/usr/bin/run-flatpak b/host/rootfs/image/usr/bin/run-flatpak
index fafbc82ff6966d8be9c8083a17618f1fb192bd07..cdd13fb52d97751f92864d34a5221367df6d6f0d 100755
--- a/host/rootfs/image/usr/bin/run-flatpak
+++ b/host/rootfs/image/usr/bin/run-flatpak
@@ -87,9 +87,12 @@ if {
fdclose 4
foreground { run-vmm $id }
-}
-if { s6-instance-delete -- /run/service/vm-services $id }
+ # Even if s6-instance-delete fails, cgroup-purge is enough
+ # to ensure that the VMM and all per-VM services are killed.
+ foreground { s6-instance-delete /run/service/vm-services $id }
+ cgroup-purge /vm-services.slice/vm-${id}.slice
+}
if { umount -R /run/vm/by-id/${id}/ns }
rm -r -- $diskdir /run/vm/by-id/${id} /run/configs/${id}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 15/19] host/rootfs: dbus: Run in cgroup
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (13 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 14/19] host/rootfs: run-flatpak: " Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 16/19] host/rootfs: vhost-user-fs: " Demi Marie Obenour
` (4 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../run-image/service/vm-services/template/data/service/dbus/finish | 5 +++++
.../run-image/service/vm-services/template/data/service/dbus/run | 2 ++
3 files changed, 8 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 5f7b0c3fc1d7d7f8dc184ecc991126a793e316c6..ea88f25aa4d5b96d29ed86975db12c2b047a0468 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -28,6 +28,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/serial-getty/template/run \
image/etc/s6-linux-init/run-image/service/vm-services/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish
new file mode 100755
index 0000000000000000000000000000000000000000..95defd0c29f350d7a1e7badc1e1bb36bd2a36cf6
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-purge dbus
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
index 331d9a57e8d99eb8d6f7b33031cd130b410a6157..7979bd76610e0efaa832996db70a75a01df7b16e 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
@@ -4,6 +4,8 @@
importas -i VM VM
+cgroup-setup --leaf dbus
+
if {
redirfd -w 1 data/dbus.conf
sed "s/@XDP_SPECTRUM_USER@/xdp-spectrum-${VM}/g" /etc/dbus-portal.conf.in
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 16/19] host/rootfs: vhost-user-fs: Run in cgroup
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (14 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 15/19] host/rootfs: dbus: Run in cgroup Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 17/19] host/rootfs: vhost-user-gpu: " Demi Marie Obenour
` (3 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../service/vm-services/template/data/service/vhost-user-fs/finish | 5 +++++
.../service/vm-services/template/data/service/vhost-user-fs/run | 5 ++++-
3 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index ea88f25aa4d5b96d29ed86975db12c2b047a0468..e9b15f7ed466da0535809717de3c655b5f93d0f7 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -34,6 +34,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/notification-fd \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish
new file mode 100755
index 0000000000000000000000000000000000000000..b1fb5799a949a5e69d5926a2145873da5f09e0a3
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-purge $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
index cb059aebc8dfe2c8a5b059d5bfea884de53bf922..fe5e2f76d50bab0a75b897649041d46d9e95ff6b 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
@@ -2,9 +2,12 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2021-2025 Alyssa Ross <hi@alyssa.is>
-s6-ipcserver-socketbinder -a 0700 -B env/virtiofsd.sock
+cgroup-setup --leaf -- $1
importas -i VM VM
+
+s6-ipcserver-socketbinder -a 0700 -B env/virtiofsd.sock
+
if { chown vmm-${VM} env/virtiofsd.sock }
if { fdmove 1 3 echo }
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 17/19] host/rootfs: vhost-user-gpu: Run in cgroup
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (15 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 16/19] host/rootfs: vhost-user-fs: " Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 18/19] host/rootfs: xdg-desktop-portal-spectrum-host: " Demi Marie Obenour
` (2 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../service/vm-services/template/data/service/vhost-user-gpu/finish | 5 +++++
.../service/vm-services/template/data/service/vhost-user-gpu/run | 2 ++
3 files changed, 8 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index e9b15f7ed466da0535809717de3c655b5f93d0f7..9560265b06c083e3e39fb6bbfa93778ac93f041a 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -37,6 +37,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/notification-fd \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish
new file mode 100755
index 0000000000000000000000000000000000000000..2e53d99eda295a9f7943ec419952eb8c8416431c
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-purge vhost-user-gpu
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
index c3bfafe02ec6d2ee418fe56b033bbc4c7a73b186..a8ec06122e47605e7230c864ef22ef84c7de54b2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
@@ -3,6 +3,8 @@
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
+cgroup-setup --leaf vhost-user-gpu
+
s6-ipcserver-socketbinder -a 0700 -b 1 env/crosvm.sock
multisubstitute {
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 18/19] host/rootfs: xdg-desktop-portal-spectrum-host: Run in cgroup
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (16 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 17/19] host/rootfs: vhost-user-gpu: " Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 19/19] host/rootfs: spectrum-router: " Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../template/data/service/xdg-desktop-portal-spectrum-host/finish | 5 +++++
.../template/data/service/xdg-desktop-portal-spectrum-host/run | 2 ++
3 files changed, 8 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 9560265b06c083e3e39fb6bbfa93778ac93f041a..a6435b0f9c73f45198deab39ea33f9e0326b2856 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -40,6 +40,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish
new file mode 100755
index 0000000000000000000000000000000000000000..b1fb5799a949a5e69d5926a2145873da5f09e0a3
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-purge $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
index 909957b817136149dd01c3b98847238ecda8032d..3f5ebce4d2ac91f9240cb7ebeb9b307eeb4c73bf 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
@@ -2,6 +2,8 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup --leaf -- $1
+
importas -i VM VM
export DBUS_SESSION_BUS_ADDRESS unix:path=/run/portal-bus/${VM}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v5 19/19] host/rootfs: spectrum-router: Run in cgroup
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (17 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 18/19] host/rootfs: xdg-desktop-portal-spectrum-host: " Demi Marie Obenour
@ 2026-07-31 21:54 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-07-31 21:54 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../service/vm-services/template/data/service/spectrum-router/finish | 5 +++++
.../service/vm-services/template/data/service/spectrum-router/run | 2 ++
3 files changed, 8 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index a6435b0f9c73f45198deab39ea33f9e0326b2856..c8a987a6ae20301134f6340f4e3a63b2ad050e17 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -32,6 +32,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish
new file mode 100755
index 0000000000000000000000000000000000000000..9ceaa8ca1723d29361ce1cbb48efb644b6ad3948
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WP
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-purge spectrum-router
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
index 3de58c27facc78c48176d8f9a6dd1827ac926f1a..a2ae8499e65590830b90c9bdda7d0f8a3336ef09 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
@@ -3,6 +3,8 @@
# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup --leaf spectrum-router
+
importas -i VM VM
s6-ipcserver-socketbinder -a 0770 /run/vm/by-id/${VM}/router-driver.sock
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* Re: [PATCH v5 02/19] tools: Add control group manager
2026-07-31 21:54 ` [PATCH v5 02/19] tools: Add control group manager Demi Marie Obenour
@ 2026-08-03 12:47 ` Alyssa Ross
2026-08-05 1:36 ` Demi Marie Obenour
0 siblings, 1 reply; 132+ messages in thread
From: Alyssa Ross @ 2026-08-03 12:47 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
Demi Marie Obenour <demiobenour@gmail.com> writes:
> The cgroup-setup Rust program can create and purge cgroups. It can also
> wait for one to become empty, spawn a program in a cgroup, and more. In
> the future, it will also support cgroup-based resource control. Locking
> is used to ensure that concurrent invocations are safe.
>
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> .codespellrc | 2 +-
> host/rootfs/default.nix | 6 +-
> pkgs/default.nix | 1 +
> tools/cgroup-setup/Cargo.lock | 67 ++++++++
> tools/cgroup-setup/Cargo.lock.license | 2 +
> tools/cgroup-setup/Cargo.toml | 10 ++
> tools/cgroup-setup/default.nix | 22 +++
> tools/cgroup-setup/src/cgroup.rs | 308 ++++++++++++++++++++++++++++++++++
> tools/cgroup-setup/src/main.rs | 186 ++++++++++++++++++++
> 9 files changed, 600 insertions(+), 4 deletions(-)
Looking much better, thank you!
> +fn push_child_fds(fds: &mut Vec<(Rc<RefCell<Dir>>, PathBuf)>, fd: OwnedFd) {
Would it not make more sense to take Dir than OwnedFd?
> + // The rustix source code shows that Dir::new() never fails.
> + let child_fd = Rc::new(RefCell::new(Dir::new(fd).unwrap()));
> + while let Some(element) = child_fd.borrow_mut().next() {
> + let element = element.expect("Iterating through a cgroup directory failed?");
> + if element.file_type() != rustix::fs::FileType::Directory {
> + continue;
> + }
> + match element.file_name().to_bytes() {
> + b"." | b".." => {}
> + other => {
> + let other = Path::new(OsStr::from_bytes(other)).to_owned();
> + assert_single_component(&other);
> + fds.push((child_fd.clone(), other));
The data structures used here are still very confusing. Why are we
storing a reference to the same file descriptor in every entry in the
Vec?
> + }
> + }
> + }
> +}
> +
> +// Remove all subdirectories of the given directory recursively,
> +// but not the directory itself. The directory file descriptor
> +// is closed.
> +//
> +// This isn't the most efficient possible algorithm, but
> +// simplicity is more important than performance in this
> +// case. Also, it keeps open more file descriptors than
> +// strictly necessary, but Spectrum runs with a very high
> +// limit for the number of open file descriptors, and it
> +// uses shallow control group hierarchies.
> +fn remove_child_directories(dirfd: OwnedFd) -> Result<(), Errno> {
> + let mut fds = Vec::new();
> + // Push the children of this directory onto the stack.
> + push_child_fds(&mut fds, dirfd);
> + while let Some((d, path)) = fds.pop() {
Couldn't we call push_child_fds() once here, rather than twice as is
currently done? (And then consider inlining it, depending on how
complex it's looking at the time.)
> + assert_single_component(&path);
> + // Try to delete the directory. If that fails because there are child
> + // directories, push the child directories onto the stack, then push
> + // this directory again.
> + match rustix::fs::unlinkat(
> + // The rustix source code shows that Dir::fd() never fails.
> + d.borrow().fd().unwrap(),
> + &path,
> + AtFlags::REMOVEDIR,
> + ) {
> + Err(Errno::NOTEMPTY) => {}
> + Ok(()) => continue,
> + Err(bad) => return Err(bad),
> + }
> + let fd = openat2_simple(
> + // The rustix source code shows that Dir::fd() never fails.
> + &d.borrow().fd().unwrap(),
> + &path,
> + OFlags::DIRECTORY | OFlags::RDONLY,
> + )?;
> + // Process child directories first, then attempt to delete the
> + // directory again.
> + fds.push((d, path));
> + push_child_fds(&mut fds, fd);
> + }
> + Ok(())
> +}
> +impl Cgroup {
> + pub fn open_beneath(&self, path: &Path, flags: OFlags) -> Result<OwnedFd, Errno> {
> + openat2_simple(self, path, flags)
> + }
This method looks pretty redundant now.
> + pub fn new(path: &Path) -> Result<Self, String> {
> + let cgroup_root = rustix::fs::openat2(
> + CWD,
> + Path::new("/sys/fs/cgroup"),
> + OFlags::CLOEXEC | OFlags::DIRECTORY | OFlags::RDONLY,
> + Mode::empty(),
> + ResolveFlags::NO_SYMLINKS | ResolveFlags::NO_MAGICLINKS,
> + )
> + .map_err(|e| format!("Cannot open /sys/fs/cgroup: {e}"))?;
Can we not use openat2_simple here? It's missing e.g. NOCTTY.
> + let mut cgroup = Self {
> + fd: vec![(cgroup_root)],
> + };
> +
> + let path = prepend_current_cgroup_if_needed(path);
> + for component in path.components() {
> + let component = match component {
> + Component::Normal(component) => component,
> + _ => unreachable!(),
> + };
I think it would be slightly more idiomatic to do:
let Component::Normal(component) = component else {
unreachable!()
};
> + let sub_fd = cgroup
> + .open_beneath(Path::new(component), OFlags::RDONLY | OFlags::DIRECTORY)
> + .map_err(|e| format!("Cannot open sub-cgroup {component:?}: {e}"))?;
> + // Take a shared lock on the *previous* file descriptor.
> + rustix::fs::flock(&cgroup, FlockOperation::LockShared)
> + .map_err(|e| format!("Cannot lock sub-cgroup {component:?}: {e}"))?;
> + cgroup.fd.push(sub_fd);
> + }
> + // Take an exclusive lock on the final file descriptor.
> + rustix::fs::flock(&cgroup, FlockOperation::LockExclusive)
> + .map_err(|e| format!("Cannot lock {path:?}: {e}"))?;
> + Ok(cgroup)
> + }
> +
> + pub fn wait_for_empty(fd: &dyn AsFd) -> std::io::Result<()> {
> + let wait_file = openat2_simple(fd, Path::new("cgroup.events"), OFlags::RDONLY)?;
> + let poll_fd = wait_file.as_raw_fd();
> + let mut wait_fd = File::from(wait_file);
> + let mut fds = libc::pollfd {
> + fd: poll_fd,
I would inline poll_fd here. RawFd is easy to misuse, so I like to
avoid having them hang around.
> + events: libc::POLLPRI | libc::POLLERR,
> + revents: 0,
> + };
> + let mut v = vec![];
> + loop {
> + v.clear();
> + wait_fd
> + .seek(std::io::SeekFrom::Start(0))
> + .expect("Seek on control group file should succeed");
> + wait_fd
> + .read_to_end(&mut v)
> + .expect("reading from control group should work");
> + // Check that the cgroup isn't already empty. If it was,
> + // the kernel would not send an event and poll() would wait
> + // forever.
> + if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
> + break;
> + }
> + // SAFETY: FFI call, valid arguments, fds contains 1 element
> + if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
> + panic!("poll failed");
> + }
> + }
> + Ok(())
> + }
> +
> + pub fn purge_child(&mut self, path: &Path) -> Result<(), String> {
> + assert_single_component(path);
> + // See if we can just delete the child directly.
> + match rustix::fs::unlinkat(&self, Path::new(path), AtFlags::REMOVEDIR) {
> + // If the cgroup was successfully deleted, or if it
> + // has already been deleted, we are done.
> + Ok(()) | Err(Errno::NOENT) => return Ok(()),
> + // If this cgroup is in use, keep going.
> + Err(Errno::BUSY) => {}
> + Err(e) => return Err(format!("Cannot purge {path:?}: {e}")),
> + }
> +
> + let sub_fd = match self.open_beneath(path, OFlags::RDONLY | OFlags::DIRECTORY) {
> + Ok(sub_fd) => sub_fd,
> + Err(Errno::NOENT) => return Ok(()),
> + Err(e) => {
> + return Err(format!("Cannot open sub-cgroup {path:?}: {e}",));
> + }
> + };
> +
> + // Take an exclusive lock on the cgroup that is about to be
> + // removed. This avoids concurrent executions of this program
> + // operating on deleted sub-cgroups.
> + rustix::fs::flock(&sub_fd, FlockOperation::LockExclusive)
> + .map_err(|e| format!("Cannot lock sub-cgroup: {e}"))?;
> +
> + // Drop the exclusive lock on the original cgroup,
> + // This avoids blocking concurrent operations on other
> + // child cgroups while the cgroup is being purged,
> + // or while waiting for programs to exit.
> + rustix::fs::flock(&self, FlockOperation::LockShared)
> + .map_err(|e| format!("Cannot relock: {e}"))?;
Could you add some extra explanation here of why it's okay for the
exclusive lock to be temporarily dropped here?
I'm wondering whether taking a lock, then dropping it temporarily is a
sign that we're taking the lock too early in the first place, and should
scope it better to where it's actually needed.
> +
> + // Kill all processes in the child cgroup.
> + write_value(&sub_fd, Path::new("cgroup.kill"), b"1")?;
> +
> + // Wait for the child cgroup to become empty.
> + Self::wait_for_empty(&sub_fd)
> + .map_err(|e| format!("Cannot wait for cgroup to become empty: {e}"))
> + .inspect_err(|_| {
> + self.fd.pop().unwrap();
> + })?;
> +
> + // Remove the child cgroup and its contents recursively.
> + remove_child_directories(sub_fd).map_err(|e| format!("Cannot remove: {e}"))?;
> + // Re-take an exclusive lock on the parent of the cgroup being purged.
> + // Otherwise, a concurrent instance of cgroup-setup might create a cgroup
> + // only for this one to delete it. The other instance could then try to
> + // create a sub-cgroup of a deleted cgroup, which would fail. Waiting
> + // until nobody is using the parent cgroup ensures these problems can't
> + // happen.
> + //
> + // This must happen *after* the lock on the cgroup being purged is released.
> + // Another instance of the program might have a shared lock on the parent
> + // and be waiting for an exclusive lock on the child. Trying to take an
> + // exclusive lock on the parent while a lock is held on the child would
> + // result in an ABBA deadlock.
> + rustix::fs::flock(&self, FlockOperation::LockExclusive)
> + .map_err(|e| format!("Cannot re-lock exclusively: {e}"))?;
> + // Delete the cgroup. If it's been re-created in the meantime
> + // and is currently in use, this is not an error. Another
> + // process deleting the cgroup is also not an error. Both of
> + // these can happen because of the time period between
> + // remove_child_directories() closing the file descriptor
> + // (releasing its lock) and the above call to flock().
> + match rustix::fs::unlinkat(&self, path, AtFlags::REMOVEDIR) {
> + Ok(()) | Err(Errno::BUSY) | Err(Errno::NOENT) => Ok(()),
> + Err(e) => Err(format!("Cannot delete: {e}")),
> + }
> + }
> +}
> diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
> new file mode 100644
> index 0000000000000000000000000000000000000000..e8d9e9d7c2111857cc25b36433d8f33ddb28c27b
> --- /dev/null
> +++ b/tools/cgroup-setup/src/main.rs
> @@ -0,0 +1,186 @@
> +// SPDX-License-Identifier: EUPL-1.2+
> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +
> +mod cgroup;
> +
> +use cgroup::{Cgroup, openat2_simple, write_value};
> +use rustix::{
> + fs::{FlockOperation, Mode, OFlags, XattrFlags},
> + io::Errno,
> +};
> +use std::{
> + env::ArgsOs,
> + ffi::OsStr,
> + fs::File,
> + io::Read as _,
> + os::unix::prelude::*,
> + path::{Path, PathBuf},
> +};
> +
> +// Check that the path is canonical,
> +// then split it into basename and filename.
> +fn split_path(path: &Path) -> Result<(&Path, &Path), String> {
> + cgroup::check_path(path)
> + .map(|()| (path.parent().unwrap(), Path::new(path.file_name().unwrap())))
Doing this with map rather than ? is a little strange.
> +}
> +
> +fn read_control_file(fd: &dyn AsFd, p: &Path) -> Result<Vec<u8>, String> {
> + let mut buf = Vec::new();
> + File::from(
> + openat2_simple(&fd, Path::new(p), OFlags::RDONLY)
> + .map_err(|e| format!("Cannot open {p:?}: {e}"))?,
> + )
> + .read_to_end(&mut buf)
> + .map_err(|e| format!("Cannot read {p:?}: {e}"))?;
> + Ok(buf)
> +}
Nothing control-file-specific about this method. It just reads a file.
And a bit odd for write_value to be in cgroup.rs while this is here.
> +
> +fn enable_subtree_control(fd: &dyn AsFd) -> Result<(), String> {
Would it not make sense for this to be an instance method on Cgroup,
since it's a Cgroup-specific operation?
> + let p = Path::new("cgroup.controllers");
> + let buf = read_control_file(fd, p)?;
p is only used here, so can just be inlined. If read_control_file took
AsRef<Path> like the standard library functions do, you wouldn't even
need to construct the path here.
> + let mut subtree = vec![];
> + for controller in buf.split(|&b| b == b' ').filter(|e| !e.is_empty()) {
Are there ever likely to be empty works in this file?
> + if !subtree.is_empty() {
> + subtree.push(b' ');
> + }
> + subtree.push(b'+');
> + subtree.extend_from_slice(controller);
> + }
> + if !subtree.is_empty() {
> + write_value(&fd, Path::new("cgroup.subtree_control"), &subtree)?;
> + }
> + Ok(())
> +}
> +
> +fn cgroup_setup(mut args: ArgsOs) -> Result<(), String> {
> + let mut leaf = false;
> + let mut cgroup_path;
> + let mut systemd_compat = false;
> + let mut wait = true;
> + loop {
> + cgroup_path = args.next();
> + let Some(ref arg_) = cgroup_path else {
> + break;
> + };
> + let arg_ = arg_.as_bytes();
> + if arg_ == b"--" {
> + cgroup_path = args.next();
This cgroup_path thing is a bit complicated. I feel like this could
probably be cleaned up with a peekable iterator and a while loop.
> + break;
> + }
> + if !arg_.starts_with(b"-") {
> + break;
> + }
> +
> + if !arg_.starts_with(b"--") {
> + return Err("takes no short options".to_owned());
> + }
> +
> + match &arg_[2..] {
> + b"leaf" => leaf = true,
> + b"wait" => wait = true,
> + b"no-wait" => wait = false,
> + b"systemd-compat" => systemd_compat = true,
Why do we have --wait and --no-wait, but no --no-leaf or --no-systemd-compat?
> + arg => return Err(format!("unknown long option {:?}", OsStr::from_bytes(arg))),
> + }
> + }
> + let Some(cgroup_path) = cgroup_path.map(PathBuf::from) else {
> + return Err("have no positional arguments, expected at least 1".to_owned());
> + };
> +
> + let (parent_cgroup_path, child_cgroup_path) = split_path(&cgroup_path)?;
> + let cgroup = Cgroup::new(parent_cgroup_path)?;
> + match rustix::fs::mkdirat(&cgroup, child_cgroup_path, Mode::from_raw_mode(0o755)) {
> + Ok(()) | Err(Errno::EXIST) => {}
> + Err(e) => return Err(format!("Cannot make child cgroup: {e}")),
> + }
> + let child = cgroup
> + .open_beneath(child_cgroup_path, OFlags::RDONLY | OFlags::DIRECTORY)
> + .map_err(|e| format!("Cannot make child cgroup: {e}"))?;
> + if wait {
> + // While waiting, only hold an exclusive lock on the child, not the parent.
> + rustix::fs::flock(&child, FlockOperation::LockExclusive)
> + .map_err(|e| format!("Cannot take an exclusive lock on child cgroup: {e}"))?;
> + rustix::fs::flock(&cgroup, FlockOperation::LockShared)
> + .map_err(|e| format!("Cannot downgrade lock on cgroup to a shared lock: {e}"))?;
> + Cgroup::wait_for_empty(&child)
> + .map_err(|e| format!("Cannot wait for {parent_cgroup_path:?} to be empty: {e}"))?;
> + }
> + let pid = std::process::id().to_string();
> + if leaf {
> + if args.len() != 0 {
> + // If we aren't delegating any cgroups, don't create a sub-cgroup.
> + write_value(&child, Path::new("cgroup.procs"), pid.as_bytes())
> + .map_err(|e| format!("Cannot move process to child cgroup: {e}"))?;
> + }
> + } else {
> + // If the child process will need to manage cgroups itself, it will need
> + // to set up a sub-cgroup due to the "no internal processes" rule. It's
> + // simplest to just do it automatically. If the cgroup already exists,
> + // that isn't an error.
> + match rustix::fs::mkdirat(&child, cgroup::DEFAULT_LEAF, Mode::from_raw_mode(0o755)) {
> + Ok(()) | Err(Errno::EXIST) => {}
> + Err(e) => return Err(format!("Cannot make child cgroup: {e}")),
> + }
> + if args.len() != 0 {
> + let child_proc_path = Path::new(cgroup::DEFAULT_LEAF).join(Path::new("cgroup.procs"));
> + write_value(&child, &child_proc_path, pid.as_bytes())
> + .map_err(|e| format!("Cannot move process to child cgroup: {e}"))?;
> + }
> + if systemd_compat {
> + // systemd-aware programs expect to have user.delegate=1
> + // and to set cgroup.subtree_control themselves
> + rustix::fs::fsetxattr(&child, c"user.delegate", b"1", XattrFlags::empty()).map_err(
> + |e| format!("Cannot enable cgroup delegation in {parent_cgroup_path:?}: {e}"),
> + )?
> + } else {
> + // Spectrum's programs do not check for user.delegate=1
> + // and expect the caller to set cgroup.subtree_control.
> + enable_subtree_control(&child)?;
> + }
> + }
So looking at this I still see several different modes and am wondering
whether we could simplify this further.
• Why do we need a separate leaf mode? Why not just still use a
$inner.service in that case?
• What would the consequences be if we took the systemd_compat branch
for a non-cgroup-aware Spectrum program?
> + let Some(program_name) = args.next() else {
> + return Ok(());
> + };
> + let e = std::process::Command::new(&program_name).args(args).exec();
> + Err(format!("Cannot spawn child {:?}: {}", program_name, e))
> +}
> +
> +fn cgroup_purge(mut args: ArgsOs) -> Result<(), String> {
> + if args.len() != 1 {
> + return Err("usage: cgroup-purge CGROUP_TO_PURGE".to_owned());
> + }
> + let arg = args.next().unwrap();
> + let (parent, child) = split_path(Path::new(&arg))?;
> + Cgroup::new(parent)?.purge_child(child)
> +}
> +
> +fn run(prog_name: &OsStr, args: ArgsOs) -> Result<(), String> {
> + match prog_name
> + .as_bytes()
> + .split(|&b| b == b'/')
> + .next_back()
> + .unwrap()
prog_name.file_name(), where prog_name is &Path?
> + {
> + b"cgroup-setup" => cgroup_setup(args),
> + b"cgroup-purge" => cgroup_purge(args),
> + _ => Err(format!(
> + "must be invoked as \"cgroup-setup\" or \
> + \"cgroup-purge\", got {prog_name:?}",
> + )),
> + }
> +}
> +
> +fn main() {
> + let mut args = std::env::args_os();
> + let Some(prog_name) = args.next() else {
> + eprintln!("No command line arguments (argv[0] is NULL)");
> + std::process::exit(1);
> + };
> + match run(&prog_name, args) {
> + Ok(()) => {}
> + Err(e) => {
> + eprintln!("{prog_name:?}: {}", e);
> + std::process::exit(1);
> + }
> + }
> +}
>
> --
> 2.55.0
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v5 03/19] Documentation: Mention control groups
2026-07-31 21:54 ` [PATCH v5 03/19] Documentation: Mention control groups Demi Marie Obenour
@ 2026-08-03 13:22 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-08-03 13:22 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development, Valentin Gagarin
[-- Attachment #1: Type: text/plain, Size: 435 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
> Documentation/doc/development/control-groups.adoc | 95 +++++++++++++++++++++++
> 1 file changed, 95 insertions(+)
My comments from last time seem to have gone mostly unaddressed.
(Also CCing Valentin again — make sure that people who have previously
reviewed your patches are CCed on revisions.)
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v5 02/19] tools: Add control group manager
2026-08-03 12:47 ` Alyssa Ross
@ 2026-08-05 1:36 ` Demi Marie Obenour
2026-08-05 16:39 ` Alyssa Ross
0 siblings, 1 reply; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-05 1:36 UTC (permalink / raw)
To: Alyssa Ross; +Cc: Spectrum OS Development
[-- Attachment #1.1: Type: text/plain, Size: 23782 bytes --]
On 8/3/26 08:47, Alyssa Ross wrote:
> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> The cgroup-setup Rust program can create and purge cgroups. It can also
>> wait for one to become empty, spawn a program in a cgroup, and more. In
>> the future, it will also support cgroup-based resource control. Locking
>> is used to ensure that concurrent invocations are safe.
>>
>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>> ---
>> .codespellrc | 2 +-
>> host/rootfs/default.nix | 6 +-
>> pkgs/default.nix | 1 +
>> tools/cgroup-setup/Cargo.lock | 67 ++++++++
>> tools/cgroup-setup/Cargo.lock.license | 2 +
>> tools/cgroup-setup/Cargo.toml | 10 ++
>> tools/cgroup-setup/default.nix | 22 +++
>> tools/cgroup-setup/src/cgroup.rs | 308 ++++++++++++++++++++++++++++++++++
>> tools/cgroup-setup/src/main.rs | 186 ++++++++++++++++++++
>> 9 files changed, 600 insertions(+), 4 deletions(-)
>
> Looking much better, thank you!
>
>> +fn push_child_fds(fds: &mut Vec<(Rc<RefCell<Dir>>, PathBuf)>, fd: OwnedFd) {
>
> Would it not make more sense to take Dir than OwnedFd?
Easy fix.
>> + // The rustix source code shows that Dir::new() never fails.
>> + let child_fd = Rc::new(RefCell::new(Dir::new(fd).unwrap()));
>> + while let Some(element) = child_fd.borrow_mut().next() {
>> + let element = element.expect("Iterating through a cgroup directory failed?");
>> + if element.file_type() != rustix::fs::FileType::Directory {
>> + continue;
>> + }
>> + match element.file_name().to_bytes() {
>> + b"." | b".." => {}
>> + other => {
>> + let other = Path::new(OsStr::from_bytes(other)).to_owned();
>> + assert_single_component(&other);
>> + fds.push((child_fd.clone(), other));
>
> The data structures used here are still very confusing. Why are we
> storing a reference to the same file descriptor in every entry in the
> Vec?
Consider the recursive implementation (in pseudo-Rust):
fn recursive_remove(fd) {
for entry in get_entries(&fd) {
if entry.is_dir_and_not_dot_or_dotdot() {
let directory = open_dir(&fd, &entry.path())?;
recursive_remove(directory)?;
remove_dir(&fd, entry.path())?;
}
}
}
The compiler knows that fd will stay open through recursive calls,
so this doesn't need any unsafe code. Using an explicit stack takes
away this information from the compiler, so unsafe code is required.
Using Rc<RefCell<Dir>> avoids the need for unsafe code at a cost
in performance.
For what it is worth, the standard library implementation of
fs::remove_dir_all() is recursive. Standard library security hole?
>> + }
>> + }
>> + }
>> +}
>> +
>> +// Remove all subdirectories of the given directory recursively,
>> +// but not the directory itself. The directory file descriptor
>> +// is closed.
>> +//
>> +// This isn't the most efficient possible algorithm, but
>> +// simplicity is more important than performance in this
>> +// case. Also, it keeps open more file descriptors than
>> +// strictly necessary, but Spectrum runs with a very high
>> +// limit for the number of open file descriptors, and it
>> +// uses shallow control group hierarchies.
>> +fn remove_child_directories(dirfd: OwnedFd) -> Result<(), Errno> {
>> + let mut fds = Vec::new();
>> + // Push the children of this directory onto the stack.
>> + push_child_fds(&mut fds, dirfd);
>> + while let Some((d, path)) = fds.pop() {
>
> Couldn't we call push_child_fds() once here, rather than twice as is
> currently done? (And then consider inlining it, depending on how
> complex it's looking at the time.)
Can you provide an example? I don't see how to make this change
while preserving semantics. Only directories meant for deletion
can appear on the stack, and the root of the traversal must not
be deleted (yet).
>> + assert_single_component(&path);
>> + // Try to delete the directory. If that fails because there are child
>> + // directories, push the child directories onto the stack, then push
>> + // this directory again.
>> + match rustix::fs::unlinkat(
>> + // The rustix source code shows that Dir::fd() never fails.
>> + d.borrow().fd().unwrap(),
>> + &path,
>> + AtFlags::REMOVEDIR,
>> + ) {
>> + Err(Errno::NOTEMPTY) => {}
>> + Ok(()) => continue,
>> + Err(bad) => return Err(bad),
>> + }
>> + let fd = openat2_simple(
>> + // The rustix source code shows that Dir::fd() never fails.
>> + &d.borrow().fd().unwrap(),
>> + &path,
>> + OFlags::DIRECTORY | OFlags::RDONLY,
>> + )?;
>> + // Process child directories first, then attempt to delete the
>> + // directory again.
>> + fds.push((d, path));
>> + push_child_fds(&mut fds, fd);
>> + }
>> + Ok(())
>> +}
>
>> +impl Cgroup {
>> + pub fn open_beneath(&self, path: &Path, flags: OFlags) -> Result<OwnedFd, Errno> {
>> + openat2_simple(self, path, flags)
>> + }
>
> This method looks pretty redundant now.
Will remove.
>> + pub fn new(path: &Path) -> Result<Self, String> {
>> + let cgroup_root = rustix::fs::openat2(
>> + CWD,
>> + Path::new("/sys/fs/cgroup"),
>> + OFlags::CLOEXEC | OFlags::DIRECTORY | OFlags::RDONLY,
>> + Mode::empty(),
>> + ResolveFlags::NO_SYMLINKS | ResolveFlags::NO_MAGICLINKS,
>> + )
>> + .map_err(|e| format!("Cannot open /sys/fs/cgroup: {e}"))?;
>
> Can we not use openat2_simple here? It's missing e.g. NOCTTY.
NOCTTY is redundant with DIRECTORY, as directories can't be terminals.
openat2_simple() uses NO_XDEV, which won't work here because
/sys/fs/cgroup is not the same filesystem as /.
>> + let mut cgroup = Self {
>> + fd: vec![(cgroup_root)],
>> + };
>> +
>> + let path = prepend_current_cgroup_if_needed(path);
>> + for component in path.components() {
>> + let component = match component {
>> + Component::Normal(component) => component,
>> + _ => unreachable!(),
>> + };
>
> I think it would be slightly more idiomatic to do:
>
> let Component::Normal(component) = component else {
> unreachable!()
> };
Will fix in v6.
>> + let sub_fd = cgroup
>> + .open_beneath(Path::new(component), OFlags::RDONLY | OFlags::DIRECTORY)
>> + .map_err(|e| format!("Cannot open sub-cgroup {component:?}: {e}"))?;
>> + // Take a shared lock on the *previous* file descriptor.
>> + rustix::fs::flock(&cgroup, FlockOperation::LockShared)
>> + .map_err(|e| format!("Cannot lock sub-cgroup {component:?}: {e}"))?;
>> + cgroup.fd.push(sub_fd);
>> + }
>> + // Take an exclusive lock on the final file descriptor.
>> + rustix::fs::flock(&cgroup, FlockOperation::LockExclusive)
>> + .map_err(|e| format!("Cannot lock {path:?}: {e}"))?;
>> + Ok(cgroup)
>> + }
>> +
>> + pub fn wait_for_empty(fd: &dyn AsFd) -> std::io::Result<()> {
>> + let wait_file = openat2_simple(fd, Path::new("cgroup.events"), OFlags::RDONLY)?;
>> + let poll_fd = wait_file.as_raw_fd();
>> + let mut wait_fd = File::from(wait_file);
>> + let mut fds = libc::pollfd {
>> + fd: poll_fd,
>
> I would inline poll_fd here. RawFd is easy to misuse, so I like to
> avoid having them hang around.
I will move `fds` into the inner loop.
>> + events: libc::POLLPRI | libc::POLLERR,
>> + revents: 0,
>> + };
>> + let mut v = vec![];
>> + loop {
>> + v.clear();
>> + wait_fd
>> + .seek(std::io::SeekFrom::Start(0))
>> + .expect("Seek on control group file should succeed");
>> + wait_fd
>> + .read_to_end(&mut v)
>> + .expect("reading from control group should work");
>> + // Check that the cgroup isn't already empty. If it was,
>> + // the kernel would not send an event and poll() would wait
>> + // forever.
>> + if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
>> + break;
>> + }
>> + // SAFETY: FFI call, valid arguments, fds contains 1 element
>> + if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
>> + panic!("poll failed");
>> + }
>> + }
>> + Ok(())
>> + }
>> +
>> + pub fn purge_child(&mut self, path: &Path) -> Result<(), String> {
>> + assert_single_component(path);
>> + // See if we can just delete the child directly.
>> + match rustix::fs::unlinkat(&self, Path::new(path), AtFlags::REMOVEDIR) {
>> + // If the cgroup was successfully deleted, or if it
>> + // has already been deleted, we are done.
>> + Ok(()) | Err(Errno::NOENT) => return Ok(()),
>> + // If this cgroup is in use, keep going.
>> + Err(Errno::BUSY) => {}
>> + Err(e) => return Err(format!("Cannot purge {path:?}: {e}")),
>> + }
>> +
>> + let sub_fd = match self.open_beneath(path, OFlags::RDONLY | OFlags::DIRECTORY) {
>> + Ok(sub_fd) => sub_fd,
>> + Err(Errno::NOENT) => return Ok(()),
>> + Err(e) => {
>> + return Err(format!("Cannot open sub-cgroup {path:?}: {e}",));
>> + }
>> + };
>> +
>> + // Take an exclusive lock on the cgroup that is about to be
>> + // removed. This avoids concurrent executions of this program
>> + // operating on deleted sub-cgroups.
>> + rustix::fs::flock(&sub_fd, FlockOperation::LockExclusive)
>> + .map_err(|e| format!("Cannot lock sub-cgroup: {e}"))?;
>> +
>> + // Drop the exclusive lock on the original cgroup,
>> + // This avoids blocking concurrent operations on other
>> + // child cgroups while the cgroup is being purged,
>> + // or while waiting for programs to exit.
>> + rustix::fs::flock(&self, FlockOperation::LockShared)
>> + .map_err(|e| format!("Cannot relock: {e}"))?;
>
> Could you add some extra explanation here of why it's okay for the
> exclusive lock to be temporarily dropped here?
>
> I'm wondering whether taking a lock, then dropping it temporarily is a
> sign that we're taking the lock too early in the first place, and should
> scope it better to where it's actually needed.
Indeed so. Programs that are modifying a cgroup need an exclusive
lock on it. Adding or removing to the cgroup does *not* count as
modification: both operations are idempotent, removing an in-use
cgroup fails with -EBUSY, and operating on a deleted cgroup fails
with -ENODEV or -ENOENT depending on what one is doing. Operations on
control files *do* require an exclusive lock.
>> +
>> + // Kill all processes in the child cgroup.
>> + write_value(&sub_fd, Path::new("cgroup.kill"), b"1")?;
>> +
>> + // Wait for the child cgroup to become empty.
>> + Self::wait_for_empty(&sub_fd)
>> + .map_err(|e| format!("Cannot wait for cgroup to become empty: {e}"))
>> + .inspect_err(|_| {
>> + self.fd.pop().unwrap();
>> + })?;
>> +
>> + // Remove the child cgroup and its contents recursively.
>> + remove_child_directories(sub_fd).map_err(|e| format!("Cannot remove: {e}"))?;
>> + // Re-take an exclusive lock on the parent of the cgroup being purged.
>> + // Otherwise, a concurrent instance of cgroup-setup might create a cgroup
>> + // only for this one to delete it. The other instance could then try to
>> + // create a sub-cgroup of a deleted cgroup, which would fail. Waiting
>> + // until nobody is using the parent cgroup ensures these problems can't
>> + // happen.
>> + //
>> + // This must happen *after* the lock on the cgroup being purged is released.
>> + // Another instance of the program might have a shared lock on the parent
>> + // and be waiting for an exclusive lock on the child. Trying to take an
>> + // exclusive lock on the parent while a lock is held on the child would
>> + // result in an ABBA deadlock.
>> + rustix::fs::flock(&self, FlockOperation::LockExclusive)
>> + .map_err(|e| format!("Cannot re-lock exclusively: {e}"))?;
>> + // Delete the cgroup. If it's been re-created in the meantime
>> + // and is currently in use, this is not an error. Another
>> + // process deleting the cgroup is also not an error. Both of
>> + // these can happen because of the time period between
>> + // remove_child_directories() closing the file descriptor
>> + // (releasing its lock) and the above call to flock().
>> + match rustix::fs::unlinkat(&self, path, AtFlags::REMOVEDIR) {
>> + Ok(()) | Err(Errno::BUSY) | Err(Errno::NOENT) => Ok(()),
>> + Err(e) => Err(format!("Cannot delete: {e}")),
>> + }
>> + }
>> +}
>> diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
>> new file mode 100644
>> index 0000000000000000000000000000000000000000..e8d9e9d7c2111857cc25b36433d8f33ddb28c27b
>> --- /dev/null
>> +++ b/tools/cgroup-setup/src/main.rs
>> @@ -0,0 +1,186 @@
>> +// SPDX-License-Identifier: EUPL-1.2+
>> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
>> +
>> +mod cgroup;
>> +
>> +use cgroup::{Cgroup, openat2_simple, write_value};
>> +use rustix::{
>> + fs::{FlockOperation, Mode, OFlags, XattrFlags},
>> + io::Errno,
>> +};
>> +use std::{
>> + env::ArgsOs,
>> + ffi::OsStr,
>> + fs::File,
>> + io::Read as _,
>> + os::unix::prelude::*,
>> + path::{Path, PathBuf},
>> +};
>> +
>> +// Check that the path is canonical,
>> +// then split it into basename and filename.
>> +fn split_path(path: &Path) -> Result<(&Path, &Path), String> {
>> + cgroup::check_path(path)
>> + .map(|()| (path.parent().unwrap(), Path::new(path.file_name().unwrap())))
>
> Doing this with map rather than ? is a little strange.
Will change in v6.
>> +}
>> +
>> +fn read_control_file(fd: &dyn AsFd, p: &Path) -> Result<Vec<u8>, String> {
>> + let mut buf = Vec::new();
>> + File::from(
>> + openat2_simple(&fd, Path::new(p), OFlags::RDONLY)
>> + .map_err(|e| format!("Cannot open {p:?}: {e}"))?,
>> + )
>> + .read_to_end(&mut buf)
>> + .map_err(|e| format!("Cannot read {p:?}: {e}"))?;
>> + Ok(buf)
>> +}
>
> Nothing control-file-specific about this method. It just reads a file.
> And a bit odd for write_value to be in cgroup.rs while this is here.
It's only used once, so I'll inline it.
>> +
>> +fn enable_subtree_control(fd: &dyn AsFd) -> Result<(), String> {
>
> Would it not make sense for this to be an instance method on Cgroup,
> since it's a Cgroup-specific operation?
We don't create a Cgroup struct for the child cgroup
FD on which this function is called.
>> + let p = Path::new("cgroup.controllers");
>> + let buf = read_control_file(fd, p)?;
>
> p is only used here, so can just be inlined. If read_control_file took
> AsRef<Path> like the standard library functions do, you wouldn't even
> need to construct the path here.
I will just inline the whole read_control_file function.
It's only used once.
>> + let mut subtree = vec![];
>> + for controller in buf.split(|&b| b == b' ').filter(|e| !e.is_empty()) {
>
> Are there ever likely to be empty works in this file?
No, there will not be unless there is a kernel bug.
>> + if !subtree.is_empty() {
>> + subtree.push(b' ');
>> + }
>> + subtree.push(b'+');
>> + subtree.extend_from_slice(controller);
>> + }
>> + if !subtree.is_empty() {
>> + write_value(&fd, Path::new("cgroup.subtree_control"), &subtree)?;
>> + }
>> + Ok(())
>> +}
>> +
>> +fn cgroup_setup(mut args: ArgsOs) -> Result<(), String> {
>> + let mut leaf = false;
>> + let mut cgroup_path;
>> + let mut systemd_compat = false;
>> + let mut wait = true;
>> + loop {
>> + cgroup_path = args.next();
>> + let Some(ref arg_) = cgroup_path else {
>> + break;
>> + };
>> + let arg_ = arg_.as_bytes();
>> + if arg_ == b"--" {
>> + cgroup_path = args.next();
>
> This cgroup_path thing is a bit complicated. I feel like this could
> probably be cleaned up with a peekable iterator and a while loop.
Easy to fix.
>> + break;
>> + }
>> + if !arg_.starts_with(b"-") {
>> + break;
>> + }
>> +
>> + if !arg_.starts_with(b"--") {
>> + return Err("takes no short options".to_owned());
>> + }
>> +
>> + match &arg_[2..] {
>> + b"leaf" => leaf = true,
>> + b"wait" => wait = true,
>> + b"no-wait" => wait = false,
>> + b"systemd-compat" => systemd_compat = true,
>
> Why do we have --wait and --no-wait, but no --no-leaf or --no-systemd-compat?
Spectrum never needs --wait, so I'll remove it.
>> + arg => return Err(format!("unknown long option {:?}", OsStr::from_bytes(arg))),
>> + }
>> + }
>> + let Some(cgroup_path) = cgroup_path.map(PathBuf::from) else {
>> + return Err("have no positional arguments, expected at least 1".to_owned());
>> + };
>> +
>> + let (parent_cgroup_path, child_cgroup_path) = split_path(&cgroup_path)?;
>> + let cgroup = Cgroup::new(parent_cgroup_path)?;
>> + match rustix::fs::mkdirat(&cgroup, child_cgroup_path, Mode::from_raw_mode(0o755)) {
>> + Ok(()) | Err(Errno::EXIST) => {}
>> + Err(e) => return Err(format!("Cannot make child cgroup: {e}")),
>> + }
>> + let child = cgroup
>> + .open_beneath(child_cgroup_path, OFlags::RDONLY | OFlags::DIRECTORY)
>> + .map_err(|e| format!("Cannot make child cgroup: {e}"))?;
>> + if wait {
>> + // While waiting, only hold an exclusive lock on the child, not the parent.
>> + rustix::fs::flock(&child, FlockOperation::LockExclusive)
>> + .map_err(|e| format!("Cannot take an exclusive lock on child cgroup: {e}"))?;
>> + rustix::fs::flock(&cgroup, FlockOperation::LockShared)
>> + .map_err(|e| format!("Cannot downgrade lock on cgroup to a shared lock: {e}"))?;
>> + Cgroup::wait_for_empty(&child)
>> + .map_err(|e| format!("Cannot wait for {parent_cgroup_path:?} to be empty: {e}"))?;
>> + }
>> + let pid = std::process::id().to_string();
>> + if leaf {
>> + if args.len() != 0 {
>> + // If we aren't delegating any cgroups, don't create a sub-cgroup.
>> + write_value(&child, Path::new("cgroup.procs"), pid.as_bytes())
>> + .map_err(|e| format!("Cannot move process to child cgroup: {e}"))?;
>> + }
>> + } else {
>> + // If the child process will need to manage cgroups itself, it will need
>> + // to set up a sub-cgroup due to the "no internal processes" rule. It's
>> + // simplest to just do it automatically. If the cgroup already exists,
>> + // that isn't an error.
>> + match rustix::fs::mkdirat(&child, cgroup::DEFAULT_LEAF, Mode::from_raw_mode(0o755)) {
>> + Ok(()) | Err(Errno::EXIST) => {}
>> + Err(e) => return Err(format!("Cannot make child cgroup: {e}")),
>> + }
>> + if args.len() != 0 {
>> + let child_proc_path = Path::new(cgroup::DEFAULT_LEAF).join(Path::new("cgroup.procs"));
>> + write_value(&child, &child_proc_path, pid.as_bytes())
>> + .map_err(|e| format!("Cannot move process to child cgroup: {e}"))?;
>> + }
>> + if systemd_compat {
>> + // systemd-aware programs expect to have user.delegate=1
>> + // and to set cgroup.subtree_control themselves
>> + rustix::fs::fsetxattr(&child, c"user.delegate", b"1", XattrFlags::empty()).map_err(
>> + |e| format!("Cannot enable cgroup delegation in {parent_cgroup_path:?}: {e}"),
>> + )?
>> + } else {
>> + // Spectrum's programs do not check for user.delegate=1
>> + // and expect the caller to set cgroup.subtree_control.
>> + enable_subtree_control(&child)?;
>> + }
>> + }
>
> So looking at this I still see several different modes and am wondering
> whether we could simplify this further.
>
> • Why do we need a separate leaf mode? Why not just still use a
> $inner.service in that case?
$inner.service is just wasteful and makes it harder to inspect the cgroup
tree by hand.
> • What would the consequences be if we took the systemd_compat branch
> for a non-cgroup-aware Spectrum program?
Non-cgroup-aware programs would be fine, but nested calls to cgroup-setup
would break because they need the enable_subtree_control() call. However,
in the future, I would like to check the user.delegate xattr to determine
if one can safely write to the control files of the cgroup or if the cgroup
is owned by another program.
>> + let Some(program_name) = args.next() else {
>> + return Ok(());
>> + };
>> + let e = std::process::Command::new(&program_name).args(args).exec();
>> + Err(format!("Cannot spawn child {:?}: {}", program_name, e))
>> +}
>> +
>> +fn cgroup_purge(mut args: ArgsOs) -> Result<(), String> {
>> + if args.len() != 1 {
>> + return Err("usage: cgroup-purge CGROUP_TO_PURGE".to_owned());
>> + }
>> + let arg = args.next().unwrap();
>> + let (parent, child) = split_path(Path::new(&arg))?;
>> + Cgroup::new(parent)?.purge_child(child)
>> +}
>> +
>> +fn run(prog_name: &OsStr, args: ArgsOs) -> Result<(), String> {
>> + match prog_name
>> + .as_bytes()
>> + .split(|&b| b == b'/')
>> + .next_back()
>> + .unwrap()
>
> prog_name.file_name(), where prog_name is &Path?
Works for me :)
>> + {
>> + b"cgroup-setup" => cgroup_setup(args),
>> + b"cgroup-purge" => cgroup_purge(args),
>> + _ => Err(format!(
>> + "must be invoked as \"cgroup-setup\" or \
>> + \"cgroup-purge\", got {prog_name:?}",
>> + )),
>> + }
>> +}
>> +
>> +fn main() {
>> + let mut args = std::env::args_os();
>> + let Some(prog_name) = args.next() else {
>> + eprintln!("No command line arguments (argv[0] is NULL)");
>> + std::process::exit(1);
>> + };
>> + match run(&prog_name, args) {
>> + Ok(()) => {}
>> + Err(e) => {
>> + eprintln!("{prog_name:?}: {}", e);
>> + std::process::exit(1);
>> + }
>> + }
>> +}
>>
>> --
>> 2.55.0
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v5 02/19] tools: Add control group manager
2026-08-05 1:36 ` Demi Marie Obenour
@ 2026-08-05 16:39 ` Alyssa Ross
0 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-08-05 16:39 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 9423 bytes --]
On Tue, Aug 04, 2026 at 09:36:00PM -0400, Demi Marie Obenour wrote:
> On 8/3/26 08:47, Alyssa Ross wrote:
> > Demi Marie Obenour <demiobenour@gmail.com> writes:
> >
> >> + // The rustix source code shows that Dir::new() never fails.
> >> + let child_fd = Rc::new(RefCell::new(Dir::new(fd).unwrap()));
> >> + while let Some(element) = child_fd.borrow_mut().next() {
> >> + let element = element.expect("Iterating through a cgroup directory failed?");
> >> + if element.file_type() != rustix::fs::FileType::Directory {
> >> + continue;
> >> + }
> >> + match element.file_name().to_bytes() {
> >> + b"." | b".." => {}
> >> + other => {
> >> + let other = Path::new(OsStr::from_bytes(other)).to_owned();
> >> + assert_single_component(&other);
> >> + fds.push((child_fd.clone(), other));
> >
> > The data structures used here are still very confusing. Why are we
> > storing a reference to the same file descriptor in every entry in the
> > Vec?
>
> Consider the recursive implementation (in pseudo-Rust):
>
> fn recursive_remove(fd) {
> for entry in get_entries(&fd) {
> if entry.is_dir_and_not_dot_or_dotdot() {
> let directory = open_dir(&fd, &entry.path())?;
> recursive_remove(directory)?;
> remove_dir(&fd, entry.path())?;
> }
> }
> }
>
> The compiler knows that fd will stay open through recursive calls,
> so this doesn't need any unsafe code. Using an explicit stack takes
> away this information from the compiler, so unsafe code is required.
> Using Rc<RefCell<Dir>> avoids the need for unsafe code at a cost
> in performance.
Hmm, but isn't it exactly child_fd that we're storing in the stack every
time? Why store it in the stack at all rather than just using the
child_fd binding that exists for the whole life
> For what it is worth, the standard library implementation of
> fs::remove_dir_all() is recursive. Standard library security hole?
Depends on their security model. Doesn't seem ideal though, unless they
can use unstable features to do tail recursion or something, if that
would even be possible in this case.
> >> + }
> >> + }
> >> + }
> >> +}
> >> +
> >> +// Remove all subdirectories of the given directory recursively,
> >> +// but not the directory itself. The directory file descriptor
> >> +// is closed.
> >> +//
> >> +// This isn't the most efficient possible algorithm, but
> >> +// simplicity is more important than performance in this
> >> +// case. Also, it keeps open more file descriptors than
> >> +// strictly necessary, but Spectrum runs with a very high
> >> +// limit for the number of open file descriptors, and it
> >> +// uses shallow control group hierarchies.
> >> +fn remove_child_directories(dirfd: OwnedFd) -> Result<(), Errno> {
> >> + let mut fds = Vec::new();
> >> + // Push the children of this directory onto the stack.
> >> + push_child_fds(&mut fds, dirfd);
> >> + while let Some((d, path)) = fds.pop() {
> >
> > Couldn't we call push_child_fds() once here, rather than twice as is
> > currently done? (And then consider inlining it, depending on how
> > complex it's looking at the time.)
>
> Can you provide an example? I don't see how to make this change
> while preserving semantics. Only directories meant for deletion
> can appear on the stack, and the root of the traversal must not
> be deleted (yet).
Again I think I probably misunderstood, sorry.
> >> + let sub_fd = cgroup
> >> + .open_beneath(Path::new(component), OFlags::RDONLY | OFlags::DIRECTORY)
> >> + .map_err(|e| format!("Cannot open sub-cgroup {component:?}: {e}"))?;
> >> + // Take a shared lock on the *previous* file descriptor.
> >> + rustix::fs::flock(&cgroup, FlockOperation::LockShared)
> >> + .map_err(|e| format!("Cannot lock sub-cgroup {component:?}: {e}"))?;
> >> + cgroup.fd.push(sub_fd);
> >> + }
> >> + // Take an exclusive lock on the final file descriptor.
> >> + rustix::fs::flock(&cgroup, FlockOperation::LockExclusive)
> >> + .map_err(|e| format!("Cannot lock {path:?}: {e}"))?;
> >> + Ok(cgroup)
> >> + }
> >> +
> >> + pub fn wait_for_empty(fd: &dyn AsFd) -> std::io::Result<()> {
> >> + let wait_file = openat2_simple(fd, Path::new("cgroup.events"), OFlags::RDONLY)?;
> >> + let poll_fd = wait_file.as_raw_fd();
> >> + let mut wait_fd = File::from(wait_file);
> >> + let mut fds = libc::pollfd {
> >> + fd: poll_fd,
> >
> > I would inline poll_fd here. RawFd is easy to misuse, so I like to
> > avoid having them hang around.
>
> I will move `fds` into the inner loop.
I'd still like to see fd: wait_file.as_raw_fd() as well.
> >> + pub fn purge_child(&mut self, path: &Path) -> Result<(), String> {
> >> + assert_single_component(path);
> >> + // See if we can just delete the child directly.
> >> + match rustix::fs::unlinkat(&self, Path::new(path), AtFlags::REMOVEDIR) {
> >> + // If the cgroup was successfully deleted, or if it
> >> + // has already been deleted, we are done.
> >> + Ok(()) | Err(Errno::NOENT) => return Ok(()),
> >> + // If this cgroup is in use, keep going.
> >> + Err(Errno::BUSY) => {}
> >> + Err(e) => return Err(format!("Cannot purge {path:?}: {e}")),
> >> + }
> >> +
> >> + let sub_fd = match self.open_beneath(path, OFlags::RDONLY | OFlags::DIRECTORY) {
> >> + Ok(sub_fd) => sub_fd,
> >> + Err(Errno::NOENT) => return Ok(()),
> >> + Err(e) => {
> >> + return Err(format!("Cannot open sub-cgroup {path:?}: {e}",));
> >> + }
> >> + };
> >> +
> >> + // Take an exclusive lock on the cgroup that is about to be
> >> + // removed. This avoids concurrent executions of this program
> >> + // operating on deleted sub-cgroups.
> >> + rustix::fs::flock(&sub_fd, FlockOperation::LockExclusive)
> >> + .map_err(|e| format!("Cannot lock sub-cgroup: {e}"))?;
> >> +
> >> + // Drop the exclusive lock on the original cgroup,
> >> + // This avoids blocking concurrent operations on other
> >> + // child cgroups while the cgroup is being purged,
> >> + // or while waiting for programs to exit.
> >> + rustix::fs::flock(&self, FlockOperation::LockShared)
> >> + .map_err(|e| format!("Cannot relock: {e}"))?;
> >
> > Could you add some extra explanation here of why it's okay for the
> > exclusive lock to be temporarily dropped here?
> >
> > I'm wondering whether taking a lock, then dropping it temporarily is a
> > sign that we're taking the lock too early in the first place, and should
> > scope it better to where it's actually needed.
>
> Indeed so. Programs that are modifying a cgroup need an exclusive
> lock on it. Adding or removing to the cgroup does *not* count as
> modification: both operations are idempotent, removing an in-use
> cgroup fails with -EBUSY, and operating on a deleted cgroup fails
> with -ENODEV or -ENOENT depending on what one is doing. Operations on
> control files *do* require an exclusive lock.
Good, let's have that written down somehow. Preferably it'd be encoded
in the type system but I don't know if that's easily achievable.
> >> +
> >> +fn enable_subtree_control(fd: &dyn AsFd) -> Result<(), String> {
> >
> > Would it not make sense for this to be an instance method on Cgroup,
> > since it's a Cgroup-specific operation?
>
> We don't create a Cgroup struct for the child cgroup
> FD on which this function is called.
But we could!
> >> + let mut subtree = vec![];
> >> + for controller in buf.split(|&b| b == b' ').filter(|e| !e.is_empty()) {
> >
> > Are there ever likely to be empty works in this file?
>
> No, there will not be unless there is a kernel bug.
Right, so then we don't need the filter?
> > So looking at this I still see several different modes and am wondering
> > whether we could simplify this further.
> >
> > • Why do we need a separate leaf mode? Why not just still use a
> > $inner.service in that case?
>
> $inner.service is just wasteful and makes it harder to inspect the cgroup
> tree by hand.
It can't be that wasteful, can it? Surely cgroups are designed to
scale. I'd rather have the consistency.
> > • What would the consequences be if we took the systemd_compat branch
> > for a non-cgroup-aware Spectrum program?
>
> Non-cgroup-aware programs would be fine, but nested calls to cgroup-setup
> would break because they need the enable_subtree_control() call. However,
> in the future, I would like to check the user.delegate xattr to determine
> if one can safely write to the control files of the cgroup or if the cgroup
> is owned by another program.
And it wouldn't be correct to write to cgroup.subtree_control in all
cases? Programs that expect cgroup delegation expect it to start empty,
and so won't disable controllers that they need to be disabled? Or
would that be fine?
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* [PATCH v6 00/19] Control group support
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
` (18 preceding siblings ...)
2026-07-31 21:54 ` [PATCH v5 19/19] host/rootfs: spectrum-router: " Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 01/19] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
` (19 more replies)
19 siblings, 20 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross, Valentin Gagarin
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Changes in v6:
- Simplify command-line argument parsing.
- Place SPDX-FileCopyrightText before SPDX-License-Identifier, except in
patches that already have Reviewed-by tags.
- Clean up cgroup-setup.
- Make openat2_simple() take an enum instead of a flags argument.
- Use recursive function to remove directories. The standard library
remove_dir_all() function is recursive, and the complexity of being
more robust than the standard library is not worth it here. Spectrum
does not allow code running as non-root to create cgroups, so there is
no security concern.
- Link to v5: https://spectrum-os.org/lists/archives/spectrum-devel/20260731-cgroups-v5-0-b325bac9d34f@gmail.com
Changes in v5:
- Rename open_subtree_raw() to open_beneath().
- Use consistent file modes.
- Avoid using O_NOFOLLOW when RESOLVE_NO_SYMLINKS is also used.
- Drop tracking of specific cgroup paths (at the cost of worse error messages).
- Have functions take &dyn AsFd where that makes sense.
- Drop exclusive vs shared lock tracking.
- Remove vm-service-run wrapper script.
- Use explicit cgroup names in VM service run and finish scripts.
- Use sed to write to cgroup.subtree_control in the root cgroup,
avoiding a special case in cgroup-setup.
- Avoid mutating the Cgroup struct when creating child cgroups.
- Avoid mutating the Cgroup struct when purging cgroups.
- Improve documentation.
- Drop cgroup-s6-finish and call cgroup-purge directly.
- Remove support for operating on . or / in cgroup-setup.
- Fix comments.
- Link to v4: https://spectrum-os.org/lists/archives/spectrum-devel/20260721-cgroups-v4-0-46b2e5fff7b6@gmail.com
Changes in v4:
- Implement proper locking to make concurrent operations safe.
- Purge VMM cgroup in vmm service finish script.
- Delete /run/vsock/${VM}/vsock before running Cloud Hypervisor.
- Massively refactor cgroup-setup tool.
- Link to v3: https://spectrum-os.org/lists/archives/spectrum-devel/20260711-cgroups-v3-1-5cba61a20cba@gmail.com
Changes in v3:
- Remove the implicit .service suffix on leaf control groups.
- Make cgroup-setup acts as an s6 finish script when called as "finish".
- Put the VMMs in the same cgroups as the per-VM services.
- Add common helper script for the per-VM services.
- Significantly refactor the Rust code.
- Link to v2: https://spectrum-os.org/lists/archives/spectrum-devel/20260620-cgroups-v2-1-ccae224b6c85@gmail.com
Changes in v2:
- Omit resource control support. It was completely broken, and the way
Spectrum run scripts work means that it is better to setup resource
controls just before the execve() into the final service process.
This will be done by a separate tool.
- Link to v1: https://spectrum-os.org/lists/archives/spectrum-devel/20260620-cgroups-v1-1-0e5abf35101b@gmail.com
---
Demi Marie Obenour (19):
host/rootfs: Mount filesystems before s6-rc-init
tools: Add control group manager
Documentation: Mention control groups
Mount cgroup2 filesystem at /sys/fs/cgroup
host/rootfs: Enable controllers in non-root cgroups
host/rootfs: Add comments where cgroups are intentionally not used
host/rootfs: serial-getty-generator: Use cgroups
host/rootfs: systemd-udevd: Run in cgroup
host/rootfs: weston: Run in cgroup
host/rootfs: Set up parent cgroup for all per-VM services
host/rootfs: Create per-VM cgroup for all of the VM's services
host/rootfs: run-vmm: Create per-VM cgroup
host/rootfs: run-appimage: Purge the per-VM cgroup
host/rootfs: run-flatpak: Purge the per-VM cgroup
host/rootfs: dbus: Run in cgroup
host/rootfs: vhost-user-fs: Run in cgroup
host/rootfs: vhost-user-gpu: Run in cgroup
host/rootfs: xdg-desktop-portal-spectrum-host: Run in cgroup
host/rootfs: spectrum-router: Run in cgroup
.codespellrc | 2 +-
Documentation/doc/development/control-groups.adoc | 95 ++++++++
host/rootfs/default.nix | 6 +-
host/rootfs/file-list.mk | 10 +
host/rootfs/image/etc/fstab | 1 +
host/rootfs/image/etc/init | 14 +-
.../s6-linux-init/run-image/service/getty-tty2/run | 1 +
.../s6-linux-init/run-image/service/getty-tty3/run | 1 +
.../s6-linux-init/run-image/service/getty-tty4/run | 1 +
.../run-image/service/root-terminal/run | 1 +
.../run-image/service/s6-linux-init-shutdownd/run | 1 +
.../run-image/service/s6-svscan-log/run | 1 +
.../service/serial-getty-generator/finish | 5 +
.../run-image/service/serial-getty-generator/run | 4 +-
.../run-image/service/serial-getty/run | 1 +
.../run-image/service/serial-getty/template/run | 1 +
.../run-image/service/vm-services/run | 3 +
.../vm-services/template/data/service/dbus/finish | 5 +
.../vm-services/template/data/service/dbus/run | 2 +
.../template/data/service/spectrum-router/finish | 5 +
.../template/data/service/spectrum-router/run | 2 +
.../template/data/service/vhost-user-fs/finish | 5 +
.../template/data/service/vhost-user-fs/run | 5 +-
.../template/data/service/vhost-user-gpu/finish | 5 +
.../template/data/service/vhost-user-gpu/run | 2 +
.../xdg-desktop-portal-spectrum-host/finish | 5 +
.../service/xdg-desktop-portal-spectrum-host/run | 2 +
.../run-image/service/vm-services/template/finish | 5 +
.../run-image/service/vm-services/template/run | 4 +
.../run-image/service/vmm/template/finish | 5 +
.../rootfs/image/etc/s6-linux-init/scripts/rc.init | 7 -
host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +
host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +-
host/rootfs/image/etc/s6-rc/weston/finish | 5 +
host/rootfs/image/etc/s6-rc/weston/run | 2 +
host/rootfs/image/usr/bin/run-appimage | 7 +-
host/rootfs/image/usr/bin/run-flatpak | 7 +-
host/rootfs/image/usr/bin/run-vmm | 4 +
host/rootfs/image/usr/bin/vm-stop | 5 +-
pkgs/default.nix | 1 +
tools/cgroup-setup/Cargo.lock | 67 +++++
tools/cgroup-setup/Cargo.lock.license | 2 +
tools/cgroup-setup/Cargo.toml | 10 +
tools/cgroup-setup/default.nix | 22 ++
tools/cgroup-setup/src/cgroup.rs | 269 +++++++++++++++++++++
tools/cgroup-setup/src/main.rs | 167 +++++++++++++
46 files changed, 765 insertions(+), 20 deletions(-)
---
base-commit: a7762d6f54b40560dd5255ce902e6e6a5d980fe9
change-id: 20260528-cgroups-d609e270e649
--
Sincerely,
Demi Marie Obenour (she/her/hers)
^ permalink raw reply [flat|nested] 132+ messages in thread
* [PATCH v6 01/19] host/rootfs: Mount filesystems before s6-rc-init
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 02/19] tools: Add control group manager Demi Marie Obenour
` (18 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
cgroup-setup, a tool which will be introduced in a later patch, is
written in Rust and uses the Rust standard library. It also relies on
cgroupfs being mounted at /sys/fs/cgroup. Furthermore, it runs very,
*very* early on, since it sets up the cgroups in which s6-svscan and
s6-supervise processes run. This allows setting up sub-cgroups without
having to worry about name collisions.
While it would be possible to only mount the bare minimum necessary,
it's simpler to just mount everything before starting any services.
These mounts are all pseudo-filesystems or tmpfs instances, and none
have any dependencies.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/image/etc/init | 11 ++++++++++-
host/rootfs/image/etc/s6-linux-init/scripts/rc.init | 7 -------
2 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
index db8a6d9c747e3d212c21b6c983c8f7e299822abe..8ec7859ab00e24007a93d4000c8f34e353de50ae 100755
--- a/host/rootfs/image/etc/init
+++ b/host/rootfs/image/etc/init
@@ -1,6 +1,15 @@
#!/bin/execlineb -Ws0
# SPDX-License-Identifier: EUPL-1.2+
-# SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2020-2022, 2024 Alyssa Ross <hi@alyssa.is>
/bin/setpriv --no-new-privs --
+
+if { /bin/ln -s /proc/self/fd /dev }
+if { /bin/ln -s /proc/self/fd/0 /dev/stdin }
+if { /bin/ln -s /proc/self/fd/1 /dev/stdout }
+if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
+
+if { /bin/mount --make-shared / }
+if { /bin/mount -a --mkdir }
+
/bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
diff --git a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
index f638e373589884acb959c868462fdf532380a851..db766ba0ca0a11982f5b55497c17eaf5e95afb2c 100755
--- a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
+++ b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init
@@ -4,13 +4,6 @@
if { s6-rc-init -c /etc/s6-rc /run/service }
-if { ln -s /proc/self/fd /dev }
-if { ln -s /proc/self/fd/0 /dev/stdin }
-if { ln -s /proc/self/fd/1 /dev/stdout }
-if { ln -s /proc/self/fd/2 /dev/stderr }
-
-if { mount --make-shared / }
if { mount --make-shared /run }
-if { mount -a --mkdir }
s6-rc change ok-all
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 02/19] tools: Add control group manager
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 01/19] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 6:58 ` Demi Marie Obenour
2026-08-12 21:10 ` Alyssa Ross
2026-08-06 1:16 ` [PATCH v6 03/19] Documentation: Mention control groups Demi Marie Obenour
` (17 subsequent siblings)
19 siblings, 2 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
The cgroup-setup Rust program can create and purge cgroups. It can also
wait for one to become empty, spawn a program in a cgroup, and more. In
the future, it will also support cgroup-based resource control. Locking
is used to ensure that concurrent invocations are safe.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
.codespellrc | 2 +-
host/rootfs/default.nix | 6 +-
pkgs/default.nix | 1 +
tools/cgroup-setup/Cargo.lock | 67 +++++++++
tools/cgroup-setup/Cargo.lock.license | 2 +
tools/cgroup-setup/Cargo.toml | 10 ++
tools/cgroup-setup/default.nix | 22 +++
tools/cgroup-setup/src/cgroup.rs | 269 ++++++++++++++++++++++++++++++++++
tools/cgroup-setup/src/main.rs | 167 +++++++++++++++++++++
9 files changed, 542 insertions(+), 4 deletions(-)
diff --git a/.codespellrc b/.codespellrc
index d8023afc64ec44e98a88c5e397c8d5c681dde063..ae20da8309530759ac729689825a4afc683afa09 100644
--- a/.codespellrc
+++ b/.codespellrc
@@ -2,4 +2,4 @@
# SPDX-License-Identifier: CC0-1.0
[codespell]
-ignore-words-list = crate,passt,rouge,ser
+ignore-words-list = crate,passt,rouge,ser,WRONLY
diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix
index 6bfeefbe0a5f76c1538ccb40e5eb8f291f5d3592..ccf626e2ec0f4bf96573dc5edf058c9375bb65f6 100644
--- a/host/rootfs/default.nix
+++ b/host/rootfs/default.nix
@@ -8,7 +8,7 @@ import ../../lib/call-package.nix (
}:
pkgsMusl.callPackage (
-{ spectrum-host-tools, spectrum-router
+{ spectrum-host-tools, spectrum-router, spectrum-cgroup-setup
, lib, stdenvNoCC, nixos, runCommand, writeClosure, erofs-utils, s6-rc
, btrfs-progs, bubblewrap, busybox, cloud-hypervisor, cosmic-files
, crosvm, cryptsetup, dejavu_fonts, dbus, execline, foot, fuse3
@@ -27,8 +27,8 @@ let
packages = [
btrfs-progs bubblewrap cloud-hypervisor cosmic-files crosvm cryptsetup dbus
execline fuse3 inotify-tools iproute2 jq kmod mdevd mount-flatpak s6
- s6-linux-init s6-rc shadow socat spectrum-host-tools spectrum-router
- virtiofsd xdg-desktop-portal-spectrum-host
+ s6-linux-init s6-rc shadow socat spectrum-cgroup-setup spectrum-host-tools
+ spectrum-router virtiofsd xdg-desktop-portal-spectrum-host
(foot.override { allowPgo = false; })
diff --git a/pkgs/default.nix b/pkgs/default.nix
index 44f7b5ff78cb6b9e755292a6a417d0b627ed3fb0..0a13393164ad5d7f752e630763f3f97166479af5 100644
--- a/pkgs/default.nix
+++ b/pkgs/default.nix
@@ -51,6 +51,7 @@ let
driverSupport = true;
};
spectrum-router = self.callSpectrumPackage ../tools/router {};
+ spectrum-cgroup-setup = self.callSpectrumPackage ../tools/cgroup-setup {};
xdg-desktop-portal-spectrum-host =
self.callSpectrumPackage ../tools/xdg-desktop-portal-spectrum-host {};
diff --git a/tools/cgroup-setup/Cargo.lock b/tools/cgroup-setup/Cargo.lock
new file mode 100644
index 0000000000000000000000000000000000000000..fe967b3aa02c296c87b6b36ac59253dbe0a32de9
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.lock
@@ -0,0 +1,67 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 4
+
+[[package]]
+name = "bitflags"
+version = "2.11.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3"
+
+[[package]]
+name = "cgroup-setup"
+version = "0.0.0"
+dependencies = [
+ "libc",
+ "rustix",
+]
+
+[[package]]
+name = "errno"
+version = "0.3.14"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
+dependencies = [
+ "libc",
+ "windows-sys",
+]
+
+[[package]]
+name = "libc"
+version = "0.2.186"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
+
+[[package]]
+name = "linux-raw-sys"
+version = "0.12.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
+
+[[package]]
+name = "rustix"
+version = "1.1.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
+dependencies = [
+ "bitflags",
+ "errno",
+ "libc",
+ "linux-raw-sys",
+ "windows-sys",
+]
+
+[[package]]
+name = "windows-link"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
+
+[[package]]
+name = "windows-sys"
+version = "0.61.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
+dependencies = [
+ "windows-link",
+]
diff --git a/tools/cgroup-setup/Cargo.lock.license b/tools/cgroup-setup/Cargo.lock.license
new file mode 100644
index 0000000000000000000000000000000000000000..f80078163f4428881b86cb6fef0c90c0c1ebaa91
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.lock.license
@@ -0,0 +1,2 @@
+SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+SPDX-License-Identifier: CC0-1.0
diff --git a/tools/cgroup-setup/Cargo.toml b/tools/cgroup-setup/Cargo.toml
new file mode 100644
index 0000000000000000000000000000000000000000..e44ef64119aefc23a3119fe0ce01566f5d7ec98c
--- /dev/null
+++ b/tools/cgroup-setup/Cargo.toml
@@ -0,0 +1,10 @@
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# SPDX-License-Identifier: CC0-1.0
+
+[package]
+name = "cgroup-setup"
+edition = "2024"
+
+[dependencies]
+libc = "0.2.177"
+rustix = { version = "1.1.2", features = ["fs"] }
diff --git a/tools/cgroup-setup/default.nix b/tools/cgroup-setup/default.nix
new file mode 100644
index 0000000000000000000000000000000000000000..fe3a8bcd4e6118f3cbf49c2fc0e3cda01570c484
--- /dev/null
+++ b/tools/cgroup-setup/default.nix
@@ -0,0 +1,22 @@
+# SPDX-FileCopyrightText: 2024 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
+# SPDX-License-Identifier: MIT
+
+import ../../lib/call-package.nix (
+{ src, lib, rustPlatform }:
+
+rustPlatform.buildRustPackage {
+ name = "spectrum-cgroup-setup";
+
+ src = lib.fileset.toSource {
+ root = ../..;
+ fileset = lib.fileset.intersection src ./.;
+ };
+ sourceRoot = "source/tools/cgroup-setup";
+
+ cargoLock.lockFile = ./Cargo.lock;
+
+ postInstall = ''
+ ln -s -- cgroup-setup "$out/bin/cgroup-purge"
+ '';
+}) (_: {})
diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs
new file mode 100644
index 0000000000000000000000000000000000000000..c63d7e5a4aa73429578704401c58bbafc79e7c3f
--- /dev/null
+++ b/tools/cgroup-setup/src/cgroup.rs
@@ -0,0 +1,269 @@
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+// SPDX-License-Identifier: EUPL-1.2+
+
+use std::ffi::OsStr;
+use std::fs::File;
+use std::io::{Read as _, Seek as _, Write as _};
+use std::os::unix::prelude::*;
+
+use std::path::{Component, Path, PathBuf};
+
+use rustix::fs::{AtFlags, CWD, Dir, FlockOperation};
+use rustix::path;
+use rustix::{
+ fs::{Mode, OFlags, ResolveFlags},
+ io::Errno,
+};
+
+pub enum OpenFlags {
+ Read,
+ Write,
+ Directory,
+}
+
+#[derive(Debug)]
+pub(crate) struct Cgroup {
+ fd: Vec<OwnedFd>,
+}
+
+impl AsFd for Cgroup {
+ fn as_fd(&self) -> BorrowedFd<'_> {
+ self.fd.last().unwrap().as_fd()
+ }
+}
+
+fn assert_single_component(component: &Path) {
+ match component.as_os_str().as_bytes() {
+ b"" | b"." | b".." => panic!("bad component"),
+ c if c.contains(&b'\0') => panic!("NUL in component"),
+ c if c.contains(&b'/') => panic!("/ in component"),
+ _ => {}
+ }
+}
+
+// Wrapper around openat2() with better defaults.
+pub fn openat2_simple(
+ fd: impl AsFd,
+ path: impl path::Arg,
+ flags: OpenFlags,
+) -> Result<OwnedFd, Errno> {
+ rustix::fs::openat2(
+ fd.as_fd(),
+ path,
+ OFlags::CLOEXEC
+ | match flags {
+ OpenFlags::Read => OFlags::RDONLY | OFlags::NOCTTY,
+ OpenFlags::Write => OFlags::WRONLY | OFlags::NOCTTY,
+ OpenFlags::Directory => OFlags::RDONLY | OFlags::DIRECTORY,
+ },
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_XDEV,
+ )
+}
+
+pub const DEFAULT_LEAF: &str = "$inner.service";
+
+pub fn check_path(path: &Path) -> Result<(), String> {
+ let bytes = path.as_os_str().as_bytes();
+ // Path::components() skips ., so use string manipulation instead.
+ for component in bytes[path.is_absolute() as usize..].split(|&b| b == b'/') {
+ if matches!(component, b"" | b"." | b"..") {
+ return Err(format!("cgroup path {path:?} isn't canonical"));
+ }
+ }
+ Ok(())
+}
+
+// Remove all subdirectories of the given directory recursively,
+// but not the directory itself. The directory file descriptor
+// is closed.
+//
+// This isn't the most efficient possible algorithm, but
+// simplicity is more important than performance in this
+// case. Also, it keeps open more file descriptors than
+// strictly necessary, but Spectrum runs with a very high
+// limit for the number of open file descriptors, and it
+// uses shallow control group hierarchies.
+//
+// This uses a recursive algorith, but so does std::fs::remove_dir_all().
+// Trying to be more robust than the standard library is not worthwhile.
+// In particular, the standard library function must be safe on systems
+// where untrusted users (or even network endpoints!) can create deeply
+// nested directory trees, whereas in Spectrum cgroups are only writeable
+// by root.
+fn remove_recursively(mut dirfd: Dir, remaining_depth: usize) -> Result<(), Errno> {
+ if remaining_depth < 1 {
+ panic!("control groups too deeply nested");
+ }
+ while let Some(element) = dirfd.next() {
+ let parent_fd = dirfd.fd().unwrap();
+ let element = element.expect("Iterating through a cgroup directory failed?");
+ let path = element.file_name();
+ if element.file_type() != rustix::fs::FileType::Directory || path == c"." || path == c".." {
+ continue;
+ }
+ let fd = openat2_simple(parent_fd, path, OpenFlags::Directory)?;
+ remove_recursively(Dir::new(fd).unwrap(), remaining_depth - 1)?;
+ match rustix::fs::unlinkat(parent_fd, path, AtFlags::REMOVEDIR) {
+ Err(Errno::NOTEMPTY | Errno::BUSY | Errno::NOENT) | Ok(()) => {}
+ bad => return bad,
+ }
+ }
+ Ok(())
+}
+
+// If the path is absolute, make it relative.
+// Otherwise, read the current cgroup from /proc/thread-self/cgroup
+// and prepend it to the path.
+fn prepend_current_cgroup_if_needed(path: &Path) -> PathBuf {
+ if let Ok(suffix) = path.strip_prefix("/") {
+ suffix.to_owned()
+ } else {
+ // /proc/thread-self is the same as /proc/self, except for the current
+ // thread instead of the initial thread. In this case, the two are
+ // identical, but using /proc/thread-self is better practice as it is
+ // correct in more cases. Reading /proc/thread-self/cgroup should
+ // never fail unless the system is seriously broken.
+ let current_cgroup = std::fs::read("/proc/thread-self/cgroup")
+ .expect("cannot read /proc/thread-self/cgroup");
+ // Using this on a system without cgroups v2 mounted is user error
+ // and not supported.
+ let current_cgroup = current_cgroup
+ .strip_prefix(b"0::/")
+ .and_then(|e| e.strip_suffix(b"\n"))
+ .expect("you don't have cgroups v2 mounted");
+ let mut current_cgroup = PathBuf::from(OsStr::from_bytes(current_cgroup));
+ // Strip the implied $inner.service suffix.
+ // This is used to satisfy the "no internal processes" rule.
+ if current_cgroup.ends_with(Path::new(DEFAULT_LEAF)) {
+ assert!(current_cgroup.pop());
+ }
+ current_cgroup.push(path);
+ current_cgroup
+ }
+}
+
+pub(crate) fn write_value(fd: &dyn AsFd, name: &Path, value: &[u8]) -> Result<(), String> {
+ let fd = openat2_simple(fd, name, OpenFlags::Write)
+ .map_err(|e| format!("Cannot open {name:?}: {e}"))?;
+ File::from(fd).write_all(value).map_err(|e| {
+ format!(
+ "Cannot write {:?} to {name:?}: {e}",
+ OsStr::from_bytes(value)
+ )
+ })
+}
+
+impl Cgroup {
+ pub fn new(path: &Path) -> Result<Self, String> {
+ let cgroup_root = rustix::fs::openat2(
+ CWD,
+ Path::new("/sys/fs/cgroup"),
+ OFlags::CLOEXEC | OFlags::DIRECTORY | OFlags::RDONLY,
+ Mode::empty(),
+ ResolveFlags::NO_SYMLINKS | ResolveFlags::NO_MAGICLINKS,
+ )
+ .map_err(|e| format!("Cannot open /sys/fs/cgroup: {e}"))?;
+ let mut cgroup = Self {
+ fd: vec![(cgroup_root)],
+ };
+
+ let path = prepend_current_cgroup_if_needed(path);
+ for component in path.components() {
+ let Component::Normal(component) = component else {
+ unreachable!()
+ };
+ let sub_fd = openat2_simple(&cgroup, component, OpenFlags::Directory)
+ .map_err(|e| format!("Cannot open sub-cgroup {component:?}: {e}"))?;
+ // Take a shared lock on the cgroup.
+ rustix::fs::flock(&sub_fd, FlockOperation::LockShared)
+ .map_err(|e| format!("Cannot lock sub-cgroup {component:?}: {e}"))?;
+ cgroup.fd.push(sub_fd);
+ }
+ Ok(cgroup)
+ }
+
+ pub fn wait_for_empty(fd: &dyn AsFd) -> std::io::Result<()> {
+ let wait_file = openat2_simple(fd, c"cgroup.events", OpenFlags::Read)?;
+ let mut wait_fd = File::from(wait_file);
+ let mut v = vec![];
+ loop {
+ v.clear();
+ wait_fd
+ .seek(std::io::SeekFrom::Start(0))
+ .expect("Seek on control group file should succeed");
+ wait_fd
+ .read_to_end(&mut v)
+ .expect("reading from control group should work");
+ // Check that the cgroup isn't already empty. If it was,
+ // the kernel would not send an event and poll() would wait
+ // forever.
+ if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
+ break;
+ }
+ let mut fds = libc::pollfd {
+ fd: wait_fd.as_raw_fd(),
+ events: libc::POLLPRI | libc::POLLERR,
+ revents: 0,
+ };
+ // SAFETY: FFI call, valid arguments, fds contains 1 element
+ if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
+ panic!("poll failed");
+ }
+ }
+ drop(wait_fd);
+ Ok(())
+ }
+
+ pub fn purge_child(&mut self, path: &Path) -> Result<(), String> {
+ assert_single_component(path);
+ // See if we can just delete the child directly.
+ match rustix::fs::unlinkat(&self, Path::new(path), AtFlags::REMOVEDIR) {
+ // If the cgroup was successfully deleted, or if it
+ // has already been deleted, we are done.
+ Ok(()) | Err(Errno::NOENT) => return Ok(()),
+ // If this cgroup is in use, keep going.
+ Err(Errno::BUSY) => {}
+ Err(e) => return Err(format!("Cannot purge {path:?}: {e}")),
+ }
+
+ let sub_fd = match openat2_simple(&self, path, OpenFlags::Directory) {
+ Ok(sub_fd) => sub_fd,
+ Err(Errno::NOENT) => return Ok(()),
+ Err(e) => {
+ return Err(format!("Cannot open sub-cgroup {path:?}: {e}",));
+ }
+ };
+
+ // Take an exclusive lock on the cgroup that is about to be
+ // removed. This avoids concurrent executions of this program
+ // operating on deleted sub-cgroups.
+ rustix::fs::flock(&sub_fd, FlockOperation::LockExclusive)
+ .map_err(|e| format!("Cannot lock sub-cgroup: {e}"))?;
+
+ // Kill all processes in the child cgroup.
+ write_value(&sub_fd, Path::new("cgroup.kill"), b"1")?;
+
+ // Wait for the child cgroup to become empty.
+ Self::wait_for_empty(&sub_fd)
+ .map_err(|e| format!("Cannot wait for cgroup to become empty: {e}"))
+ .inspect_err(|_| {
+ self.fd.pop().unwrap();
+ })?;
+
+ // Remove the child cgroup and its contents recursively.
+ remove_recursively(Dir::new(sub_fd).unwrap(), 1000)
+ .map_err(|e| format!("Cannot remove: {e}"))?;
+
+ // Delete the cgroup. If it's been re-created in the meantime
+ // and is currently in use, this is not an error. Another
+ // process deleting the cgroup is also not an error. Both of
+ // these can happen because of the time period between
+ // remove_child_directories() closing the file descriptor
+ // (releasing its lock) and the above call to flock().
+ match rustix::fs::unlinkat(&self, path, AtFlags::REMOVEDIR) {
+ Ok(()) | Err(Errno::BUSY) | Err(Errno::NOENT) => Ok(()),
+ Err(e) => Err(format!("Cannot delete: {e}")),
+ }
+ }
+}
diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
new file mode 100644
index 0000000000000000000000000000000000000000..58ca7295bb348a2c92640a68c60b285b2d7a1494
--- /dev/null
+++ b/tools/cgroup-setup/src/main.rs
@@ -0,0 +1,167 @@
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+// SPDX-License-Identifier: EUPL-1.2+
+
+mod cgroup;
+
+use cgroup::{Cgroup, OpenFlags, openat2_simple, write_value};
+use rustix::{
+ fs::{FlockOperation, Mode, XattrFlags},
+ io::Errno,
+};
+use std::{
+ env::ArgsOs,
+ fs::File,
+ io::Read as _,
+ os::unix::prelude::*,
+ path::{Path, PathBuf},
+};
+
+// Check that the path is canonical,
+// then split it into basename and filename.
+fn split_path(path: &Path) -> Result<(&Path, &Path), String> {
+ cgroup::check_path(path)?;
+ Ok((path.parent().unwrap(), Path::new(path.file_name().unwrap())))
+}
+
+fn enable_subtree_control(fd: &dyn AsFd) -> Result<(), String> {
+ let mut buf = Vec::new();
+ File::from(
+ openat2_simple(fd, c"cgroup.controllers", OpenFlags::Read)
+ .map_err(|e| format!("Cannot open cgroup.controllers: {e}"))?,
+ )
+ .read_to_end(&mut buf)
+ .map_err(|e| format!("Cannot read cgroup.controllers: {e}"))?;
+ let mut subtree = vec![];
+ for controller in buf.split(|&b| b == b' ') {
+ if !subtree.is_empty() {
+ subtree.push(b' ');
+ }
+ subtree.push(b'+');
+ subtree.extend_from_slice(controller);
+ }
+ if !subtree.is_empty() {
+ write_value(&fd, Path::new("cgroup.subtree_control"), &subtree)?;
+ }
+ Ok(())
+}
+
+fn cgroup_setup(args: ArgsOs) -> Result<(), String> {
+ let mut leaf = false;
+ let mut systemd_delegate = false;
+ let mut wait = true;
+ let mut args = args.peekable();
+ while let Some(arg) = args.peek() {
+ if !arg.as_bytes().starts_with(b"-") {
+ break;
+ }
+ let arg = args.next().unwrap();
+ let Some(arg_) = arg.as_bytes().strip_prefix(b"--") else {
+ return Err("takes no short options".to_owned());
+ };
+ match arg_ {
+ b"" => break,
+ b"leaf" => leaf = true,
+ b"no-wait" => wait = false,
+ b"systemd-delegate" => systemd_delegate = true,
+ _ => return Err(format!("unknown long option {arg:?}")),
+ }
+ }
+ let Some(cgroup_path) = args.next().map(PathBuf::from) else {
+ return Err("have no positional arguments, expected at least 1".to_owned());
+ };
+
+ let (parent_cgroup_path, child_cgroup_path) = split_path(&cgroup_path)?;
+ let cgroup = Cgroup::new(parent_cgroup_path)?;
+ match rustix::fs::mkdirat(&cgroup, child_cgroup_path, Mode::from_raw_mode(0o755)) {
+ Ok(()) | Err(Errno::EXIST) => {}
+ Err(e) => {
+ return Err(format!(
+ "Cannot make child cgroup {child_cgroup_path:?}: {e}"
+ ));
+ }
+ }
+ let child = openat2_simple(&cgroup, child_cgroup_path, OpenFlags::Directory)
+ .map_err(|e| format!("Cannot make child cgroup: {e}"))?;
+ // While waiting, hold an exclusive lock on the child.
+ // This avoids two processes both waiting for the same cgroup to become
+ // empty, then spawning processes in the same cgroup.
+ rustix::fs::flock(&child, FlockOperation::LockExclusive)
+ .map_err(|e| format!("Cannot take an exclusive lock on child cgroup: {e}"))?;
+ if wait {
+ Cgroup::wait_for_empty(&child)
+ .map_err(|e| format!("Cannot wait for {parent_cgroup_path:?} to be empty: {e}"))?;
+ }
+ let pid = std::process::id().to_string();
+ if leaf {
+ if args.len() != 0 {
+ // If we aren't delegating any cgroups, don't create a sub-cgroup.
+ write_value(&child, Path::new("cgroup.procs"), pid.as_bytes())
+ .map_err(|e| format!("Cannot move process to child cgroup: {e}"))?;
+ }
+ } else {
+ // If the child process will need to manage cgroups itself, it will need
+ // to set up a sub-cgroup due to the "no internal processes" rule. It's
+ // simplest to just do it automatically. If the cgroup already exists,
+ // that isn't an error.
+ match rustix::fs::mkdirat(&child, cgroup::DEFAULT_LEAF, Mode::from_raw_mode(0o755)) {
+ Ok(()) | Err(Errno::EXIST) => {}
+ Err(e) => return Err(format!("Cannot make child cgroup: {e}")),
+ }
+ if args.len() != 0 {
+ let child_proc_path = Path::new(cgroup::DEFAULT_LEAF).join(Path::new("cgroup.procs"));
+ write_value(&child, &child_proc_path, pid.as_bytes())
+ .map_err(|e| format!("Cannot move process to child cgroup: {e}"))?;
+ }
+ if systemd_delegate {
+ // systemd-aware programs expect to have user.delegate=1
+ // and to set cgroup.subtree_control themselves
+ rustix::fs::fsetxattr(&child, c"user.delegate", b"1", XattrFlags::empty()).map_err(
+ |e| format!("Cannot enable cgroup delegation in {parent_cgroup_path:?}: {e}"),
+ )?
+ } else {
+ // Spectrum's programs do not check for user.delegate=1
+ // and expect the caller to set cgroup.subtree_control.
+ enable_subtree_control(&child)?;
+ }
+ }
+ let Some(program_name) = args.next() else {
+ return Ok(());
+ };
+ let e = std::process::Command::new(&program_name).args(args).exec();
+ Err(format!("Cannot spawn child {program_name:?}: {e}",))
+}
+
+fn cgroup_purge(mut args: ArgsOs) -> Result<(), String> {
+ if args.len() != 1 {
+ return Err("usage: cgroup-purge CGROUP_TO_PURGE".to_owned());
+ }
+ let arg = args.next().unwrap();
+ let (parent, child) = split_path(Path::new(&arg))?;
+ Cgroup::new(parent)?.purge_child(child)
+}
+
+fn run(prog_name: &Path, args: ArgsOs) -> Result<(), String> {
+ match prog_name.file_name().map(|f| f.as_bytes()) {
+ Some(b"cgroup-setup") => cgroup_setup(args),
+ Some(b"cgroup-purge") => cgroup_purge(args),
+ _ => Err(format!(
+ "must be invoked as \"cgroup-setup\" or \
+ \"cgroup-purge\", got {prog_name:?}",
+ )),
+ }
+}
+
+fn main() {
+ let mut args = std::env::args_os();
+ let Some(prog_name) = args.next() else {
+ eprintln!("No command line arguments (argv[0] is NULL)");
+ std::process::exit(1);
+ };
+ match run(Path::new(&prog_name), args) {
+ Ok(()) => {}
+ Err(e) => {
+ eprintln!("{prog_name:?}: {}", e);
+ std::process::exit(1);
+ }
+ }
+}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 03/19] Documentation: Mention control groups
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 01/19] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 02/19] tools: Add control group manager Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 04/19] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
` (16 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross, Valentin Gagarin
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Cc: Valentin Gagarin <valentin@gagarin.work>
---
Documentation/doc/development/control-groups.adoc | 95 +++++++++++++++++++++++
1 file changed, 95 insertions(+)
diff --git a/Documentation/doc/development/control-groups.adoc b/Documentation/doc/development/control-groups.adoc
new file mode 100644
index 0000000000000000000000000000000000000000..fcf6f82f755b8cd542e5d45db4d1f3b3d43a375e
--- /dev/null
+++ b/Documentation/doc/development/control-groups.adoc
@@ -0,0 +1,95 @@
+= Control groups in Spectrum
+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0
+
+https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html[Linux control groups] (cgroups) can be used for several purposes:
+
+1. Waiting for a group of processes to exit.
+2. Terminating a group of processes.
+3. Limiting a group of processes' access to resources.
+
+Spectrum currently uses the first two. The third is not yet used,
+but will be in the future.
+
+== Control Group Hierarchy
+
+Spectrum uses the following cgroup hierarchy:
+
+1. There is a `+/vm-services.slice+` cgroup that contains all the per-VM
+ services on the system.
+2. The per-VM services for each VM are under `+/vm-services.slice/vm-${VM}.slice+`,
+ where `+${VM}+` is replaced by the VM's ID.
+3. Each per-VM service is under `+/vm-services.slice/vm-${VM}.slice/${SERVICE_NAME}+`,
+ where `+${VM}+` is replaced by the VM's ID and `+${SERVICE_NAME}+` is replaced by
+ the name of the service.
+4. The virtual machine monitor (VMM) runs under `+/vm-services.slice/vm-${VM}.slice/vmm+`.
+
+If a cgroup contains child cgroups, it likely contains a `$inner.service`
+cgroup. This is where programs that would otherwise run in the cgroup itself
+are placed. Generally, these programs are instances of
+https://skarnet.org/software/s6/s6-svscan.html[`s6-svscan`] or
+https://skarnet.org/software/s6/s6-supervise.html[`s6-supervise`].
+
+== Using Control Groups
+
+Most services should be placed in a control group, with only a few
+exceptions:
+
+1. Services, such as `getty`, that spawn background processes.
+2. Loggers.
+3. Trivial services where cgroups are unnecessary.
+
+=== Setting Up Control Groups
+
+Generally, it's best to set the control group up as the first thing
+the service does using Spectrum's https://spectrum-os.org/git/spectrum/tree/tools/cgroup-setup[`cgroup-setup`] tool.
+To do that, use `+cgroup-setup --leaf -- $1 COMMAND_LINE+`,
+where `+$1+` should be the service name and `+COMMAND_LINE+` is the program
+to run in a cgroup.
+
+If the control group starts with `/`, it is interpreted as a path relative
+to `/sys/fs/cgroup`. Otherwise, it is relative to the current control
+group the program is in.
+
+For example, using https://skarnet.org/software/execline/[execline] for your run script:
+
+[source,execline]
+....
+#!/bin/execlineb -WS1
+
+# Often, your cgroup is just the parent cgroup
+# with the name of the service ($1) appended.
+cgroup-setup --leaf -- $1
+# The rest of the script goes here.
+....
+
+=== Purging Control Groups
+
+If the service exits, it's usually best to terminate any programs left
+behind with `SIGKILL` and remove the control group. To do this,
+make the `finish` script invoke `cgroup-purge`. Its sole command-line
+argument is the cgroup to remove.
+
+[source,execline]
+....
+#!/bin/execlineb -WS3
+
+# Use the same path you used in the run script.
+cgroup-purge $3
+# The rest of the script goes here.
+....
+
+One can also use `cgroup-purge` to purge a cgroup explicitly. This is
+used to stop the VMM and all per-VM services when a VM is shut down.
+
+== Future plans
+
+Control groups are designed around a single writer process controlling each
+of them. Many Linux distros use systemd for this, but Spectrum doesn't use
+systemd. The only persistent per-service process is s6-supervise, but that
+doesn't have control group support.
+
+Instead, the plan is to have a database containing this information.
+Whether this will be in the `data/` subdirectory of the service directory
+or a separate system-wide database has not yet been determined.
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 04/19] Mount cgroup2 filesystem at /sys/fs/cgroup
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (2 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 03/19] Documentation: Mention control groups Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 05/19] host/rootfs: Enable controllers in non-root cgroups Demi Marie Obenour
` (15 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Needed for cgroups to work at all.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/image/etc/fstab | 1 +
1 file changed, 1 insertion(+)
diff --git a/host/rootfs/image/etc/fstab b/host/rootfs/image/etc/fstab
index 4d52a1da8c6d673cbbcfe2e9c0ec93085f06c40f..cab97cf9c49f864845492a1e524d43b7d546317f 100644
--- a/host/rootfs/image/etc/fstab
+++ b/host/rootfs/image/etc/fstab
@@ -6,3 +6,4 @@ tmpfs /dev/shm tmpfs nosuid,nodev 0 0
tmpfs /media tmpfs nosuid,nodev,noexec,nosymfollow,mode=755 0 0
sysfs /sys sysfs nosuid,nodev,noexec 0 0
tmpfs /tmp tmpfs nosuid,nodev 0 0
+cgroup2 /sys/fs/cgroup cgroup2 nosuid,nodev,noexec,nosymfollow 0 0
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 05/19] host/rootfs: Enable controllers in non-root cgroups
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (3 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 04/19] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 06/19] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
` (14 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
This is done with sed and execline to avoid special-cases in the cgroup
manager.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/init | 3 +++
1 file changed, 3 insertions(+)
diff --git a/host/rootfs/image/etc/init b/host/rootfs/image/etc/init
index 8ec7859ab00e24007a93d4000c8f34e353de50ae..2ab09d272702c0ea9daf6cf8bc880645a32eb2f0 100755
--- a/host/rootfs/image/etc/init
+++ b/host/rootfs/image/etc/init
@@ -12,4 +12,7 @@ if { /bin/ln -s /proc/self/fd/2 /dev/stderr }
if { /bin/mount --make-shared / }
if { /bin/mount -a --mkdir }
+# Enable subtree control of all cgroups.
+if { sed -nE "s/(^| )/&+/g; w /sys/fs/cgroup/cgroup.subtree_control" /sys/fs/cgroup/cgroup.controllers }
+
/bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 06/19] host/rootfs: Add comments where cgroups are intentionally not used
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (4 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 05/19] host/rootfs: Enable controllers in non-root cgroups Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 07/19] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
` (13 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Most services should use cgroups, but some do not.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run | 1 +
.../etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run | 1 +
host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run | 1 +
.../image/etc/s6-linux-init/run-image/service/serial-getty/template/run | 1 +
8 files changed, 8 insertions(+)
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
index 4d2cd7172e887e90d9ce87489cdacf41e51f77e8..5b2421a3dedf7c282ea818f4064535d0fc29d511 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty2/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty2 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
index 06abf5422365ff4edc9379e5cbcab716e6830399..3ca382f76368174ee1f99ea880827958c57b602c 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty3/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty3 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
index 3dd235fc93d403fc8971e923ec62cb025684b67a..983bd82927e468bab6ae20f61411680d173e4924 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/getty-tty4/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is>
+# no cgroups so that nohup works
getty -i -n -l /etc/login 0 tty4 linux
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
index 86b9a1ef70300de52f33a84cbfac22a77e00df36..fb55327dff5f189f543fb3629702caad9ff247e2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/root-terminal/run
@@ -2,6 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# cgroups make no sense here
s6-ipcserver-socketbinder -a 0700 /run/root-terminal
if { chown wayland /run/root-terminal }
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
index 327f4ccc5056e7ef87c089e666090307845cae3a..a6733604e91692f8440fd68efeb1a6b7378be167 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# cgroups make no sense for shutdownd
s6-linux-init-shutdownd -Bc /etc/s6-linux-init
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
index 8f7bd889e50d76a49b98d8ba9e2c6172f287277c..cf3f5c7b499d997fd426062651eca64b3accc146 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/s6-svscan-log/run
@@ -2,5 +2,6 @@
# SPDX-License-Identifier: ISC
# SPDX-FileCopyrightText: Copyright (c) 2015-2024 Laurent Bercot <ska-skaware@skarnet.org>
+# cgroups make no sense for a logger
redirfd -rnb 0 fifo
s6-log -bpd3 -- T /run/log
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
index 78f794202bf174f3c036f3e20755ac087a988277..8fb1b6779c0bb1e3cdf67d1fa14722caacc25994 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# No need for a cgroup here.
s6-svscan -d3 instance
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
index f3ed9a7f08e60206afd222f4fc02c5fb19527561..775b828c72b9d73bef8a3355fb3ae0536975f474 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty/template/run
@@ -2,4 +2,5 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2020-2021, 2024 Alyssa Ross <hi@alyssa.is>
+# Don't put this in a cgroup. We want nohup to work.
getty -i -n -l /etc/login 0,115200,57600,38400,9600 $1 dumb
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 07/19] host/rootfs: serial-getty-generator: Use cgroups
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (5 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 06/19] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 08/19] host/rootfs: systemd-udevd: Run in cgroup Demi Marie Obenour
` (12 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Avoids concurrent execution and leaked child processes.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/file-list.mk | 1 +
.../s6-linux-init/run-image/service/serial-getty-generator/finish | 5 +++++
.../etc/s6-linux-init/run-image/service/serial-getty-generator/run | 4 +++-
3 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 3899d620717fc97f42e669e5313c4100dcf5b1cd..deb4937fe380ff143fc2aafccee1b846fe15b974 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -21,6 +21,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run \
image/etc/s6-linux-init/run-image/service/s6-svscan-log/notification-fd \
image/etc/s6-linux-init/run-image/service/s6-svscan-log/run \
+ image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish \
image/etc/s6-linux-init/run-image/service/serial-getty-generator/run \
image/etc/s6-linux-init/run-image/service/serial-getty/notification-fd \
image/etc/s6-linux-init/run-image/service/serial-getty/run \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
new file mode 100755
index 0000000000000000000000000000000000000000..b1fb5799a949a5e69d5926a2145873da5f09e0a3
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-License-Identifier: EUPL-1.2+
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+
+cgroup-purge $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
index bf66ab0878fc6d8e77ca71a4a89e50c139a6bb11..a1846a29cf55892127feb7ecfc0adcd12f0497bd 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-getty-generator/run
@@ -1,7 +1,9 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+# avoid concurrent operation
+cgroup-setup --leaf -- $1
piperw 3 4
background {
fdclose 3
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 08/19] host/rootfs: systemd-udevd: Run in cgroup
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (6 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 07/19] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 09/19] host/rootfs: weston: " Demi Marie Obenour
` (11 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
host/rootfs/image/etc/s6-rc/systemd-udevd/finish | 5 +++++
host/rootfs/image/etc/s6-rc/systemd-udevd/run | 5 +++--
3 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index deb4937fe380ff143fc2aafccee1b846fe15b974..5403e464966e4946e2089841e9f2f529c0e856a5 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -97,6 +97,7 @@ S6_RC_FILES = \
image/etc/s6-rc/systemd-udevd-coldplug/dependencies.d/systemd-udevd \
image/etc/s6-rc/systemd-udevd-coldplug/type \
image/etc/s6-rc/systemd-udevd-coldplug/up \
+ image/etc/s6-rc/systemd-udevd/finish \
image/etc/s6-rc/systemd-udevd/notification-fd \
image/etc/s6-rc/systemd-udevd/run \
image/etc/s6-rc/systemd-udevd/type \
diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/finish b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
new file mode 100755
index 0000000000000000000000000000000000000000..8b63639fb196abfbb703f7ac6dc974f9838124d9
--- /dev/null
+++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# SPDX-License-Identifier: EUPL-1.2+
+
+cgroup-purge $3
diff --git a/host/rootfs/image/etc/s6-rc/systemd-udevd/run b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
old mode 100644
new mode 100755
index aec6444e951503eae988e666b77fda8f2ae33d72..7a4994d06060febd2256f1c4f11f48bb9b7b7efc
--- a/host/rootfs/image/etc/s6-rc/systemd-udevd/run
+++ b/host/rootfs/image/etc/s6-rc/systemd-udevd/run
@@ -1,7 +1,8 @@
-#!/bin/execlineb -WP
+#!/bin/execlineb -WS1
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
-s6-setlock /run/sd-notify-wrapper/systemd-udevd.lock
+
+cgroup-setup --systemd-delegate -- $1
s6-ipcserver-socketbinder -b0 -m -a 0600 /run/sd-notify-wrapper/systemd-udevd.sock
background -d {
fdmove 1 3
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 09/19] host/rootfs: weston: Run in cgroup
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (7 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 08/19] host/rootfs: systemd-udevd: Run in cgroup Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 10/19] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
` (10 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
host/rootfs/image/etc/s6-rc/weston/finish | 5 +++++
host/rootfs/image/etc/s6-rc/weston/run | 2 ++
3 files changed, 8 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 5403e464966e4946e2089841e9f2f529c0e856a5..2b76b31dad7f73dd83c6a24f94f3ed946988bbea 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -110,6 +110,7 @@ S6_RC_FILES = \
image/etc/s6-rc/vmm-env/contents.d/systemd-udevd-coldplug \
image/etc/s6-rc/vmm-env/type \
image/etc/s6-rc/weston/dependencies.d/systemd-udevd-coldplug \
+ image/etc/s6-rc/weston/finish \
image/etc/s6-rc/weston/notification-fd \
image/etc/s6-rc/weston/run \
image/etc/s6-rc/weston/type
diff --git a/host/rootfs/image/etc/s6-rc/weston/finish b/host/rootfs/image/etc/s6-rc/weston/finish
new file mode 100755
index 0000000000000000000000000000000000000000..8b63639fb196abfbb703f7ac6dc974f9838124d9
--- /dev/null
+++ b/host/rootfs/image/etc/s6-rc/weston/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# SPDX-License-Identifier: EUPL-1.2+
+
+cgroup-purge $3
diff --git a/host/rootfs/image/etc/s6-rc/weston/run b/host/rootfs/image/etc/s6-rc/weston/run
old mode 100644
new mode 100755
index 8b065ee9a535d0a08e9255900c73b88a0c376d4b..5ae6ca2f5527ca3ca4d58ce566dd58cbf1f79b5e
--- a/host/rootfs/image/etc/s6-rc/weston/run
+++ b/host/rootfs/image/etc/s6-rc/weston/run
@@ -4,6 +4,8 @@
importas -Siu WAYLAND_DISPLAY
+cgroup-setup --leaf --no-wait weston
+
piperw 4 3
background {
fdclose 3
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 10/19] host/rootfs: Set up parent cgroup for all per-VM services
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (8 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 09/19] host/rootfs: weston: " Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 11/19] host/rootfs: Create per-VM cgroup for all of the VM's services Demi Marie Obenour
` (9 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run | 3 +++
1 file changed, 3 insertions(+)
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
index 78f794202bf174f3c036f3e20755ac087a988277..d35fe5484ea48da874e4b19a8c910a1ad001f715 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/run
@@ -1,5 +1,8 @@
#!/bin/execlineb -WP
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# Set up the parent cgroup of all VMMs and VM services
+cgroup-setup --no-wait /vm-services.slice
s6-svscan -d3 instance
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 11/19] host/rootfs: Create per-VM cgroup for all of the VM's services
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (9 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 10/19] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 12/19] host/rootfs: run-vmm: Create per-VM cgroup Demi Marie Obenour
` (8 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../etc/s6-linux-init/run-image/service/vm-services/template/finish | 5 +++++
.../etc/s6-linux-init/run-image/service/vm-services/template/run | 4 ++++
3 files changed, 10 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 2b76b31dad7f73dd83c6a24f94f3ed946988bbea..36f508962c0515b2132aa9ab30cde929e2de100e 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -39,6 +39,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/run \
image/etc/s6-linux-init/run-image/service/vmm/notification-fd \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
new file mode 100755
index 0000000000000000000000000000000000000000..4448c811de5d382347b470afa07f7c90b0b08acc
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# SPDX-License-Identifier: EUPL-1.2+
+
+cgroup-purge /vm-services.slice/vm-${3}.slice
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
index bf90e99eef4106a9ad17dfba310b6edb3eaa64a3..7b1f20d6ffe826f92806c601572a11bac3fc5fca 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/run
@@ -2,6 +2,10 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+# Move into the cgroup with all per-VM services
+# Do not wait for the existing processes in the slice to finish.
+cgroup-setup --no-wait /vm-services.slice/vm-${1}.slice
+
export VM $1
s6-svscan -d3 data/service
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 12/19] host/rootfs: run-vmm: Create per-VM cgroup
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (10 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 11/19] host/rootfs: Create per-VM cgroup for all of the VM's services Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 13/19] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
` (7 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../image/etc/s6-linux-init/run-image/service/vmm/template/finish | 5 +++++
host/rootfs/image/usr/bin/run-vmm | 4 ++++
host/rootfs/image/usr/bin/vm-stop | 5 ++++-
4 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 36f508962c0515b2132aa9ab30cde929e2de100e..5f7b0c3fc1d7d7f8dc184ecc991126a793e316c6 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -44,6 +44,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/run \
image/etc/s6-linux-init/run-image/service/vmm/notification-fd \
image/etc/s6-linux-init/run-image/service/vmm/run \
+ image/etc/s6-linux-init/run-image/service/vmm/template/finish \
image/etc/s6-linux-init/run-image/service/vmm/template/notification-fd \
image/etc/s6-linux-init/scripts/rc.init \
image/etc/s6-linux-init/scripts/rc.shutdown \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/template/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/template/finish
new file mode 100755
index 0000000000000000000000000000000000000000..e316407f3a9cdee6961191480c5cab124254be02
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vmm/template/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# SPDX-License-Identifier: EUPL-1.2+
+
+cgroup-purge /vm-services.slice/vm-${3}.slice/vmm
diff --git a/host/rootfs/image/usr/bin/run-vmm b/host/rootfs/image/usr/bin/run-vmm
index 6967dc5586335d999f8030371fe0f38502409919..12896c276fbffa024fa0f89f42a2967506b1cb74 100755
--- a/host/rootfs/image/usr/bin/run-vmm
+++ b/host/rootfs/image/usr/bin/run-vmm
@@ -5,6 +5,7 @@
if {
backtick -D "" mnt {
importas -Siu 1
+ if { cgroup-setup --no-wait /vm-services.slice/vm-${1}.slice }
nsenter --mount=/run/vm/by-id/${1}/ns/mnt
findmnt -no FSTYPE,SOURCE /run/fs/${1}/disk
}
@@ -33,6 +34,8 @@ if {
elgetpositionals
+if { rm -f /run/vsock/${1}/vsock }
+
s6-ipcserver-socketbinder -B /run/vm/by-id/${1}/vmm
getpid -E vmm_pid
@@ -86,6 +89,7 @@ redirfd -r 0 /dev/null
s6-softlimit -H -l 18446744073709551615
if { udevadm wait /dev/kvm }
+cgroup-setup --leaf /vm-services.slice/vm-${1}.slice/vmm
s6-envuidgid vmm-${1}
s6-applyuidgid -Uz
bwrap
diff --git a/host/rootfs/image/usr/bin/vm-stop b/host/rootfs/image/usr/bin/vm-stop
index 6fb6aaed46de684eb035243e4f7c79111e93bf8c..23a815c0c260452ff9b1bcce7a091519b7420196 100755
--- a/host/rootfs/image/usr/bin/vm-stop
+++ b/host/rootfs/image/usr/bin/vm-stop
@@ -2,4 +2,7 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi@alyssa.is>
-ch-remote --api-socket /run/vm/by-id/${1}/vmm shutdown
+foreground { ch-remote --api-socket /run/vm/by-id/${1}/vmm shutdown }
+if { cgroup-purge /vm-services.slice/vm-${1}.slice/vmm }
+importas -iS ?
+exit $?
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 13/19] host/rootfs: run-appimage: Purge the per-VM cgroup
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (11 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 12/19] host/rootfs: run-vmm: Create per-VM cgroup Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 14/19] host/rootfs: run-flatpak: " Demi Marie Obenour
` (6 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/image/usr/bin/run-appimage | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/image/usr/bin/run-appimage b/host/rootfs/image/usr/bin/run-appimage
index 672ec22d4b815fb9c2940e13ae08ed4093baee4e..03020c89b02f295844f5ed814034ba66be741f4b 100755
--- a/host/rootfs/image/usr/bin/run-appimage
+++ b/host/rootfs/image/usr/bin/run-appimage
@@ -89,7 +89,12 @@ fdclose 4
foreground { run-vmm $id }
fdclose 3
-if { s6-instance-delete /run/service/vm-services $id }
+if {
+ # Even if s6-instance-delete fails, cgroup-purge is enough
+ # to ensure that the VMM and all per-VM services are killed.
+ foreground { s6-instance-delete /run/service/vm-services $id }
+ cgroup-purge /vm-services.slice/vm-${id}.slice
+}
if { umount -R /run/vm/by-id/${id}/ns }
rm -r -- $diskdir /run/vm/by-id/${id} /run/configs/${id}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 14/19] host/rootfs: run-flatpak: Purge the per-VM cgroup
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (12 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 13/19] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 15/19] host/rootfs: dbus: Run in cgroup Demi Marie Obenour
` (5 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Reviewed-by: Alyssa Ross <hi@alyssa.is>
---
host/rootfs/image/usr/bin/run-flatpak | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/host/rootfs/image/usr/bin/run-flatpak b/host/rootfs/image/usr/bin/run-flatpak
index fafbc82ff6966d8be9c8083a17618f1fb192bd07..cdd13fb52d97751f92864d34a5221367df6d6f0d 100755
--- a/host/rootfs/image/usr/bin/run-flatpak
+++ b/host/rootfs/image/usr/bin/run-flatpak
@@ -87,9 +87,12 @@ if {
fdclose 4
foreground { run-vmm $id }
-}
-if { s6-instance-delete -- /run/service/vm-services $id }
+ # Even if s6-instance-delete fails, cgroup-purge is enough
+ # to ensure that the VMM and all per-VM services are killed.
+ foreground { s6-instance-delete /run/service/vm-services $id }
+ cgroup-purge /vm-services.slice/vm-${id}.slice
+}
if { umount -R /run/vm/by-id/${id}/ns }
rm -r -- $diskdir /run/vm/by-id/${id} /run/configs/${id}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 15/19] host/rootfs: dbus: Run in cgroup
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (13 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 14/19] host/rootfs: run-flatpak: " Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 16/19] host/rootfs: vhost-user-fs: " Demi Marie Obenour
` (4 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../run-image/service/vm-services/template/data/service/dbus/finish | 5 +++++
.../run-image/service/vm-services/template/data/service/dbus/run | 2 ++
3 files changed, 8 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 5f7b0c3fc1d7d7f8dc184ecc991126a793e316c6..ea88f25aa4d5b96d29ed86975db12c2b047a0468 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -28,6 +28,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/serial-getty/template/run \
image/etc/s6-linux-init/run-image/service/vm-services/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish
new file mode 100755
index 0000000000000000000000000000000000000000..60ab4aad95aa17e28a97a91c885228ddeb53ccd3
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WP
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# SPDX-License-Identifier: EUPL-1.2+
+
+cgroup-purge dbus
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
index 331d9a57e8d99eb8d6f7b33031cd130b410a6157..7979bd76610e0efaa832996db70a75a01df7b16e 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run
@@ -4,6 +4,8 @@
importas -i VM VM
+cgroup-setup --leaf dbus
+
if {
redirfd -w 1 data/dbus.conf
sed "s/@XDP_SPECTRUM_USER@/xdp-spectrum-${VM}/g" /etc/dbus-portal.conf.in
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 16/19] host/rootfs: vhost-user-fs: Run in cgroup
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (14 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 15/19] host/rootfs: dbus: Run in cgroup Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 17/19] host/rootfs: vhost-user-gpu: " Demi Marie Obenour
` (3 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../service/vm-services/template/data/service/vhost-user-fs/finish | 5 +++++
.../service/vm-services/template/data/service/vhost-user-fs/run | 5 ++++-
3 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index ea88f25aa4d5b96d29ed86975db12c2b047a0468..e9b15f7ed466da0535809717de3c655b5f93d0f7 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -34,6 +34,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/notification-fd \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish
new file mode 100755
index 0000000000000000000000000000000000000000..8b63639fb196abfbb703f7ac6dc974f9838124d9
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# SPDX-License-Identifier: EUPL-1.2+
+
+cgroup-purge $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
index cb059aebc8dfe2c8a5b059d5bfea884de53bf922..fe5e2f76d50bab0a75b897649041d46d9e95ff6b 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run
@@ -2,9 +2,12 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2021-2025 Alyssa Ross <hi@alyssa.is>
-s6-ipcserver-socketbinder -a 0700 -B env/virtiofsd.sock
+cgroup-setup --leaf -- $1
importas -i VM VM
+
+s6-ipcserver-socketbinder -a 0700 -B env/virtiofsd.sock
+
if { chown vmm-${VM} env/virtiofsd.sock }
if { fdmove 1 3 echo }
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 17/19] host/rootfs: vhost-user-gpu: Run in cgroup
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (15 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 16/19] host/rootfs: vhost-user-fs: " Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 18/19] host/rootfs: xdg-desktop-portal-spectrum-host: " Demi Marie Obenour
` (2 subsequent siblings)
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../service/vm-services/template/data/service/vhost-user-gpu/finish | 5 +++++
.../service/vm-services/template/data/service/vhost-user-gpu/run | 2 ++
3 files changed, 8 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index e9b15f7ed466da0535809717de3c655b5f93d0f7..9560265b06c083e3e39fb6bbfa93778ac93f041a 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -37,6 +37,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/notification-fd \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish
new file mode 100755
index 0000000000000000000000000000000000000000..199ba1ea2f91b4a3cf4fb3344d9d8bbf98d19bde
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WP
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# SPDX-License-Identifier: EUPL-1.2+
+
+cgroup-purge vhost-user-gpu
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
index c3bfafe02ec6d2ee418fe56b033bbc4c7a73b186..a8ec06122e47605e7230c864ef22ef84c7de54b2 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run
@@ -3,6 +3,8 @@
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com>
+cgroup-setup --leaf vhost-user-gpu
+
s6-ipcserver-socketbinder -a 0700 -b 1 env/crosvm.sock
multisubstitute {
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 18/19] host/rootfs: xdg-desktop-portal-spectrum-host: Run in cgroup
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (16 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 17/19] host/rootfs: vhost-user-gpu: " Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 19/19] host/rootfs: spectrum-router: " Demi Marie Obenour
2026-08-12 21:06 ` [PATCH v6 00/19] Control group support Alyssa Ross
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../template/data/service/xdg-desktop-portal-spectrum-host/finish | 5 +++++
.../template/data/service/xdg-desktop-portal-spectrum-host/run | 2 ++
3 files changed, 8 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index 9560265b06c083e3e39fb6bbfa93778ac93f041a..a6435b0f9c73f45198deab39ea33f9e0326b2856 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -40,6 +40,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/run \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/finish \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish
new file mode 100755
index 0000000000000000000000000000000000000000..8b63639fb196abfbb703f7ac6dc974f9838124d9
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WS3
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# SPDX-License-Identifier: EUPL-1.2+
+
+cgroup-purge $3
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
index 909957b817136149dd01c3b98847238ecda8032d..3f5ebce4d2ac91f9240cb7ebeb9b307eeb4c73bf 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/xdg-desktop-portal-spectrum-host/run
@@ -2,6 +2,8 @@
# SPDX-License-Identifier: EUPL-1.2+
# SPDX-FileCopyrightText: 2024-2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup --leaf -- $1
+
importas -i VM VM
export DBUS_SESSION_BUS_ADDRESS unix:path=/run/portal-bus/${VM}
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* [PATCH v6 19/19] host/rootfs: spectrum-router: Run in cgroup
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (17 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 18/19] host/rootfs: xdg-desktop-portal-spectrum-host: " Demi Marie Obenour
@ 2026-08-06 1:16 ` Demi Marie Obenour
2026-08-12 21:06 ` [PATCH v6 00/19] Control group support Alyssa Ross
19 siblings, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 1:16 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
host/rootfs/file-list.mk | 1 +
.../service/vm-services/template/data/service/spectrum-router/finish | 5 +++++
.../service/vm-services/template/data/service/spectrum-router/run | 2 ++
3 files changed, 8 insertions(+)
diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk
index a6435b0f9c73f45198deab39ea33f9e0326b2856..c8a987a6ae20301134f6340f4e3a63b2ad050e17 100644
--- a/host/rootfs/file-list.mk
+++ b/host/rootfs/file-list.mk
@@ -32,6 +32,7 @@ FILES = \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down \
+ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/notification-fd \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run \
image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/finish \
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish
new file mode 100755
index 0000000000000000000000000000000000000000..e4bc1c1eb08113fb2a2b4c829f9671e3bdbd49aa
--- /dev/null
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/finish
@@ -0,0 +1,5 @@
+#!/usr/bin/execlineb -WP
+# SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+# SPDX-License-Identifier: EUPL-1.2+
+
+cgroup-purge spectrum-router
diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
index 3de58c27facc78c48176d8f9a6dd1827ac926f1a..a2ae8499e65590830b90c9bdda7d0f8a3336ef09 100755
--- a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
+++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run
@@ -3,6 +3,8 @@
# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
# SPDX-FileCopyrightText: 2025 Alyssa Ross <hi@alyssa.is>
+cgroup-setup --leaf spectrum-router
+
importas -i VM VM
s6-ipcserver-socketbinder -a 0770 /run/vm/by-id/${VM}/router-driver.sock
--
2.55.0
^ permalink raw reply related [flat|nested] 132+ messages in thread
* Re: [PATCH v6 02/19] tools: Add control group manager
2026-08-06 1:16 ` [PATCH v6 02/19] tools: Add control group manager Demi Marie Obenour
@ 2026-08-06 6:58 ` Demi Marie Obenour
2026-08-12 21:10 ` Alyssa Ross
1 sibling, 0 replies; 132+ messages in thread
From: Demi Marie Obenour @ 2026-08-06 6:58 UTC (permalink / raw)
To: Spectrum OS Development; +Cc: Alyssa Ross
[-- Attachment #1.1: Type: text/plain, Size: 852 bytes --]
On 8/5/26 21:16, Demi Marie Obenour wrote:
> The cgroup-setup Rust program can create and purge cgroups. It can also
> wait for one to become empty, spawn a program in a cgroup, and more. In
> the future, it will also support cgroup-based resource control. Locking
> is used to ensure that concurrent invocations are safe.
>
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
> ---
(snip)
> +// This uses a recursive algorith, but so does std::fs::remove_dir_all().
Typo: "algorith" should be "algorithm". Caught by codespell. Feel free
to fix up on commit.
With this fixed, the following command succeeds:
git rebase main -x 'nix-build --log-format bar-with-logs release.nix'
indicating that the final commit and all intermediate commits build
and pass tests.
--
Sincerely,
Demi Marie Obenour (she/her/hers)
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v6 00/19] Control group support
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
` (18 preceding siblings ...)
2026-08-06 1:16 ` [PATCH v6 19/19] host/rootfs: spectrum-router: " Demi Marie Obenour
@ 2026-08-12 21:06 ` Alyssa Ross
19 siblings, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-08-12 21:06 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development, Valentin Gagarin
[-- Attachment #1: Type: text/plain, Size: 985 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> - Place SPDX-FileCopyrightText before SPDX-License-Identifier, except in
> patches that already have Reviewed-by tags.
For the future, Reviewed-by does not mean you can't change the patch at
all:
> Both Tested-by and Reviewed-by tags, once received on mailing list
> from tester or reviewer, should be added by author to the applicable
> patches when sending next versions. However if the patch has changed
> substantially in following version, these tags might not be applicable
> anymore and thus should be removed. Usually removal of someone's
> Acked-by, Tested-by or Reviewed-by tags should be mentioned in the
> patch changelog with an explanation (after the '---' separator).
https://www.kernel.org/doc/html/latest/process/submitting-patches.html#using-reported-by-tested-by-reviewed-by-suggested-by-and-fixes
I suppose this is somewhere we probably ought to link to the kernel
docs as Valentin recently suggested.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
* Re: [PATCH v6 02/19] tools: Add control group manager
2026-08-06 1:16 ` [PATCH v6 02/19] tools: Add control group manager Demi Marie Obenour
2026-08-06 6:58 ` Demi Marie Obenour
@ 2026-08-12 21:10 ` Alyssa Ross
1 sibling, 0 replies; 132+ messages in thread
From: Alyssa Ross @ 2026-08-12 21:10 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: Spectrum OS Development
[-- Attachment #1: Type: text/plain, Size: 20586 bytes --]
On Wed, Aug 05, 2026 at 09:16:09PM -0400, Demi Marie Obenour wrote:
> The cgroup-setup Rust program can create and purge cgroups. It can also
> wait for one to become empty, spawn a program in a cgroup, and more. In
> the future, it will also support cgroup-based resource control. Locking
> is used to ensure that concurrent invocations are safe.
>
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Mostly just unclear comments/messages, so we're getting very close,
but some of my questions from last time still stand, too. We should
figure those out before another round is submitted, because there's no
point in me seeing the same things and asking the same questions again
and again.
> diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs
> new file mode 100644
> index 0000000000000000000000000000000000000000..c63d7e5a4aa73429578704401c58bbafc79e7c3f
> --- /dev/null
> +++ b/tools/cgroup-setup/src/cgroup.rs
> @@ -0,0 +1,269 @@
> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +// SPDX-License-Identifier: EUPL-1.2+
> +
> +use std::ffi::OsStr;
> +use std::fs::File;
> +use std::io::{Read as _, Seek as _, Write as _};
> +use std::os::unix::prelude::*;
> +
> +use std::path::{Component, Path, PathBuf};
> +
> +use rustix::fs::{AtFlags, CWD, Dir, FlockOperation};
> +use rustix::path;
> +use rustix::{
> + fs::{Mode, OFlags, ResolveFlags},
> + io::Errno,
> +};
> +
> +pub enum OpenFlags {
> + Read,
> + Write,
> + Directory,
> +}
> +
> +#[derive(Debug)]
> +pub(crate) struct Cgroup {
> + fd: Vec<OwnedFd>,
> +}
> +
> +impl AsFd for Cgroup {
> + fn as_fd(&self) -> BorrowedFd<'_> {
> + self.fd.last().unwrap().as_fd()
> + }
> +}
> +
> +fn assert_single_component(component: &Path) {
> + match component.as_os_str().as_bytes() {
> + b"" | b"." | b".." => panic!("bad component"),
> + c if c.contains(&b'\0') => panic!("NUL in component"),
> + c if c.contains(&b'/') => panic!("/ in component"),
> + _ => {}
> + }
> +}
> +
> +// Wrapper around openat2() with better defaults.
> +pub fn openat2_simple(
> + fd: impl AsFd,
> + path: impl path::Arg,
> + flags: OpenFlags,
> +) -> Result<OwnedFd, Errno> {
> + rustix::fs::openat2(
> + fd.as_fd(),
> + path,
> + OFlags::CLOEXEC
> + | match flags {
> + OpenFlags::Read => OFlags::RDONLY | OFlags::NOCTTY,
> + OpenFlags::Write => OFlags::WRONLY | OFlags::NOCTTY,
> + OpenFlags::Directory => OFlags::RDONLY | OFlags::DIRECTORY,
> + },
> + Mode::empty(),
> + ResolveFlags::NO_SYMLINKS | ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_XDEV,
> + )
> +}
> +
> +pub const DEFAULT_LEAF: &str = "$inner.service";
> +
> +pub fn check_path(path: &Path) -> Result<(), String> {
> + let bytes = path.as_os_str().as_bytes();
> + // Path::components() skips ., so use string manipulation instead.
> + for component in bytes[path.is_absolute() as usize..].split(|&b| b == b'/') {
> + if matches!(component, b"" | b"." | b"..") {
> + return Err(format!("cgroup path {path:?} isn't canonical"));
> + }
> + }
> + Ok(())
> +}
> +
> +// Remove all subdirectories of the given directory recursively,
> +// but not the directory itself. The directory file descriptor
> +// is closed.
> +//
> +// This isn't the most efficient possible algorithm, but
> +// simplicity is more important than performance in this
> +// case. Also, it keeps open more file descriptors than
> +// strictly necessary, but Spectrum runs with a very high
> +// limit for the number of open file descriptors, and it
> +// uses shallow control group hierarchies.
> +//
> +// This uses a recursive algorith, but so does std::fs::remove_dir_all().
> +// Trying to be more robust than the standard library is not worthwhile.
> +// In particular, the standard library function must be safe on systems
> +// where untrusted users (or even network endpoints!) can create deeply
> +// nested directory trees, whereas in Spectrum cgroups are only writeable
> +// by root.
This comment seems to be wrapped at two different widths. Per Rust
style I think it would be idiomatic for it to be wrapped at 83 (80 from
the start of text after the comment marker and space).
> +fn remove_recursively(mut dirfd: Dir, remaining_depth: usize) -> Result<(), Errno> {
> + if remaining_depth < 1 {
> + panic!("control groups too deeply nested");
> + }
> + while let Some(element) = dirfd.next() {
> + let parent_fd = dirfd.fd().unwrap();
> + let element = element.expect("Iterating through a cgroup directory failed?");
> + let path = element.file_name();
> + if element.file_type() != rustix::fs::FileType::Directory || path == c"." || path == c".." {
> + continue;
> + }
> + let fd = openat2_simple(parent_fd, path, OpenFlags::Directory)?;
> + remove_recursively(Dir::new(fd).unwrap(), remaining_depth - 1)?;
> + match rustix::fs::unlinkat(parent_fd, path, AtFlags::REMOVEDIR) {
> + Err(Errno::NOTEMPTY | Errno::BUSY | Errno::NOENT) | Ok(()) => {}
I assume this BUSY exception is for the child cgroup reason explained
in a comment later? It probably ought to be explained here too,
because it's not all obvious that a function named remove_recursively
would silently ignore EBUSY.
> + bad => return bad,
> + }
> + }
> + Ok(())
> +}
> +
> +// If the path is absolute, make it relative.
> +// Otherwise, read the current cgroup from /proc/thread-self/cgroup
> +// and prepend it to the path.
This makes it sound like it's doing two completely different things,
rather than putting something into some standardized format.
> +fn prepend_current_cgroup_if_needed(path: &Path) -> PathBuf {
> + if let Ok(suffix) = path.strip_prefix("/") {
> + suffix.to_owned()
> + } else {
> + // /proc/thread-self is the same as /proc/self, except for the current
> + // thread instead of the initial thread. In this case, the two are
> + // identical, but using /proc/thread-self is better practice as it is
> + // correct in more cases. Reading /proc/thread-self/cgroup should
> + // never fail unless the system is seriously broken.
> + let current_cgroup = std::fs::read("/proc/thread-self/cgroup")
> + .expect("cannot read /proc/thread-self/cgroup");
> + // Using this on a system without cgroups v2 mounted is user error
> + // and not supported.
> + let current_cgroup = current_cgroup
> + .strip_prefix(b"0::/")
> + .and_then(|e| e.strip_suffix(b"\n"))
> + .expect("you don't have cgroups v2 mounted");
Not strictly the cause of the error. You don't need cgroupfs mounted
to read /proc/thread-self/cgroup I assume. The actual problem would
be that you're in a v1 cgroup, right?
> + let mut current_cgroup = PathBuf::from(OsStr::from_bytes(current_cgroup));
> + // Strip the implied $inner.service suffix.
> + // This is used to satisfy the "no internal processes" rule.
> + if current_cgroup.ends_with(Path::new(DEFAULT_LEAF)) {
> + assert!(current_cgroup.pop());
> + }
> + current_cgroup.push(path);
> + current_cgroup
> + }
> +}
> +
> +pub(crate) fn write_value(fd: &dyn AsFd, name: &Path, value: &[u8]) -> Result<(), String> {
> + let fd = openat2_simple(fd, name, OpenFlags::Write)
> + .map_err(|e| format!("Cannot open {name:?}: {e}"))?;
> + File::from(fd).write_all(value).map_err(|e| {
> + format!(
> + "Cannot write {:?} to {name:?}: {e}",
> + OsStr::from_bytes(value)
> + )
> + })
> +}
> +
> +impl Cgroup {
> + pub fn new(path: &Path) -> Result<Self, String> {
> + let cgroup_root = rustix::fs::openat2(
> + CWD,
> + Path::new("/sys/fs/cgroup"),
> + OFlags::CLOEXEC | OFlags::DIRECTORY | OFlags::RDONLY,
> + Mode::empty(),
> + ResolveFlags::NO_SYMLINKS | ResolveFlags::NO_MAGICLINKS,
> + )
> + .map_err(|e| format!("Cannot open /sys/fs/cgroup: {e}"))?;
> + let mut cgroup = Self {
> + fd: vec![(cgroup_root)],
> + };
We don't take a lock on this, so what's it kept around for?
> +
> + let path = prepend_current_cgroup_if_needed(path);
> + for component in path.components() {
> + let Component::Normal(component) = component else {
> + unreachable!()
> + };
> + let sub_fd = openat2_simple(&cgroup, component, OpenFlags::Directory)
> + .map_err(|e| format!("Cannot open sub-cgroup {component:?}: {e}"))?;
> + // Take a shared lock on the cgroup.
> + rustix::fs::flock(&sub_fd, FlockOperation::LockShared)
> + .map_err(|e| format!("Cannot lock sub-cgroup {component:?}: {e}"))?;
> + cgroup.fd.push(sub_fd);
> + }
> + Ok(cgroup)
> + }
> +
> + pub fn wait_for_empty(fd: &dyn AsFd) -> std::io::Result<()> {
> + let wait_file = openat2_simple(fd, c"cgroup.events", OpenFlags::Read)?;
> + let mut wait_fd = File::from(wait_file);
> + let mut v = vec![];
> + loop {
> + v.clear();
> + wait_fd
> + .seek(std::io::SeekFrom::Start(0))
> + .expect("Seek on control group file should succeed");
> + wait_fd
> + .read_to_end(&mut v)
> + .expect("reading from control group should work");
> + // Check that the cgroup isn't already empty. If it was,
> + // the kernel would not send an event and poll() would wait
> + // forever.
> + if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") {
> + break;
> + }
> + let mut fds = libc::pollfd {
> + fd: wait_fd.as_raw_fd(),
> + events: libc::POLLPRI | libc::POLLERR,
> + revents: 0,
> + };
> + // SAFETY: FFI call, valid arguments, fds contains 1 element
> + if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 {
> + panic!("poll failed");
> + }
> + }
> + drop(wait_fd);
This happens automatically.
> + Ok(())
> + }
> +
> + pub fn purge_child(&mut self, path: &Path) -> Result<(), String> {
> + assert_single_component(path);
> + // See if we can just delete the child directly.
> + match rustix::fs::unlinkat(&self, Path::new(path), AtFlags::REMOVEDIR) {
path is already a Path.
> + // If the cgroup was successfully deleted, or if it
> + // has already been deleted, we are done.
> + Ok(()) | Err(Errno::NOENT) => return Ok(()),
> + // If this cgroup is in use, keep going.
> + Err(Errno::BUSY) => {}
> + Err(e) => return Err(format!("Cannot purge {path:?}: {e}")),
> + }
> +
> + let sub_fd = match openat2_simple(&self, path, OpenFlags::Directory) {
> + Ok(sub_fd) => sub_fd,
Maybe it would be nicer to do the Dir::new here?
> + Err(Errno::NOENT) => return Ok(()),
> + Err(e) => {
> + return Err(format!("Cannot open sub-cgroup {path:?}: {e}",));
> + }
> + };
> +
> + // Take an exclusive lock on the cgroup that is about to be
> + // removed. This avoids concurrent executions of this program
> + // operating on deleted sub-cgroups.
> + rustix::fs::flock(&sub_fd, FlockOperation::LockExclusive)
> + .map_err(|e| format!("Cannot lock sub-cgroup: {e}"))?;
Wow, if this is all the locking we need, that's a great simplification!
> +
> + // Kill all processes in the child cgroup.
> + write_value(&sub_fd, Path::new("cgroup.kill"), b"1")?;
> +
> + // Wait for the child cgroup to become empty.
> + Self::wait_for_empty(&sub_fd)
> + .map_err(|e| format!("Cannot wait for cgroup to become empty: {e}"))
> + .inspect_err(|_| {
> + self.fd.pop().unwrap();
> + })?;
> +
> + // Remove the child cgroup and its contents recursively.
> + remove_recursively(Dir::new(sub_fd).unwrap(), 1000)
> + .map_err(|e| format!("Cannot remove: {e}"))?;
> +
> + // Delete the cgroup. If it's been re-created in the meantime
> + // and is currently in use, this is not an error. Another
> + // process deleting the cgroup is also not an error. Both of
> + // these can happen because of the time period between
> + // remove_child_directories() closing the file descriptor
> + // (releasing its lock) and the above call to flock().
> + match rustix::fs::unlinkat(&self, path, AtFlags::REMOVEDIR) {
> + Ok(()) | Err(Errno::BUSY) | Err(Errno::NOENT) => Ok(()),
> + Err(e) => Err(format!("Cannot delete: {e}")),
> + }
> + }
> +}
> diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs
> new file mode 100644
> index 0000000000000000000000000000000000000000..58ca7295bb348a2c92640a68c60b285b2d7a1494
> --- /dev/null
> +++ b/tools/cgroup-setup/src/main.rs
> @@ -0,0 +1,167 @@
> +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
> +// SPDX-License-Identifier: EUPL-1.2+
> +
> +mod cgroup;
> +
> +use cgroup::{Cgroup, OpenFlags, openat2_simple, write_value};
> +use rustix::{
> + fs::{FlockOperation, Mode, XattrFlags},
> + io::Errno,
> +};
> +use std::{
> + env::ArgsOs,
> + fs::File,
> + io::Read as _,
> + os::unix::prelude::*,
> + path::{Path, PathBuf},
> +};
> +
> +// Check that the path is canonical,
> +// then split it into basename and filename.
> +fn split_path(path: &Path) -> Result<(&Path, &Path), String> {
> + cgroup::check_path(path)?;
> + Ok((path.parent().unwrap(), Path::new(path.file_name().unwrap())))
> +}
> +
> +fn enable_subtree_control(fd: &dyn AsFd) -> Result<(), String> {
> + let mut buf = Vec::new();
> + File::from(
> + openat2_simple(fd, c"cgroup.controllers", OpenFlags::Read)
> + .map_err(|e| format!("Cannot open cgroup.controllers: {e}"))?,
> + )
> + .read_to_end(&mut buf)
> + .map_err(|e| format!("Cannot read cgroup.controllers: {e}"))?;
> + let mut subtree = vec![];
> + for controller in buf.split(|&b| b == b' ') {
> + if !subtree.is_empty() {
> + subtree.push(b' ');
> + }
> + subtree.push(b'+');
> + subtree.extend_from_slice(controller);
> + }
> + if !subtree.is_empty() {
> + write_value(&fd, Path::new("cgroup.subtree_control"), &subtree)?;
> + }
> + Ok(())
> +}
> +
> +fn cgroup_setup(args: ArgsOs) -> Result<(), String> {
> + let mut leaf = false;
> + let mut systemd_delegate = false;
> + let mut wait = true;
> + let mut args = args.peekable();
> + while let Some(arg) = args.peek() {
> + if !arg.as_bytes().starts_with(b"-") {
> + break;
> + }
> + let arg = args.next().unwrap();
> + let Some(arg_) = arg.as_bytes().strip_prefix(b"--") else {
Why not just shadow arg?
> + return Err("takes no short options".to_owned());
> + };
> + match arg_ {
> + b"" => break,
> + b"leaf" => leaf = true,
> + b"no-wait" => wait = false,
> + b"systemd-delegate" => systemd_delegate = true,
> + _ => return Err(format!("unknown long option {arg:?}")),
> + }
> + }
Lovely and straightforward now.
> + let Some(cgroup_path) = args.next().map(PathBuf::from) else {
> + return Err("have no positional arguments, expected at least 1".to_owned());
> + };
> +
> + let (parent_cgroup_path, child_cgroup_path) = split_path(&cgroup_path)?;
> + let cgroup = Cgroup::new(parent_cgroup_path)?;
> + match rustix::fs::mkdirat(&cgroup, child_cgroup_path, Mode::from_raw_mode(0o755)) {
> + Ok(()) | Err(Errno::EXIST) => {}
> + Err(e) => {
> + return Err(format!(
> + "Cannot make child cgroup {child_cgroup_path:?}: {e}"
> + ));
> + }
> + }
> + let child = openat2_simple(&cgroup, child_cgroup_path, OpenFlags::Directory)
> + .map_err(|e| format!("Cannot make child cgroup: {e}"))?;
It's not correct to say "make" here.
> + // While waiting, hold an exclusive lock on the child.
> + // This avoids two processes both waiting for the same cgroup to become
> + // empty, then spawning processes in the same cgroup.
> + rustix::fs::flock(&child, FlockOperation::LockExclusive)
> + .map_err(|e| format!("Cannot take an exclusive lock on child cgroup: {e}"))?;
> + if wait {
> + Cgroup::wait_for_empty(&child)
> + .map_err(|e| format!("Cannot wait for {parent_cgroup_path:?} to be empty: {e}"))?;
> + }
> + let pid = std::process::id().to_string();
> + if leaf {
> + if args.len() != 0 {
> + // If we aren't delegating any cgroups, don't create a sub-cgroup.
> + write_value(&child, Path::new("cgroup.procs"), pid.as_bytes())
> + .map_err(|e| format!("Cannot move process to child cgroup: {e}"))?;
> + }
> + } else {
> + // If the child process will need to manage cgroups itself, it will need
> + // to set up a sub-cgroup due to the "no internal processes" rule. It's
> + // simplest to just do it automatically. If the cgroup already exists,
> + // that isn't an error.
> + match rustix::fs::mkdirat(&child, cgroup::DEFAULT_LEAF, Mode::from_raw_mode(0o755)) {
> + Ok(()) | Err(Errno::EXIST) => {}
> + Err(e) => return Err(format!("Cannot make child cgroup: {e}")),
> + }
> + if args.len() != 0 {
> + let child_proc_path = Path::new(cgroup::DEFAULT_LEAF).join(Path::new("cgroup.procs"));
> + write_value(&child, &child_proc_path, pid.as_bytes())
> + .map_err(|e| format!("Cannot move process to child cgroup: {e}"))?;
> + }
We're still complicating this by insisting on leaf mode being
different, for the extremely nebulous cause of not wasting cgroups.
> + if systemd_delegate {
> + // systemd-aware programs expect to have user.delegate=1
> + // and to set cgroup.subtree_control themselves
> + rustix::fs::fsetxattr(&child, c"user.delegate", b"1", XattrFlags::empty()).map_err(
> + |e| format!("Cannot enable cgroup delegation in {parent_cgroup_path:?}: {e}"),
> + )?
> + } else {
> + // Spectrum's programs do not check for user.delegate=1
> + // and expect the caller to set cgroup.subtree_control.
> + enable_subtree_control(&child)?;
My question from last time about doing this unconditionally has not
been answered as far as I can see.
> + }
> + }
> + let Some(program_name) = args.next() else {
> + return Ok(());
> + };
> + let e = std::process::Command::new(&program_name).args(args).exec();
> + Err(format!("Cannot spawn child {program_name:?}: {e}",))
> +}
> +
> +fn cgroup_purge(mut args: ArgsOs) -> Result<(), String> {
> + if args.len() != 1 {
> + return Err("usage: cgroup-purge CGROUP_TO_PURGE".to_owned());
> + }
> + let arg = args.next().unwrap();
> + let (parent, child) = split_path(Path::new(&arg))?;
> + Cgroup::new(parent)?.purge_child(child)
> +}
> +
> +fn run(prog_name: &Path, args: ArgsOs) -> Result<(), String> {
> + match prog_name.file_name().map(|f| f.as_bytes()) {
> + Some(b"cgroup-setup") => cgroup_setup(args),
> + Some(b"cgroup-purge") => cgroup_purge(args),
> + _ => Err(format!(
> + "must be invoked as \"cgroup-setup\" or \
> + \"cgroup-purge\", got {prog_name:?}",
> + )),
> + }
> +}
> +
> +fn main() {
> + let mut args = std::env::args_os();
> + let Some(prog_name) = args.next() else {
> + eprintln!("No command line arguments (argv[0] is NULL)");
> + std::process::exit(1);
> + };
> + match run(Path::new(&prog_name), args) {
> + Ok(()) => {}
> + Err(e) => {
> + eprintln!("{prog_name:?}: {}", e);
> + std::process::exit(1);
> + }
> + }
> +}
>
> --
> 2.55.0
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 132+ messages in thread
end of thread, other threads:[~2026-08-12 21:10 UTC | newest]
Thread overview: 132+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-06-20 14:23 [PATCH] Set up control groups for most services Demi Marie Obenour
2026-06-20 17:27 ` [PATCH v2] " Demi Marie Obenour
2026-06-24 12:13 ` Alyssa Ross
2026-06-24 12:36 ` Alyssa Ross
2026-06-25 2:03 ` Demi Marie Obenour
2026-06-25 3:03 ` Demi Marie Obenour
2026-06-25 9:55 ` Alyssa Ross
2026-06-25 9:49 ` Alyssa Ross
2026-07-11 20:12 ` [PATCH v3 00/22] Control group support Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 01/22] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
2026-07-13 9:39 ` Alyssa Ross
2026-07-13 17:27 ` Demi Marie Obenour
2026-07-15 18:28 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 00/20] Control group support Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 01/20] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 02/20] tools: Add control group manager Demi Marie Obenour
2026-07-22 16:01 ` Alyssa Ross
2026-07-23 23:07 ` Demi Marie Obenour
2026-07-27 12:10 ` Alyssa Ross
2026-07-30 0:40 ` Demi Marie Obenour
2026-07-30 14:53 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 03/20] Documentation: Mention control groups Demi Marie Obenour
2026-07-27 11:22 ` Alyssa Ross
2026-07-28 10:41 ` Valentin Gagarin
2026-07-22 1:59 ` [PATCH v4 04/20] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
2026-07-27 11:23 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 05/20] host/rootfs: Add helper program for per-VM services Demi Marie Obenour
2026-07-27 11:27 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 06/20] host/rootfs: Enable controllers in sub-cgroups Demi Marie Obenour
2026-07-27 12:11 ` Alyssa Ross
2026-07-28 2:19 ` Demi Marie Obenour
2026-07-29 14:13 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 07/20] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
2026-07-27 12:12 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 08/20] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
2026-07-27 12:14 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 09/20] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
2026-07-27 12:16 ` Alyssa Ross
2026-07-28 3:01 ` Demi Marie Obenour
2026-07-29 14:29 ` Alyssa Ross
2026-07-29 20:20 ` Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 10/20] host/rootfs: Create per-VM cgroup for all of the VM's services Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 11/20] host/rootfs: run-vmm: Create per-VM cgroup Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 12/20] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
2026-07-27 12:18 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 13/20] host/rootfs: run-flatpak: " Demi Marie Obenour
2026-07-27 12:19 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 14/20] host/rootfs: dbus: Run in cgroup Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 15/20] host/rootfs: vhost-user-fs: " Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 16/20] host/rootfs: vhost-user-gpu: " Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 17/20] host/rootfs: xdg-desktop-portal-spectrum-host: " Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 18/20] host/rootfs: systemd-udevd: " Demi Marie Obenour
2026-07-27 12:20 ` Alyssa Ross
2026-07-28 3:11 ` Demi Marie Obenour
2026-07-29 14:15 ` Alyssa Ross
2026-07-29 20:39 ` Demi Marie Obenour
2026-07-30 14:55 ` Alyssa Ross
2026-07-22 1:59 ` [PATCH v4 19/20] host/rootfs: weston: " Demi Marie Obenour
2026-07-27 12:23 ` Alyssa Ross
2026-07-28 3:14 ` Demi Marie Obenour
2026-07-22 1:59 ` [PATCH v4 20/20] host/rootfs: spectrum-router: " Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 00/19] Control group support Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 01/19] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 02/19] tools: Add control group manager Demi Marie Obenour
2026-08-03 12:47 ` Alyssa Ross
2026-08-05 1:36 ` Demi Marie Obenour
2026-08-05 16:39 ` Alyssa Ross
2026-07-31 21:54 ` [PATCH v5 03/19] Documentation: Mention control groups Demi Marie Obenour
2026-08-03 13:22 ` Alyssa Ross
2026-07-31 21:54 ` [PATCH v5 04/19] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 05/19] host/rootfs: Enable controllers in non-root cgroups Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 06/19] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 07/19] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 08/19] host/rootfs: systemd-udevd: Run in cgroup Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 09/19] host/rootfs: weston: " Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 10/19] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 11/19] host/rootfs: Create per-VM cgroup for all of the VM's services Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 12/19] host/rootfs: run-vmm: Create per-VM cgroup Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 13/19] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 14/19] host/rootfs: run-flatpak: " Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 15/19] host/rootfs: dbus: Run in cgroup Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 16/19] host/rootfs: vhost-user-fs: " Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 17/19] host/rootfs: vhost-user-gpu: " Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 18/19] host/rootfs: xdg-desktop-portal-spectrum-host: " Demi Marie Obenour
2026-07-31 21:54 ` [PATCH v5 19/19] host/rootfs: spectrum-router: " Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 00/19] Control group support Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 01/19] host/rootfs: Mount filesystems before s6-rc-init Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 02/19] tools: Add control group manager Demi Marie Obenour
2026-08-06 6:58 ` Demi Marie Obenour
2026-08-12 21:10 ` Alyssa Ross
2026-08-06 1:16 ` [PATCH v6 03/19] Documentation: Mention control groups Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 04/19] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 05/19] host/rootfs: Enable controllers in non-root cgroups Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 06/19] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 07/19] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 08/19] host/rootfs: systemd-udevd: Run in cgroup Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 09/19] host/rootfs: weston: " Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 10/19] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 11/19] host/rootfs: Create per-VM cgroup for all of the VM's services Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 12/19] host/rootfs: run-vmm: Create per-VM cgroup Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 13/19] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 14/19] host/rootfs: run-flatpak: " Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 15/19] host/rootfs: dbus: Run in cgroup Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 16/19] host/rootfs: vhost-user-fs: " Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 17/19] host/rootfs: vhost-user-gpu: " Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 18/19] host/rootfs: xdg-desktop-portal-spectrum-host: " Demi Marie Obenour
2026-08-06 1:16 ` [PATCH v6 19/19] host/rootfs: spectrum-router: " Demi Marie Obenour
2026-08-12 21:06 ` [PATCH v6 00/19] Control group support Alyssa Ross
2026-07-11 20:12 ` [PATCH v3 02/22] scripts: Support symlinks in s6-rc-compile inputs Demi Marie Obenour
2026-07-13 9:42 ` Alyssa Ross
2026-07-13 14:19 ` Demi Marie Obenour
2026-07-15 18:30 ` Alyssa Ross
2026-07-11 20:12 ` [PATCH v3 03/22] tools: Add control group manager Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 04/22] Documentation: Mention control groups Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 05/22] Mount cgroup2 filesystem at /sys/fs/cgroup Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 06/22] host/rootfs: Add helper program for per-VM services Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 07/22] host/rootfs: Enable controllers in sub-cgroups Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 08/22] host/rootfs: Add comments where cgroups are intentionally not used Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 09/22] host/rootfs: serial-getty-generator: Use cgroups Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 10/22] host/rootfs: Set up parent cgroup for all per-VM services Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 11/22] host/rootfs: Create per-VM cgroup for all of the VM's services Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 12/22] host/rootfs: run-vmm: Create per-VM cgroup Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 13/22] host/rootfs: run-appimage: Purge the " Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 14/22] host/rootfs: run-flatpak: " Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 15/22] host/rootfs: dbus: Run in cgroup Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 16/22] host/rootfs: vhost-user-fs: " Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 17/22] host/rootfs: vhost-user-gpu: " Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 18/22] host/rootfs: xdg-desktop-portal-spectrum-host: " Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 19/22] host/rootfs: systemd-udevd: " Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 20/22] host/rootfs: weston: " Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 21/22] host/rootfs: spectrum-router: " Demi Marie Obenour
2026-07-11 20:12 ` [PATCH v3 22/22] host/rootfs: vm-import: Use elglob -w Demi Marie Obenour
Code repositories for project(s) associated with this public inbox
https://spectrum-os.org/git/doc
https://spectrum-os.org/git/mktuntap
https://spectrum-os.org/git/spectrum
https://spectrum-os.org/git/ucspi-vsock
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).