From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id 41B7C65C6; Wed, 22 Jul 2026 02:16:54 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 476D36547; Wed, 22 Jul 2026 02:16:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_PASS,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=4.0.1 Received: from mail-yw1-x1131.google.com (mail-yw1-x1131.google.com [IPv6:2607:f8b0:4864:20::1131]) by atuin.qyliss.net (Postfix) with ESMTPS id 71D8B6529 for ; Wed, 22 Jul 2026 02:16:49 +0000 (UTC) Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-80dc4a68e4aso94142317b3.0 for ; Tue, 21 Jul 2026 19:16:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784686608; x=1785291408; darn=spectrum-os.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=o+eEBMpC6tE1uWEH3QJKoYHj9Gqv/C2lkToe2qNQDnE=; b=UOpJ+IBprwy/Dc5MbL3ZmuwZ1sboO33RONR+oTQQVb2okNOsWKkSUCr6AL9ug2ur7R PISYPuxLGzf8JPzCCy0cpUk8svnn9wCmDxvY5yqcN7f4T9+kDSzyBBL04yYwj0yJINWU cSJB0Yn+gbFFpi7XBLC4xNYJVcXua1tY9jw0kVPaB1SPS64Q/rlgZQRTUHtH+a7vs63y vIRp9zHLR2rWn/66HbHayQzEbcVnlGGD2ePHsYpTPYbl9pI0t7z9srD3KtteqdHihU0j elvomhvLnRtyQzqqT/af26zj6cSPCAmr0IT4YpoJNKCpubxQRudiHz1OBLAmMhUb7B/m CaWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784686608; x=1785291408; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=o+eEBMpC6tE1uWEH3QJKoYHj9Gqv/C2lkToe2qNQDnE=; b=FlF1GSeTW426FxC/h5m2e88jwqqByfK8QZYxAv3xxCerXCzYWCleVG0YkS3LJN77kk luKxZT6kgV6Mgk0eetC2dUpNE9b//LgJZOhtA19bafz7Ybr/tkssmblkD1dkE956qf8T Srqwy1CtUN2XdmaAu5NqduO8FR8kliB7nQ/Lo7qoxc3QQHrDqjvkxsyGbb0FVJVyVtzL mN5hEA7gdI69fD1Xr8v+cp3uG3J03MpPjLDe3sVTBKeD26ABD4MLlkLY8zJDOwfeN+g5 DxhXh/6g5lfVnE0AAh9aJNAgYstpNDAp5TDYoYxbaz6oYCU7zzTN5KVPXw9s0WwrcnCC ajGg== X-Gm-Message-State: AOJu0YzQbzgmg29TVQBx1YooYiLlj7VOjt2Pxzia0g9kf89y3o1oXPIO uyxSquoaQvWsOyfCwRRoaGilMoXOYtJmfoIepWGUazuisT80zUnlYCSAK+GbQA== X-Gm-Gg: AR+sD13mPk2XdzRkeSPy/fSLtXhqgNtTinpLIAl2t07yxWUfbFjfEzrw3+8SgPDAhI7 Z93tvNREuBd2g59/cC71/yVYX8FmIDEkV4CvwL9mOmP/ZKrmACdRNJk/zS+kaF9MlE2Qc9MMXub OmYY2IjG7G9KvIRxncBRD2wCPDo6bjO1NFS7c4S9E+suV/oilYYdHzMps/17K0wPJMnhwxtoPMo 9x5cIe2Atsav01mIEE+0WxcfQdrn6dFm89hpi3+ybpi2kbWlRpf3sREQJVFCU/y5oSiRLUSznz2 7KF4Osowzum783p6Mag2vFGJ/r2lm4yRvBoOkqYXfkbsHAOW2CIT2X2NbdGELz2yO5il7lMwjb4 mnKZa6Us0mzZqvFQG5yINHiNXgLRSL/Gk6GpBZdw/a9IdAlFS1RNd3BmtJGCn3t22VJ7FhWOgLb kqEHwkFBeTg+n422A= X-Received: by 2002:a05:690c:60c5:b0:80e:38c5:bc17 with SMTP id 00721157ae682-81ef22ef0a5mr60541567b3.7.1784686607626; Tue, 21 Jul 2026 19:16:47 -0700 (PDT) Received: from localhost.localdomain ([185.98.168.14]) by smtp.gmail.com with UTF8SMTPSA id 00721157ae682-81f33bf2d8bsm6868907b3.1.2026.07.21.19.16.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jul 2026 19:16:46 -0700 (PDT) From: Demi Marie Obenour Date: Tue, 21 Jul 2026 21:59:07 -0400 Subject: [PATCH v4 02/20] tools: Add control group manager MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260721-cgroups-v4-2-46b2e5fff7b6@gmail.com> References: <20260721-cgroups-v4-0-46b2e5fff7b6@gmail.com> In-Reply-To: <20260721-cgroups-v4-0-46b2e5fff7b6@gmail.com> To: Spectrum OS Development X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784685545; l=34230; i=demiobenour@gmail.com; s=20250729; h=from:subject:message-id; bh=Jc5e4jYzfxxkvMV8y7NkiEzZ4XnNFHg4fHv+BgXwjRw=; b=UCP9R7Mx/yp6oNTSjcbg372SS+99oac/Sf8Vi55jl082H5DzVE/uEEVyPrf6m1A9AvOBxcRNr yoYLrft7/hPA/WN1lCKYy657yPXAOnQPFpW1bJrIhog25GgXUlE56LN X-Developer-Key: i=demiobenour@gmail.com; a=ed25519; pk=X57Q4/YQDj9t4SBeKaDwvXYKB6quZJVx/DE2Ly2out0= Message-ID-Hash: YGARQW4KM2GJOIFDGL65OQESLUHAQMYY X-Message-ID-Hash: YGARQW4KM2GJOIFDGL65OQESLUHAQMYY X-MailFrom: demiobenour@gmail.com X-Mailman-Rule-Hits: member-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; emergency CC: Demi Marie Obenour , Alyssa Ross X-Mailman-Version: 3.3.10 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: The cgroup-setup Rust program can create and purge cgroups. It can also wait for one to become empty, spawn a program in a cgroup, and more. In the future, it will also support cgroup-based resource control. Locking is used to ensure that concurrent invocations are safe. This program can also be used in an s6 finish script. When passed the args of such a script, it automatically purges the correct cgroup. It also tells s6 to not restart the service if it dumped core. Core dumps are often due to memory corruption, and automatically restarting a service that dumped core makes memory corruption attacks easier. Signed-off-by: Demi Marie Obenour --- .codespellrc | 2 +- host/rootfs/default.nix | 6 +- host/rootfs/file-list.mk | 2 + host/rootfs/image/usr/bin/cgroup-purge | 1 + host/rootfs/image/usr/bin/cgroup-s6-finish | 1 + pkgs/default.nix | 1 + tools/cgroup-setup/Cargo.lock | 67 ++++++ tools/cgroup-setup/Cargo.lock.license | 2 + tools/cgroup-setup/Cargo.toml | 11 + tools/cgroup-setup/default.nix | 18 ++ tools/cgroup-setup/src/cgroup.rs | 349 +++++++++++++++++++++++++++++ tools/cgroup-setup/src/main.rs | 347 ++++++++++++++++++++++++++++ 12 files changed, 803 insertions(+), 4 deletions(-) diff --git a/.codespellrc b/.codespellrc index d8023afc64ec44e98a88c5e397c8d5c681dde063..ae20da8309530759ac729689825a4afc683afa09 100644 --- a/.codespellrc +++ b/.codespellrc @@ -2,4 +2,4 @@ # SPDX-License-Identifier: CC0-1.0 [codespell] -ignore-words-list = crate,passt,rouge,ser +ignore-words-list = crate,passt,rouge,ser,WRONLY diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index 6bfeefbe0a5f76c1538ccb40e5eb8f291f5d3592..ccf626e2ec0f4bf96573dc5edf058c9375bb65f6 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -8,7 +8,7 @@ import ../../lib/call-package.nix ( }: pkgsMusl.callPackage ( -{ spectrum-host-tools, spectrum-router +{ spectrum-host-tools, spectrum-router, spectrum-cgroup-setup , lib, stdenvNoCC, nixos, runCommand, writeClosure, erofs-utils, s6-rc , btrfs-progs, bubblewrap, busybox, cloud-hypervisor, cosmic-files , crosvm, cryptsetup, dejavu_fonts, dbus, execline, foot, fuse3 @@ -27,8 +27,8 @@ let packages = [ btrfs-progs bubblewrap cloud-hypervisor cosmic-files crosvm cryptsetup dbus execline fuse3 inotify-tools iproute2 jq kmod mdevd mount-flatpak s6 - s6-linux-init s6-rc shadow socat spectrum-host-tools spectrum-router - virtiofsd xdg-desktop-portal-spectrum-host + s6-linux-init s6-rc shadow socat spectrum-cgroup-setup spectrum-host-tools + spectrum-router virtiofsd xdg-desktop-portal-spectrum-host (foot.override { allowPgo = false; }) diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk index 3899d620717fc97f42e669e5313c4100dcf5b1cd..e1280ab56d8797e40b9b1c584ab0daef3cda41d7 100644 --- a/host/rootfs/file-list.mk +++ b/host/rootfs/file-list.mk @@ -79,6 +79,8 @@ LINKS = \ image/etc/s6-linux-init/run-image/service/vmm/template/run \ image/lib \ image/sbin \ + image/usr/bin/cgroup-purge \ + image/usr/bin/cgroup-s6-finish \ image/usr/bin/systemd-udevd S6_RC_FILES = \ diff --git a/host/rootfs/image/usr/bin/cgroup-purge b/host/rootfs/image/usr/bin/cgroup-purge new file mode 120000 index 0000000000000000000000000000000000000000..a0c8d8e144d72b69c613eb0613e39acc9df979df --- /dev/null +++ b/host/rootfs/image/usr/bin/cgroup-purge @@ -0,0 +1 @@ +cgroup-setup \ No newline at end of file diff --git a/host/rootfs/image/usr/bin/cgroup-s6-finish b/host/rootfs/image/usr/bin/cgroup-s6-finish new file mode 120000 index 0000000000000000000000000000000000000000..a0c8d8e144d72b69c613eb0613e39acc9df979df --- /dev/null +++ b/host/rootfs/image/usr/bin/cgroup-s6-finish @@ -0,0 +1 @@ +cgroup-setup \ No newline at end of file diff --git a/pkgs/default.nix b/pkgs/default.nix index 44f7b5ff78cb6b9e755292a6a417d0b627ed3fb0..0a13393164ad5d7f752e630763f3f97166479af5 100644 --- a/pkgs/default.nix +++ b/pkgs/default.nix @@ -51,6 +51,7 @@ let driverSupport = true; }; spectrum-router = self.callSpectrumPackage ../tools/router {}; + spectrum-cgroup-setup = self.callSpectrumPackage ../tools/cgroup-setup {}; xdg-desktop-portal-spectrum-host = self.callSpectrumPackage ../tools/xdg-desktop-portal-spectrum-host {}; diff --git a/tools/cgroup-setup/Cargo.lock b/tools/cgroup-setup/Cargo.lock new file mode 100644 index 0000000000000000000000000000000000000000..fe967b3aa02c296c87b6b36ac59253dbe0a32de9 --- /dev/null +++ b/tools/cgroup-setup/Cargo.lock @@ -0,0 +1,67 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "bitflags" +version = "2.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3" + +[[package]] +name = "cgroup-setup" +version = "0.0.0" +dependencies = [ + "libc", + "rustix", +] + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "libc" +version = "0.2.186" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "rustix" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] diff --git a/tools/cgroup-setup/Cargo.lock.license b/tools/cgroup-setup/Cargo.lock.license new file mode 100644 index 0000000000000000000000000000000000000000..aa108acd23886d8302eaf7babff90d1b08ae19fb --- /dev/null +++ b/tools/cgroup-setup/Cargo.lock.license @@ -0,0 +1,2 @@ +SPDX-License-Identifier: EUPL-1.2+ +SPDX-FileCopyrightText: 2026 Demi Marie Obenour diff --git a/tools/cgroup-setup/Cargo.toml b/tools/cgroup-setup/Cargo.toml new file mode 100644 index 0000000000000000000000000000000000000000..7ed6d6a0ea3bbfc4064b9f39383d0788c4bd84e5 --- /dev/null +++ b/tools/cgroup-setup/Cargo.toml @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Alyssa Ross +# SPDX-FileCopyrightText: 2026 Demi Marie Obenour + +[package] +name = "cgroup-setup" +edition = "2024" + +[dependencies] +libc = "0.2.177" +rustix = { version = "1.1.2", features = ["fs"] } diff --git a/tools/cgroup-setup/default.nix b/tools/cgroup-setup/default.nix new file mode 100644 index 0000000000000000000000000000000000000000..9e716b8f270828b733f39961e5ca37a290a872b4 --- /dev/null +++ b/tools/cgroup-setup/default.nix @@ -0,0 +1,18 @@ +# SPDX-FileCopyrightText: 2024 Alyssa Ross +# SPDX-FileCopyrightText: 2025 Yureka Lilian +# SPDX-License-Identifier: MIT + +import ../../lib/call-package.nix ( +{ src, lib, rustPlatform }: + +rustPlatform.buildRustPackage { + name = "spectrum-cgroup-setup"; + + src = lib.fileset.toSource { + root = ../..; + fileset = lib.fileset.intersection src ./.; + }; + sourceRoot = "source/tools/cgroup-setup"; + + cargoLock.lockFile = ./Cargo.lock; +}) (_: {}) diff --git a/tools/cgroup-setup/src/cgroup.rs b/tools/cgroup-setup/src/cgroup.rs new file mode 100644 index 0000000000000000000000000000000000000000..c953d26badfdac0a1e3d7057a867aec3b3247e18 --- /dev/null +++ b/tools/cgroup-setup/src/cgroup.rs @@ -0,0 +1,349 @@ +// SPDX-License-Identifier: EUPL-1.2+ +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour + +use std::ffi::OsStr; +use std::fmt::Display; +use std::fs::File; +use std::io::{Read as _, Seek as _, Write as _}; +use std::os::unix::prelude::*; + +use std::path::{Component, Path, PathBuf}; + +use rustix::fs::{AtFlags, FlockOperation, XattrFlags}; +use rustix::{ + fs::{Mode, OFlags, ResolveFlags}, + io::Errno, +}; + +#[derive(Debug)] +pub(crate) struct Cgroup { + path: PathBuf, + fd: Vec<(OwnedFd, bool)>, +} + +impl AsFd for Cgroup { + fn as_fd(&self) -> BorrowedFd<'_> { + self.fd.last().unwrap().0.as_fd() + } +} + +fn assert_single_component(component: &[u8]) { + match component { + b"" | b"." | b".." => panic!("bad component"), + _ if component.contains(&b'\0') => panic!("NUL in component"), + _ if component.contains(&b'/') => panic!("/ in component"), + _ => {} + } +} + +impl Cgroup { + pub fn new(exclusive: bool) -> Result { + let cgroup_root = rustix::fs::openat2( + rustix::fs::CWD, + Path::new("/sys/fs/cgroup"), + OFlags::DIRECTORY | OFlags::RDONLY | OFlags::CLOEXEC | OFlags::NOFOLLOW, + Mode::empty(), + ResolveFlags::NO_MAGICLINKS | ResolveFlags::NO_SYMLINKS, + ) + .map_err(|e| format!("Cannot open /sys/fs/cgroup: {e}"))?; + + let lock_operation = if exclusive { + FlockOperation::LockExclusive + } else { + FlockOperation::LockShared + }; + rustix::fs::flock(cgroup_root.as_fd(), lock_operation) + .map_err(|e| format!("Cannot lock /sys/fs/cgroup: {e}"))?; + Ok(Self { + path: PathBuf::from("/sys/fs/cgroup"), + fd: vec![(cgroup_root, exclusive)], + }) + } + + pub fn enable_delegation(&self, depth: usize) -> Result<(), Errno> { + let (fd, exclusive) = &self.fd[self.fd.len() - depth]; + assert!(exclusive); + rustix::fs::fsetxattr(fd.as_fd(), c"user.delegate", b"1", XattrFlags::empty()) + } + + pub fn enable_subtree_control(&self, depth: usize) -> Result<(), String> { + let (fd, exclusive) = &self.fd[self.fd.len() - depth]; + assert!(exclusive); + let p = Path::new("cgroup.controllers"); + let mut buf = self.read_control_file(fd.as_fd(), p)?; + let mut subtree = vec![]; + if buf.ends_with(b"\n") { + buf.pop(); + } + for controller in buf.split(|&b| b == b' ').filter(|e| !e.is_empty()) { + for &c in controller { + if c <= b' ' || c >= 0x7F { + return Err(format!("Bad byte {c} in cgroup.controllers")); + } + } + if !subtree.is_empty() { + subtree.push(b' '); + } + subtree.push(b'+'); + subtree.extend_from_slice(controller); + } + if !subtree.is_empty() { + self.write_cgroup_value("cgroup.subtree_control", str::from_utf8(&subtree).unwrap())?; + } + Ok(()) + } + + pub fn read_control_file(&self, fd: BorrowedFd, p: &Path) -> Result, String> { + let mut buf = Vec::new(); + let err = |e: &dyn Display, p: &Path, msg: &str| { + let path = self.path.join(p); + format!("Cannot {msg} {path:?}: {e}") + }; + File::from(open_subtree_raw(Path::new(p), fd.as_fd()).map_err(|e| err(&e, p, "open"))?) + .read_to_end(&mut buf) + .map_err(|e| err(&e, p, "read"))?; + Ok(buf) + } + + /// Open a single component as a sub-cgroup + fn open_sub_cgroup_raw(&self, access: OFlags, component: &[u8]) -> Result { + assert_single_component(component); + rustix::fs::openat2( + self.as_fd(), + Path::new(OsStr::from_bytes(component)), + OFlags::CLOEXEC | OFlags::NOFOLLOW | access, + Mode::empty(), + ResolveFlags::NO_SYMLINKS + | ResolveFlags::NO_MAGICLINKS + | ResolveFlags::BENEATH + | ResolveFlags::NO_XDEV, + ) + } + + pub fn open_sub_cgroup( + &mut self, + path: &std::path::Path, + exclusive: bool, + allow_missing: bool, + ) -> Result { + let mut iter = path.components().peekable(); + while let Some(component) = iter.next() { + let component = match component { + Component::Normal(component) => component, + _ => unreachable!(), + }; + let sub_fd = match self + .open_sub_cgroup_raw(OFlags::DIRECTORY | OFlags::RDONLY, component.as_bytes()) + { + Ok(sub_fd) => { + self.path.push(component); + sub_fd + } + Err(Errno::NOENT) if allow_missing => return Ok(false), + Err(e) => { + return Err(format!( + "Cannot open sub-cgroup {component:?} of {:?}: {e}", + self.path + )); + } + }; + let exclusive = exclusive && iter.peek().is_none(); + let lock_operation = if exclusive { + FlockOperation::LockExclusive + } else { + FlockOperation::LockShared + }; + rustix::fs::flock(sub_fd.as_fd(), lock_operation).map_err(|e| { + let msg = format!("Cannot lock sub-cgroup {:?}: {e}", self.path); + self.path.pop(); + msg + })?; + self.fd.push((sub_fd, exclusive)); + } + Ok(true) + } + + pub fn open_subtree(&self, path: &std::path::Path) -> Result { + let dirfd = self.as_fd(); + open_subtree_raw(path, dirfd) + } + + fn exclusive(&self) -> bool { + self.fd.last().unwrap().1 + } + + pub fn joined_path(&self, p: &Path) -> PathBuf { + let mut owned_p = self.path.clone(); + owned_p.push(p); + owned_p + } + + pub fn wait_for_empty(&self) -> std::io::Result<()> { + assert!(self.exclusive()); + let wait_file = self.open_subtree(std::path::Path::new("cgroup.events"))?; + let poll_fd = wait_file.as_raw_fd(); + let mut wait_fd = File::from(wait_file); + let mut fds = libc::pollfd { + fd: poll_fd, + events: libc::POLLPRI | libc::POLLERR, + revents: 0, + }; + let mut v = vec![]; + loop { + v.clear(); + wait_fd + .seek(std::io::SeekFrom::Start(0)) + .expect("Seek on control group file should succeed"); + wait_fd + .read_to_end(&mut v) + .expect("reading from control group should work"); + if v.split(|&c| c == b'\n').any(|line| line == b"populated 0") { + break; + } + // SAFETY: FFI call, valid arguments, fds contains 1 element + if unsafe { libc::poll(&raw mut fds, 1, -1) } != 1 { + panic!("poll failed"); + } + } + Ok(()) + } + + pub(crate) fn make_child(&mut self, path: &Path) -> Result<(), Errno> { + assert!(self.exclusive()); + let component = path.as_os_str().as_bytes(); + assert_single_component(component); + match rustix::fs::mkdirat( + self.as_fd(), + path, + Mode::RUSR + | Mode::WUSR + | Mode::XUSR + | Mode::RGRP + | Mode::XGRP + | Mode::ROTH + | Mode::XOTH, + ) { + Ok(()) | Err(Errno::EXIST) => {} + bad => return bad, + } + let p = self.open_sub_cgroup_raw(OFlags::RDONLY | OFlags::DIRECTORY, component)?; + // exclusive lock on parent acts as exclusive lock on child + self.fd.push((p, true)); + self.path.push(path); + Ok(()) + } + + pub(super) fn purge(&mut self, path: &Path) -> Result<(), String> { + assert!(self.exclusive()); + match rustix::fs::unlinkat(self.as_fd(), Path::new(path), AtFlags::REMOVEDIR) { + // Trying to purge a deleted cgroup is not an error. + Ok(()) | Err(Errno::NOENT) => return Ok(()), + Err(Errno::BUSY) => {} + Err(e) => return Err(format!("Cannot purge {:?}: {e}", self.joined_path(path))), + } + if !self.open_sub_cgroup(path, true, true)? { + return Ok(()); + } + + rustix::fs::flock( + self.fd[self.fd.len() - 2].0.as_fd(), + FlockOperation::LockShared, + ) + .map_err(|e| format!("Cannot relock {:?}: {e}", self.path.parent()))?; + self.write_cgroup_value("cgroup.kill", "1")?; + self.wait_for_empty() + .map_err(|e| format!("Cannot wait for cgroup to become empty: {e}"))?; + let fd = self.fd.pop().unwrap().0; + let v = (|| { + remove_recursively(fd, 1000) + .map_err(|e| format!("Cannot remove {:?}: {e}", self.path))?; + rustix::fs::flock(self.as_fd(), FlockOperation::LockExclusive) + .map_err(|e| format!("Cannot lock {:?}: {e}", self.path))?; + match rustix::fs::unlinkat( + self.as_fd(), + Path::new(self.path.file_name().unwrap()), + AtFlags::REMOVEDIR, + ) { + // something might have re-created the cgroup in the meantime, which is okay + Ok(()) | Err(Errno::BUSY) => Ok(()), + Err(e) => Err(format!("Cannot lock {:?}: {e}", self.path)), + } + })(); + assert!(self.path.pop()); + v + } + + pub(crate) fn write_cgroup_value(&self, name: &str, value: &str) -> Result<(), String> { + let path = Path::new(name); + let fd = rustix::fs::openat2( + self.as_fd(), + path, + OFlags::NOATIME | OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::WRONLY, + Mode::empty(), + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV, + ) + .map_err(|e| format!("Cannot open {:?}: {}", self.joined_path(Path::new(name)), e))?; + File::from(fd).write_all(value.as_bytes()).map_err(|e| { + format!( + "Cannot write {:?} to {:?}: {}", + value, + self.joined_path(Path::new(name)), + e + ) + }) + } +} + +fn open_subtree_raw(path: &Path, dirfd: BorrowedFd<'_>) -> Result { + rustix::fs::openat2( + dirfd, + path, + OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY, + Mode::empty(), + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV, + ) +} + +fn remove_recursively(fd: OwnedFd, remaining_depth: usize) -> Result<(), Errno> { + if remaining_depth < 1 { + panic!("control groups too deeply nested"); + } + let mut d = rustix::fs::Dir::new(fd).expect("cannot start iterating"); + while let Some(element) = d.next() { + let element = element.expect("Iterating through a cgroup directory failed?"); + if element.file_type() != rustix::fs::FileType::Directory { + continue; + } + + let remaining_depth = remaining_depth - 1; + let d: &rustix::fs::Dir = &d; + let dirfd = d.fd().unwrap(); + let path = element.file_name(); + remove_all(remaining_depth, dirfd, path)?; + } + drop(d); + Ok(()) +} + +fn remove_all( + remaining_depth: usize, + dirfd: BorrowedFd<'_>, + path: &std::ffi::CStr, +) -> Result<(), Errno> { + if path == c"." || path == c".." { + return Ok(()); + } + if rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR).is_ok() { + return Ok(()); + } + let fd = rustix::fs::openat2( + dirfd, + path, + OFlags::CLOEXEC | OFlags::NOFOLLOW | OFlags::RDONLY | OFlags::DIRECTORY, + Mode::empty(), + ResolveFlags::NO_SYMLINKS | ResolveFlags::BENEATH | ResolveFlags::NO_XDEV, + )?; + remove_recursively(fd, remaining_depth)?; + rustix::fs::unlinkat(dirfd, path, AtFlags::REMOVEDIR)?; + Ok(()) +} diff --git a/tools/cgroup-setup/src/main.rs b/tools/cgroup-setup/src/main.rs new file mode 100644 index 0000000000000000000000000000000000000000..2e7a4e25213a4aa2449b292f8005e1da23cbb1f9 --- /dev/null +++ b/tools/cgroup-setup/src/main.rs @@ -0,0 +1,347 @@ +// SPDX-License-Identifier: EUPL-1.2+ +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour + +use std::{ + ffi::{OsStr, OsString}, + os::unix::prelude::*, + path::{Path, PathBuf}, +}; + +use crate::cgroup::Cgroup; + +mod cgroup; + +fn check_path(path: &OsStr) -> Result<(), String> { + if path.is_empty() { + return Ok(()); + } + + for component in path.as_bytes().split(|&b| b == b'/') { + match component { + b"" | b"." | b".." => { + return Err(format!("Path {path:?} has empty, ., or .. component")); + } + // Cannot happen: command line arguments have no NUL byte, + // and /proc/self/cgroup having a NUL byte is a kernel bug. + _ if component.contains(&b'\0') => panic!("Path {path:?} has NUL byte"), + _ if component.len() > 255 => { + return Err(format!( + "Path {path:?} has component {:?} that is longer than 255 bytes", + OsStr::from_bytes(component) + )); + } + _ => {} + } + } + + Ok(()) +} + +/// Get the path of the cgroup for the provided command-line argument. +/// Returns an empty path if the path is "/", or if it is "." and the +/// current cgroup is "/". +/// +/// # Errors +/// +/// Fails if the provided path is invalid or empty, or if it is relative +/// and the local cgroup cannot be determined. +fn get_cgroup(cgroup_path: OsString) -> Result { + if cgroup_path.as_bytes().starts_with(b"/") { + let mut cgroup_path = cgroup_path.into_vec(); + cgroup_path.remove(0); + if cgroup_path.is_empty() { + return Err("cgroup path cannot be /".to_owned()); + } + let cgroup_path = OsString::from_vec(cgroup_path); + check_path(&cgroup_path)?; + Ok(cgroup_path.into()) + } else if cgroup_path.is_empty() { + Err("cgroup path cannot be empty".to_owned()) + } else { + check_path(&cgroup_path)?; + let mut local_cgroup = local_cgroup()?; + local_cgroup.push(cgroup_path); + Ok(local_cgroup) + } +} + +/// Open the cgroup corresponding to the provided path. +/// It must have already been made relative to `/sys/fs/cgroup`. +/// +/// # Errors +/// +/// Fails if the cgroup operation fails. +fn open_cgroup(path: &Path, exclusive: bool) -> Result { + if path.as_os_str().is_empty() { + Cgroup::new(exclusive) + } else { + let mut cgroup = Cgroup::new(false)?; + cgroup.open_sub_cgroup(path, exclusive, false)?; + Ok(cgroup) + } +} + +/// Open the cgroup corresponding to the provided path's parent. +/// It is made relative to the process's own cgroup if needed. +/// +/// # Errors +/// +/// Fails if the cgroup operation fails. +fn open_relative_cgroup(arg: OsString) -> Result<(PathBuf, Cgroup), String> { + let path = get_cgroup(arg)?; + let cgroup = open_cgroup(path.parent().expect("always has a parent"), true)?; + Ok((path, cgroup)) +} + +fn main() { + let mut args = std::env::args_os(); + let Some(prog_name) = args.next() else { + eprintln!("No command line arguments (argv[0] is NULL)"); + std::process::exit(1); + }; + match main_(&prog_name, args) { + Ok(()) => {} + Err(e) => { + eprintln!("{prog_name:?}: {}", e); + std::process::exit(1); + } + } +} + +fn main_(prog_name: &OsStr, mut args: std::env::ArgsOs) -> Result<(), String> { + match prog_name + .as_bytes() + .split(|&b| b == b'/') + .next_back() + .unwrap() + { + b"cgroup-s6-finish" => { + return s6_finish(&mut args); + } + b"cgroup-setup" => {} + b"cgroup-purge" => { + if args.len() != 1 { + return Err(format!( + "cgroup-purge takes one argument, got {}", + args.len() + )); + } + let cgroup_path = args.next().unwrap(); + let (path, mut cgroup) = open_relative_cgroup(cgroup_path)?; + let cgroup_target = Path::new(path.file_name().unwrap()); + return cgroup.purge(cgroup_target); + } + _ => { + return Err(format!( + "must be invoked as \"cgroup-setup\" \ + \"cgroup-purge\", or \"cgroup-s6-finish\", \ + got {prog_name:?}", + )); + } + }; + let mut leaf = false; + let mut cgroup_path; + let mut delegate = false; + let mut init_subtree = false; + let mut child_name: Option<&'static OsStr> = None; + let mut wait = true; + loop { + cgroup_path = args.next(); + let Some(ref arg_) = cgroup_path else { + break; + }; + let arg_ = arg_.as_bytes(); + if arg_ == b"--" { + cgroup_path = args.next(); + break; + } + if !arg_.starts_with(b"-") { + break; + } + + if !arg_.starts_with(b"--") { + return Err("takes no short options".to_owned()); + } + + match &arg_[2..] { + b"leaf" => leaf = true, + b"delegate" => delegate = true, + b"init-subtree" => init_subtree = true, + b"wait" => wait = true, + b"no-wait" => wait = false, + b"child-name" if child_name.is_none() => match args.next() { + Some(arg) => child_name = Some(arg.leak()), + None => return Err("--child-name: missing argument".to_owned()), + }, + b"child-name" => return Err("--child-name: cannot be used twice".to_owned()), + arg => match str::from_utf8(arg) { + Ok(e) => return Err(format!("unknown long option {e:?}")), + Err(_) => return Err("long option isn't UTF-8".to_owned()), + }, + } + } + + let default_child_name = OsStr::from_bytes(b"$inner.service"); + + let child_name = Path::new(child_name.unwrap_or(default_child_name)); + + let Some(mut cgroup_path) = cgroup_path else { + return Err("have no positional arguments, expected at least 1".to_owned()); + }; + + // Allow --init-subtree . + if cgroup_path.as_bytes() == b"." && init_subtree && !leaf { + cgroup_path = child_name.to_owned().into(); + leaf = true; + } + + let (path, mut cgroup) = open_relative_cgroup(cgroup_path)?; + let cgroup_target = Path::new(path.file_name().unwrap()); + cgroup + .make_child(cgroup_target) + .map_err(|e| format!("Cannot make child cgroup: {e}"))?; + if wait { + cgroup + .wait_for_empty() + .map_err(|e| format!("Cannot wait for {path:?} to be empty: {e}"))?; + } + let pid = std::process::id().to_string(); + if leaf { + if args.len() != 0 { + // If we aren't delegating any cgroups, don't create a sub-cgroup. + cgroup + .write_cgroup_value("cgroup.procs", &pid) + .map_err(|e| format!("Cannot write to {path:?}/cgroup.procs: {e}"))?; + } + } else { + // If the child process will need to manage cgroups itself, it will need + // to set up a sub-cgroup due to the "no internal processes" rule. It's + // simplest to just do it automatically. + cgroup.make_child(Path::new(child_name)).map_err(|e| { + format!( + "Cannot create child cgroup {}/{}: {e}", + path.display(), + child_name.display() + ) + })?; + if args.len() != 0 { + cgroup + .write_cgroup_value("cgroup.procs", &pid) + .map_err(|e| { + format!( + "Cannot write to {}/{}/cgroup.procs: {e}", + path.display(), + child_name.display() + ) + })?; + } + } + if !leaf { + cgroup.enable_subtree_control(2)?; + } + if init_subtree { + cgroup.enable_subtree_control(1)?; + } + if delegate { + cgroup + .enable_delegation(1) + .map_err(|e| format!("Cannot enable cgroup delegation in {path:?}: {e}"))?; + } + let Some(program_name) = args.next() else { + return Ok(()); + }; + let e = std::process::Command::new(&program_name).args(args).exec(); + Err(format!("Cannot spawn child {:?}: {}", program_name, e)) +} + +fn s6_finish(args: &mut std::env::ArgsOs) -> Result<(), String> { + if args.len() < 3 { + return Err(format!( + "s6 finish scripts take at least 3 arguments, got {}", + args.len() + )); + } + let status = parse_digit_string(&args.next().unwrap(), "exit status")?; + let signal = args.next().unwrap(); + let signal = if status == 256 { + Some(parse_digit_string(&signal, "signal number")?) + } else { + None + }; + let service = args.next().unwrap(); + + let (path, mut cgroup) = open_relative_cgroup(service)?; + let cgroup_target = Path::new(path.file_name().unwrap()); + let exit_125 = if let Some(signal) = signal { + match signal as libc::c_int { + libc::SIGBUS + | libc::SIGFPE + | libc::SIGABRT + | libc::SIGTRAP + | libc::SIGSEGV + | libc::SIGILL => { + // Process *crashed*, indicating a *possible exploit attempt*. + // s6 should *not* restart it. This is distinct from a Rust panic, + // which is much less likely to indicate memory corruption. + true + } + _ => false, + } + } else { + false + }; + if exit_125 { + // Ignore panics. Exit status is more important. + // We already had a core dump. + let _ = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { + match cgroup.purge(cgroup_target) { + Ok(()) => {} + Err(e) => { + eprintln!("cgroup-s6-finish: Failed to purge cgroup: {e}") + } + }; + })); + std::process::exit(125) + } else { + cgroup.purge(cgroup_target) + } +} + +fn parse_digit_string(digits: &OsStr, msg: &str) -> Result { + let checked = match str::from_utf8(digits.as_bytes()) { + Ok(s) => s, + Err(e) => return Err(format!("{msg} is not UTF-8: {e}")), + }; + let r = checked + .parse::() + .map_err(|e| format!("{msg} {digits:?} is a bad 16-bit number: {e}"))?; + match checked.as_bytes() { + b"0" | [b'1'..=b'9', ..] => Ok(r), + [b'0', ..] => Err(format!("{msg} {} has a leading 0", digits.display())), + _ => Err(format!("{msg} {} starts with +", digits.display())), + } +} + +fn local_cgroup() -> Result { + let mut local_cgroup: Vec = std::fs::read("/proc/thread-self/cgroup") + .map_err(|e| format!("cannot read /proc/thread-self/cgroup: {e}"))?; + let local_cgroup_len = local_cgroup.len(); + if local_cgroup_len < 5 + || local_cgroup[..4] != *b"0::/" + || local_cgroup[local_cgroup_len - 1] != b'\n' + || local_cgroup[4..local_cgroup_len - 1].contains(&b'\n') + { + // It's possible to get here if the cgroup path contains a newline, + // but that never happens in Spectrum. + return Err(format!( + "Invalid contents {local_cgroup:?} of /proc/thread-self/cgroup - \ + do you have cgroups v1 mounted instead of cgroups v2?" + )); + } + + local_cgroup.copy_within(4..local_cgroup_len - 1, 0); + local_cgroup.truncate(local_cgroup_len - 5); + let local_cgroup = OsString::from_vec(local_cgroup); + check_path(&local_cgroup).unwrap(); + Ok(PathBuf::from(local_cgroup)) +} -- 2.55.0