From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id 2C28493F9; Mon, 27 Jul 2026 11:22:47 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id C55F893CB; Mon, 27 Jul 2026 11:22:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_MISSING,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=4.0.1 Received: from fhigh-a6-smtp.messagingengine.com (fhigh-a6-smtp.messagingengine.com [103.168.172.157]) by atuin.qyliss.net (Postfix) with ESMTPS id 87FF693CA for ; Mon, 27 Jul 2026 11:22:42 +0000 (UTC) Received: from phl-compute-04.internal (phl-compute-04.internal [10.202.2.44]) by mailfhigh.phl.internal (Postfix) with ESMTP id A08A71400116; Mon, 27 Jul 2026 07:22:41 -0400 (EDT) Received: from phl-frontend-04 ([10.202.2.163]) by phl-compute-04.internal (MEProxy); Mon, 27 Jul 2026 07:22:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alyssa.is; h=cc :cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1785151361; x=1785237761; bh=ZSpWjLs9cB vx1+9SzTbuCGR4/+3Qpf7fTFeaZdcGT54=; b=IWgJDVvdFFddf4mvmGYQz4aw3x Z5PQve0rfG5uiK0+5rpT23OI5i1/FiXLLqgPyd4kTPCEfWkg9VmrP12h8eiQMmBw Sm0PGorSZc7phDPgp5HdYhdU08p46nsGZI58CqC+UfPz2KHG3NzQlWnTK+r1frpO vYwxzznWVweMjoOXwLDXAN02Dux5dqRl4LehJNFIEzDpLNg3welpdowITrz+xP+i GH9v4lDRLrXxQNuQRNtHBYIK9kn7rCdEgjeJUIXTS4frrCPJFSO2Ze8Mm2p82LzD LQTmU4oT5z4B9d16OPla4rmMxnxZ6Hp6Xg36QygIy8IwsFO7FLbLQETLLS2Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t= 1785151361; x=1785237761; bh=ZSpWjLs9cBvx1+9SzTbuCGR4/+3Qpf7fTFe aZdcGT54=; b=QJxXEjIpC/ZZ1zyvZXzelPzspqayuxWnpG0si1XRjcC126Cv53P k5w0aO4yItHVhK5/EIhC8tFkaw7HnU8wbkmxlWBB36M3ntEgtXcQ11RITEZX6TS6 u5vOLxfqihxae8B3iNllcBTfVYXw/vitJjuJZiZiAHBFtfLbH+0v70g4pg5Wv3xH qxf3WGw/Kwh6AMPwcvSg/SYnA08p22wXILg8B0spA85MuiyDmGb3ny7dSOhRGWKO E6A6uibCGOZ9VItDMtpyROiUGEIUtzvQTYiuz+glL48eQeBexoo1VxsrQqU01g1N OHd2MJ4WrpdphffwCgoVt5MrDDx7l6/s/hg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTGDqo26LvGWrpUUVan0cqQG+0eSbE9FU6Fv1O/bBq/VjH9TxtffaR20VlTB1NXP8c YE4DQXCxKaXwGsX4fpI8UJ51ZWdZsZGngTc9dNbJBHoGqjHSfVBJSuMt5WZK7quAHYn+Oj DDmzmPi74xi6p7BuHqd985RFzKpR7LRVCXj4BrwWhVh52vWyc1+ePCogh7SMeSq45/ChaY KfD7d7Aba07NStnjoLRpQlvQ1kmQMy/DzhTAODtc+MRxy20IZ6wOu5bsMmH2rVY03qcEae j5cOYMBMDT6LXXGT/JK5AaFIMLZWAt7Fam5+P+4sb5G/pOUBRsyd4rxAAlI5hWiNpFRWO5 dHXegS1KX6qfFi9zEO8crvNtNVZqZj7yBaB/k8qpbIsDQJObE53ZZSe/5MVAgRlF1dsyU4 lmETy0xU0A1OBEY+57GldBYNGxRHSqRRJ0mf8s8mLecW5A3TVdFcdTkSpTRXcIME5qs86U 2o+QCdmOinG9lpLcvM4YOfks4uihFXkAzrMq9Qow8AOrE7H67zZw4j8Cq+PqJd9jZDP5NH Jmvg7Knno9N4RtOyYuokad2CC0dou8LoOeRsw+xzHrZ1aIkfdMKmi0dKKB1Bhl39a0DzJZ ERTTzmvrM1/MSdngtG3eujkosABQBtDkkn5VVE3GTjvGUC4l3uvCsHvGM2BQ X-ME-Proxy: Feedback-ID: i12284293:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 27 Jul 2026 07:22:40 -0400 (EDT) Received: by fw12.qyliss.net (Postfix, from userid 1000) id 85B7BC4D754E; Mon, 27 Jul 2026 13:22:39 +0200 (CEST) From: Alyssa Ross To: Demi Marie Obenour Subject: Re: [PATCH v4 03/20] Documentation: Mention control groups In-Reply-To: <20260721-cgroups-v4-3-46b2e5fff7b6@gmail.com> References: <20260721-cgroups-v4-0-46b2e5fff7b6@gmail.com> <20260721-cgroups-v4-3-46b2e5fff7b6@gmail.com> Date: Mon, 27 Jul 2026 13:22:38 +0200 Message-ID: <87ldawlls1.fsf@alyssa.is> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Message-ID-Hash: FJLF36QHIBJCAG65FLW7RCBG3ESR2QBN X-Message-ID-Hash: FJLF36QHIBJCAG65FLW7RCBG3ESR2QBN X-MailFrom: hi@alyssa.is X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Spectrum OS Development , Valentin Gagarin X-Mailman-Version: 3.3.10 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --=-=-= Content-Type: text/plain Demi Marie Obenour writes: > Signed-off-by: Demi Marie Obenour > --- > Documentation/doc/development/control-groups.adoc | 88 +++++++++++++++++++++++ > 1 file changed, 88 insertions(+) CCing Valentin for documentation oversight. I found this very clearly written and easy to read. :) > diff --git a/Documentation/doc/development/control-groups.adoc b/Documentation/doc/development/control-groups.adoc > new file mode 100644 > index 0000000000000000000000000000000000000000..6ce33f21a230d012a690fc5deb2ba597d1d01ef1 > --- /dev/null > +++ b/Documentation/doc/development/control-groups.adoc > @@ -0,0 +1,88 @@ > += Control groups in Spectrum > + > +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour > +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 > + > +Linux control groups (cgroups) can be used for several purposes: > + > +1. They allow waiting for a group of processes to exit. > +2. They allow terminating a group of processes. > +3. They allow limiting a group of processes' access to resources. > + > +Spectrum currently uses the first two. The third is not yet used, > +but will be in the future. > + > +== Control Group Hierarchy > + > +Spectrum uses the following cgroup hierarchy: > + > +1. There is a `/vm-services.slice` cgroup that contains all the per-VM > + services on the system. > +2. The per-VM services for each VM are under `/vm-services.slice/vm-${VM}.slice`, > + where `${VM}` is replaced by the VM's ID. > +3. Each per-VM service is under `/vm-services.slice/vm-${VM}.slice/${SERVICE_NAME}`, > + where `${VM}` is replaced by the VM's ID and `${SERVICE_NAME}` is replaced by > + the name of the service. > +4. The VMM runs under `/vm-services.slice/vm-${VM}.slice/vmm`. > + > +If a cgroup contains child cgroups, it likely contains a `$inner.service` > +cgroup. This is where programs that would otherwise run in the cgroup itself > +are placed. Generally, these programs are instances of `s6-svscan` and/or > +`s6-supervise`. This section doesn't mention where non-per-VM services go. It might also be nice to explicitly mention the "no internal processes" rule as the reason for $inner.service, in case people are unfamiliar with it. > +== Using Control Groups > + > +When adding a new s6 service, one should carefully consider whether it > +should be placed in a control group. Most services should be placed in > +a control group, with only a few exceptions: > + > +1. Services, such as `getty`, that spawn background processes. > +2. Loggers. > +3. Trivial services that don't do anything. We shouldn't have any services that don't do anything! Maybe we could be more specific? Or just say "trivial services where cgroup are unnecessary" or something. > + > +Generally, it's best to set the control group up as the first thing > +the service does. To do that, use `cgroup-setup --leaf -- $1 COMMAND_LINE`, The " -- " here gets turned into an en-dash when rendered, so must need to be escaped somehow. > +where `$1` should be the service name and `COMMAND_LINE` is the program > +to run in a cgroup. > + > +If you use execline for your run script, this is as simple as: > + > +.run > +.... > +#!/bin/execlineb -WS1 > + > +cgroup-setup --leaf -- $1 > +# rest of script comes here > +.... > + > +If the service exits, it's usually best to terminate any programs left > +behind with SIGKILL and remove the control group. In Spectrum, this is > +called "purging" the cgroup. To purge the cgroup when a service exits, > +make the `finish` script invoke `/usr/bin/cgroup-s6-finish`. The first > +two command line arguments must be the first two arguments passed to the > +`finish` script. The third argument must be the path to the cgroup to > +be purged relative to the cgroup the program itself is in. This is > +usually, but not always, the third argument to the `finish` script. > + > +When invoked as `cgroup-s6-finish`, `cgroup-setup` checks if > +the service exited due to a signal that caused it to dump core. If it > +did, `cgroup-s6-finish` exits with status 125, ensuring that > +`s6-supervise` will *not* restart it. This is intentional: if a service > +crashes due to a fatal signal, this is possibly a sign of memory > +corruption. Restarting the service in this case can turn an unreliable > +memory corruption exploit into a reliable one. Rust panics do not cause > +core dumps, so the service will be restarted afterwards. (Just noting that in the review of cgroup-setup itself, I said that to me this does not seem likely something that should be part of cgroup-setup itself. If that does change, this will need to be updated.) > +One can also use `cgroup-purge` to purge a cgroup explicitly. This is > +used to stop the VMM and all per-VM services when a VM is shut down. > + > +== Future plans > + > +Control groups are designed around a single writer process controlling each > +of them. Many Linux distros use systemd for this, but Spectrum doesn't use > +systemd. The only persistent per-service process is s6-supervise, but that > +doesn't have control group support. > + > +Instead, the plan is to have a database containing this information. > +Whether this will be in the `data/` subdirectory of the service directory > +or a separate system-wide database has not yet been determined. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQQGoGac7QfI+H5ZtFCZddwkt31pFQUCamc/fgAKCRCZddwkt31p Ff60AP9Kq6olltgpE1heHDhapgaxydDE+yjrGqqHWtIDnshcogEAhFzGUhTNPzEn Y0J0uvIi5hrETCUmCXACNhvmfZlnLAo= =umNk -----END PGP SIGNATURE----- --=-=-=--