[PATCH v3 04/22] Documentation: Mention control groups

[Demi Marie Obenour <demiobenour@gmail.com>]

Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
 Documentation/doc/development/control-groups.adoc | 82 +++++++++++++++++++++++
 1 file changed, 82 insertions(+)

diff --git a/Documentation/doc/development/control-groups.adoc b/Documentation/doc/development/control-groups.adoc
new file mode 100644
index 0000000000000000000000000000000000000000..9866761723ba8c77f9242d4ffdabaebad3f45aa4
--- /dev/null
+++ b/Documentation/doc/development/control-groups.adoc
@@ -0,0 +1,82 @@
+= Control groups in Spectrum
+
+// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
+// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0
+
+Linux control groups (cgroups) can be used for several purposes:
+
+1. They allow waiting for a group of processes to exit.
+2. They allow terminating a group of processes.
+3. They allow limiting a group of processes' access to resources.
+
+Spectrum currently uses the first two.  The third is not yet used,
+but will be in the future.
+
+== Control Group Hierarchy
+
+Spectrum uses the following cgroup hierarchy:
+
+1. There is a `/vm-services.slice` cgroup that contains all the per-VM
+   services on the system.
+2. The per-VM services for each VM are under `/vm-services.slice/vm-${VM}.slice`,
+   where `${VM}` is replaced by the VM's ID.
+3. Each per-VM service is under `/vm-services.slice/vm-${VM}.slice/${SERVICE_NAME}`,
+   where `${VM}` is replaced by the VM's ID and `${SERVICE_NAME}` is replaced by
+   the name of the service.
+4. The VMM runs under `/vm-services.slice/vm-${VM}.slice/vmm`.
+
+If a cgroup contains child cgroups, it likely contains a `$inner.service`
+cgroup.  This is where programs that would otherwise run in the cgroup itself
+are placed.  Generally, these programs are instances of `s6-svscan` and/or
+`s6-supervise`.
+
+== Using Control Groups
+
+When adding a new s6 service, one should carefully consider whether it
+should be placed in a control group.  Most services should be placed in
+a control group, with only a few exceptions:
+
+1. Services, such as `getty`, that spawn background processes.
+2. Loggers.
+3. Trivial services that don't do anything.
+
+Generally, it's best to set the control group up as the first thing
+the service does.  To do that, use `cgroup-setup --leaf -- $1 COMMAND_LINE`,
+where `$1` should be the service name and `COMMAND_LINE` is the program
+to run in a cgroup.
+
+If you use execline for your run script, this is as simple as:
+
+.run
+....
+#!/bin/execlineb -WS1
+
+cgroup-setup --leaf -- $1
+# rest of script comes here
+....
+
+If the services exits, it's usually best to terminate any programs left
+behind with SIGKILL and remove the control group.  To do that, make
+the `finish` script be a symbolic link to `/usr/bin/cgroup-setup`.
+`cgroup-setup` recognizes when it has been invoked as `finish` and
+purges the cgroup automatically.
+
+When invoked as `finish`, `cgroup-setup` checks if the service exited
+due to a signal that caused it to dump core.  If it did, `cgroup-setup`
+exits with status 125, ensuring that `s6-supervise` will *not* restart it.
+This is intentional: if a service crashes due to a fatal signal, this
+is possibly a sign of memory corruption.  Restarting the service in this
+case can turn an unreliable memory corruption exploit into a reliable one.
+Rust panics do not cause core dumps, so the service will be restarted
+afterwards.
+
+== Future plans
+
+Control groups are designed around a single writer process controlling each
+of them.  Many Linux distros use systemd for this, but Spectrum doesn't use
+systemd.  The only persistent per-service process is s6-supervise, but that
+doesn't have control group support.
+
+Instead, the plan is to have a database containing this information.
+Whether this will be in the `data/` subdirectory of the service directory
+or a separate system-wide database has not yet been determined.

-- 
2.55.0