1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
| | = Control groups in Spectrum
// SPDX-FileCopyrightText: 2026 Demi Marie Obenour <demiobenour@gmail.com>
// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0
Linux control groups (cgroups) can be used for several purposes:
1. They allow waiting for a group of processes to exit.
2. They allow terminating a group of processes.
3. They allow limiting a group of processes' access to resources.
Spectrum currently uses the first two. The third is not yet used,
but will be in the future.
== Control Group Hierarchy
Spectrum uses the following cgroup hierarchy:
1. There is a `/vm-services.slice` cgroup that contains all the per-VM
services on the system.
2. The per-VM services for each VM are under `/vm-services.slice/vm-${VM}.slice`,
where `${VM}` is replaced by the VM's ID.
3. Each per-VM service is under `/vm-services.slice/vm-${VM}.slice/${SERVICE_NAME}`,
where `${VM}` is replaced by the VM's ID and `${SERVICE_NAME}` is replaced by
the name of the service.
4. The VMM runs under `/vm-services.slice/vm-${VM}.slice/vmm`.
If a cgroup contains child cgroups, it likely contains a `$inner.service`
cgroup. This is where programs that would otherwise run in the cgroup itself
are placed. Generally, these programs are instances of `s6-svscan` and/or
`s6-supervise`.
== Using Control Groups
When adding a new s6 service, one should carefully consider whether it
should be placed in a control group. Most services should be placed in
a control group, with only a few exceptions:
1. Services, such as `getty`, that spawn background processes.
2. Loggers.
3. Trivial services that don't do anything.
Generally, it's best to set the control group up as the first thing
the service does. To do that, use `cgroup-setup --leaf -- $1 COMMAND_LINE`,
where `$1` should be the service name and `COMMAND_LINE` is the program
to run in a cgroup.
If you use execline for your run script, this is as simple as:
.run
....
#!/bin/execlineb -WS1
cgroup-setup --leaf -- $1
# rest of script comes here
....
If the services exits, it's usually best to terminate any programs left
behind with SIGKILL and remove the control group. To do that, make
the `finish` script be a symbolic link to `/usr/bin/cgroup-setup`.
`cgroup-setup` recognizes when it has been invoked as `finish` and
purges the cgroup automatically.
When invoked as `finish`, `cgroup-setup` checks if the service exited
due to a signal that caused it to dump core. If it did, `cgroup-setup`
exits with status 125, ensuring that `s6-supervise` will *not* restart it.
This is intentional: if a service crashes due to a fatal signal, this
is possibly a sign of memory corruption. Restarting the service in this
case can turn an unreliable memory corruption exploit into a reliable one.
Rust panics do not cause core dumps, so the service will be restarted
afterwards.
== Future plans
Control groups are designed around a single writer process controlling each
of them. Many Linux distros use systemd for this, but Spectrum doesn't use
systemd. The only persistent per-service process is s6-supervise, but that
doesn't have control group support.
Instead, the plan is to have a database containing this information.
Whether this will be in the `data/` subdirectory of the service directory
or a separate system-wide database has not yet been determined.
|