From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id 4482065DA; Wed, 22 Jul 2026 02:16:55 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 0D8D26554; Wed, 22 Jul 2026 02:16:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_PASS,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=4.0.1 Received: from mail-yw1-x1135.google.com (mail-yw1-x1135.google.com [IPv6:2607:f8b0:4864:20::1135]) by atuin.qyliss.net (Postfix) with ESMTPS id 8314F653F for ; Wed, 22 Jul 2026 02:16:51 +0000 (UTC) Received: by mail-yw1-x1135.google.com with SMTP id 00721157ae682-8111c0c7561so140743567b3.3 for ; Tue, 21 Jul 2026 19:16:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1784686610; x=1785291410; darn=spectrum-os.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=HAuFK64Mffb3T+yScpPM7m0t3G9x2zGSXMVPho9JczA=; b=EJ0I8cU6SLX8hYfWF1jRB2klcLnTdHJEqEG8FvXzpPhTAv1gEO4bWu7YNpujWgslwb 8dLq8+rv5v5L+IQSH9DxN1vgaCtDwg27g9ikrnv8IXJ4MfVRTENM4EARE2CLUBTSarrb 5MrcgdQJz7UVM901i5lOUHUqqF7PgrHktZ7Xvkghp3tkhjWsltgWk/+K60CuUis5FPsw 2TKoYCeStyfV2SjOgmdo/kQ4afuY/jBDFJQVT/e/eLJm5VQWVx+bn3fFRFmbA9Nr+qER V9b+oLbrtVhbPEvM4VX96+7qiZWwrLlYdt8ireSrekSC/q/shGyRFRQZ1U8wAbAUr5S0 GvDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784686610; x=1785291410; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=HAuFK64Mffb3T+yScpPM7m0t3G9x2zGSXMVPho9JczA=; b=LvqFjEZykGPDoNM7fyRy0F6pk5DW0fFPvf7q90YkjLOkEiE7o5eYX2aN1MgdbgquXc P6u3Ysk+Z+d32XXw+/4IV2AtlK4hNmqnOpKMFyj+OiZHvV57nuG/Yxa/dLk+yUHqlfEK zujW9BTllol5F+d8AoSYjcoyC1X4r3XsP+mVA3ij/K6CeGFXiH48RijsAUxWUwrXgQfZ VI2Sdiq1k/aLXDX+T6gftXvYFrdLsg4xLKofuTFcuhkO4N9zfFIFAD39oyc3DEMkvM24 glcGspAPfIFBl3QEaYKnKKwRTUbQe339g80HnLiP6XfBdZKIM8TL5S+GcldsMclewZpH CC+w== X-Gm-Message-State: AOJu0Yxif68bDWKMlnOD967MnKfC2jgn9caaHc1H2/f9hYx2Cc6YHGxu uChHHK60A+Cs95PSnNNlXQdBOupZb7k0rzgCQrKmuB3XxlE3EAdKW687I5zhfg== X-Gm-Gg: AR+sD105tB5TbLVLX0LrMw24PLWiKwpJFTHSCRE2Cym0i07s14rb1fiir0TKa5RJlMW kzf3CdEbBwW/Vy4HzpqnRbU7SR4XjznB3qBRixDuDqzN1QpgRXqPCbDYO2ONL3sN78nf65ASp4h bPkjqim5q7l6spDLFIpn94fgV3iEKkQhI/bPQxJin7Nl+SHnrOxnnpk4pOK4DZRz/F4a5IwSjNJ V+WIBe5dw1uPfcRNfNmCvujdFreu4Futim1qSqXaG+LyQQmVgZbY8XcHRjVd/95IaF2aKEJ5Z0X 3qmQFW28TW3V3Go9BOp8rF0cRhyUVE47F0CUkhH9Ovu7D3QAdhGeKApSdRdqsqnvE5clcSJd9H3 9V1EkgZliRNBmi6Kv4pyjcMGEk871Fps5g44hpNr1U0SQKOc0+CcHsrwDgcpDs34hx/az05GVdz Bgius4ouUU+1EmyGo= X-Received: by 2002:a05:690c:c4d2:b0:7ff:1399:9317 with SMTP id 00721157ae682-81ef26dd7e2mr70184457b3.11.1784686609790; Tue, 21 Jul 2026 19:16:49 -0700 (PDT) Received: from localhost.localdomain ([185.98.168.14]) by smtp.gmail.com with UTF8SMTPSA id 00721157ae682-81f33c39907sm6926167b3.16.2026.07.21.19.16.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jul 2026 19:16:48 -0700 (PDT) From: Demi Marie Obenour Date: Tue, 21 Jul 2026 21:59:08 -0400 Subject: [PATCH v4 03/20] Documentation: Mention control groups MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260721-cgroups-v4-3-46b2e5fff7b6@gmail.com> References: <20260721-cgroups-v4-0-46b2e5fff7b6@gmail.com> In-Reply-To: <20260721-cgroups-v4-0-46b2e5fff7b6@gmail.com> To: Spectrum OS Development X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1784685545; l=4468; i=demiobenour@gmail.com; s=20250729; h=from:subject:message-id; bh=trv81EQyXdrKZIXHMeeMHhFkd6a5DCAKNIuH0ayLkhE=; b=zbyt7o5XWmF1RzYGA2+ugdo9WAEqwhyX5ZdvXbIbswSoUwyXJkt/zlW6Erd+KpRqtGWVxGmCi hjrg8dLs/jcDXCejQ7pc+G+3HDKDcgbtAzDsPQ5kKvb6HxqNN3uxQtz X-Developer-Key: i=demiobenour@gmail.com; a=ed25519; pk=X57Q4/YQDj9t4SBeKaDwvXYKB6quZJVx/DE2Ly2out0= Message-ID-Hash: F6IXHYZPJAQRW5WMSBSEH4VB4D6TVKQ6 X-Message-ID-Hash: F6IXHYZPJAQRW5WMSBSEH4VB4D6TVKQ6 X-MailFrom: demiobenour@gmail.com X-Mailman-Rule-Hits: member-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; emergency CC: Demi Marie Obenour , Alyssa Ross X-Mailman-Version: 3.3.10 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Signed-off-by: Demi Marie Obenour --- Documentation/doc/development/control-groups.adoc | 88 +++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/Documentation/doc/development/control-groups.adoc b/Documentation/doc/development/control-groups.adoc new file mode 100644 index 0000000000000000000000000000000000000000..6ce33f21a230d012a690fc5deb2ba597d1d01ef1 --- /dev/null +++ b/Documentation/doc/development/control-groups.adoc @@ -0,0 +1,88 @@ += Control groups in Spectrum + +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Linux control groups (cgroups) can be used for several purposes: + +1. They allow waiting for a group of processes to exit. +2. They allow terminating a group of processes. +3. They allow limiting a group of processes' access to resources. + +Spectrum currently uses the first two. The third is not yet used, +but will be in the future. + +== Control Group Hierarchy + +Spectrum uses the following cgroup hierarchy: + +1. There is a `/vm-services.slice` cgroup that contains all the per-VM + services on the system. +2. The per-VM services for each VM are under `/vm-services.slice/vm-${VM}.slice`, + where `${VM}` is replaced by the VM's ID. +3. Each per-VM service is under `/vm-services.slice/vm-${VM}.slice/${SERVICE_NAME}`, + where `${VM}` is replaced by the VM's ID and `${SERVICE_NAME}` is replaced by + the name of the service. +4. The VMM runs under `/vm-services.slice/vm-${VM}.slice/vmm`. + +If a cgroup contains child cgroups, it likely contains a `$inner.service` +cgroup. This is where programs that would otherwise run in the cgroup itself +are placed. Generally, these programs are instances of `s6-svscan` and/or +`s6-supervise`. + +== Using Control Groups + +When adding a new s6 service, one should carefully consider whether it +should be placed in a control group. Most services should be placed in +a control group, with only a few exceptions: + +1. Services, such as `getty`, that spawn background processes. +2. Loggers. +3. Trivial services that don't do anything. + +Generally, it's best to set the control group up as the first thing +the service does. To do that, use `cgroup-setup --leaf -- $1 COMMAND_LINE`, +where `$1` should be the service name and `COMMAND_LINE` is the program +to run in a cgroup. + +If you use execline for your run script, this is as simple as: + +.run +.... +#!/bin/execlineb -WS1 + +cgroup-setup --leaf -- $1 +# rest of script comes here +.... + +If the service exits, it's usually best to terminate any programs left +behind with SIGKILL and remove the control group. In Spectrum, this is +called "purging" the cgroup. To purge the cgroup when a service exits, +make the `finish` script invoke `/usr/bin/cgroup-s6-finish`. The first +two command line arguments must be the first two arguments passed to the +`finish` script. The third argument must be the path to the cgroup to +be purged relative to the cgroup the program itself is in. This is +usually, but not always, the third argument to the `finish` script. + +When invoked as `cgroup-s6-finish`, `cgroup-setup` checks if +the service exited due to a signal that caused it to dump core. If it +did, `cgroup-s6-finish` exits with status 125, ensuring that +`s6-supervise` will *not* restart it. This is intentional: if a service +crashes due to a fatal signal, this is possibly a sign of memory +corruption. Restarting the service in this case can turn an unreliable +memory corruption exploit into a reliable one. Rust panics do not cause +core dumps, so the service will be restarted afterwards. + +One can also use `cgroup-purge` to purge a cgroup explicitly. This is +used to stop the VMM and all per-VM services when a VM is shut down. + +== Future plans + +Control groups are designed around a single writer process controlling each +of them. Many Linux distros use systemd for this, but Spectrum doesn't use +systemd. The only persistent per-service process is s6-supervise, but that +doesn't have control group support. + +Instead, the plan is to have a database containing this information. +Whether this will be in the `data/` subdirectory of the service directory +or a separate system-wide database has not yet been determined. -- 2.55.0