From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id AA70B538E; Fri, 21 Aug 2026 06:53:43 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 7779E52E4; Fri, 21 Aug 2026 06:53:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_PASS,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=4.0.1 Received: from mail-yw1-x112f.google.com (mail-yw1-x112f.google.com [IPv6:2607:f8b0:4864:20::112f]) by atuin.qyliss.net (Postfix) with ESMTPS id 752A652D2 for ; Fri, 21 Aug 2026 06:53:39 +0000 (UTC) Received: by mail-yw1-x112f.google.com with SMTP id 00721157ae682-80bb41f7f3cso6962637b3.2 for ; Thu, 20 Aug 2026 23:53:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787295218; x=1787900018; darn=spectrum-os.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=puj2jo3Ms5QK8K9z16sGtlP7ETK62GowVflQIdqahSQ=; b=OeGGIQR8+LJuyScpu98EIdZwJbbxbBQDvJsqkCNQZrf7HGoT5aJZufrAqgOzZYHWzj UoPgqeg4yP3m1uSvfS6LMXRY+QuJ02E2DIVOsJ2RiyKzJSOst5afPZO5Nc1ifuolYSnk EDXWyzC4kCpCSEki38JSOCiYyOUgjaDJqTQTscCZKR8M+RgGvCiM5id+9RdxwORVnypm T9tlkU0tk7+X4L/mN5qBKZCmq0ykEkDdMCaU61je/59xasKet+/rrGkR4mIn+16TcP4M JexCgwaHR6S5N6ViDoC1HTKNbjRUYF9nKWz4ZlclY/ooC5A8dNu3NB05DVWZtX8fi07+ NqNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787295218; x=1787900018; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :content-type:mime-version:subject:date:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=puj2jo3Ms5QK8K9z16sGtlP7ETK62GowVflQIdqahSQ=; b=PvQcoIIBdSXE972QhYrFGgg9JbeBbBB2PxQy36IdXIm8Xfjc3wHkZaoG5NL04TsAzn GGjJIGRKtIhAAtKyOaXpVoKiSir43zVI7dOQL0hHwhm/38zEkHDX6pMGu9eOiSXOa17e NvqfTsu4m07k9AH+ApBK8NwpJsqxUnhIaxb6t/FA20mqptPMZkyukMU+i5RcFXyWd79o SjqMgAH7+7p1dNu+hyXArPe8vlHYSKu2l2fdxjOrw2EDuHbcOWYhvvkEGAKZhWkeJg27 aCkxXlzmIbr3m50LfpBJ8VWYGE937FkjO9MrxpoL6OUDR4fxJIKtlNhdg1EhPhkQsEVF t9xg== X-Gm-Message-State: AFuF++lvQl9rszSaWmNHzjOU8l6SdGKr9rm6buAmPk+x9xzBVtfSjgOT hPTOh4fe+dGI1BIsLIfmUSPKYIAMlSZl5jkOGDybzPR3gZUHlCQz+3rJ1kc+BEW/ X-Gm-Gg: AR+sD12qRcLkk4pYsHX9aEW2wXNvGAn38NkBCurnjUTtv6JmeSHaxSsJ9RAO/KyDmZP vNeXyPnwvaCgycnw4E4kbYtgPQtlpIyZItjP3xDqgu/6t5PRecYKMv0UBPP7Vl3jrJ2DXUXdS1z FSBlaZ6oKOw74mMcVa3+gVE4upYwLJ/sKQp1kBvxFPvvmvshgxF9T/Ao4LAelh4xpdbgFtYjM0J ttZtCUzIzWoAxlxpKA+qICpxEy5zE2BP/RBbnM9HsR4tU+ogFiCWQt7iu/YeiRKCGD6Xio8XjT+ qPQuDSTHCAEg/U0wlMZkM8c8oTy9k/d5p9ac/HHWyrsWrndatRVrIuTr58sCMRZXJ11sCXmtNy6 rFXBZl5gJkYJXO8WSstseVovQAjJeTAo0LzsNWUc+GweYs3jWyMD1zecER8NlwzzydDYhqRC+vS oeJqLZDJNM3OVywB6caQCEjLVPLxNNmU07TPJ5ZgPw1SlZFWIfbe+4fP3f/W2CqqlOkB2hVBdFQ CwClhoEauS9 X-Received: by 2002:a05:690c:4e03:b0:7db:a109:f12f with SMTP id 00721157ae682-849eff4a8c3mr13651977b3.1.1787295217854; Thu, 20 Aug 2026 23:53:37 -0700 (PDT) Received: from localhost.localdomain ([185.98.168.14]) by smtp.gmail.com with UTF8SMTPSA id 00721157ae682-8451772cf65sm35124767b3.32.2026.08.20.23.53.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 20 Aug 2026 23:53:36 -0700 (PDT) From: Demi Marie Obenour Date: Fri, 21 Aug 2026 02:51:46 -0400 Subject: [PATCH v7 03/19] Documentation: Mention control groups MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260821-cgroups-v7-3-7f1870dedefc@gmail.com> References: <20260821-cgroups-v7-0-7f1870dedefc@gmail.com> In-Reply-To: <20260821-cgroups-v7-0-7f1870dedefc@gmail.com> To: Spectrum OS Development X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1787295104; l=4816; i=demiobenour@gmail.com; s=20250729; h=from:subject:message-id; bh=hbUhH3t0XBoLIy/VRWbdyE3vbLU1nzfvPipq/qPbDd8=; b=eQM+6317HgzNWr/Xpatl7GZvY6XmGYvZp46O28frcmfaeKQOUnWIPZPylR1Ww34z8qco3bLgG 3TadG6mvMNgCyL01FtL0Bed72ueWRMiSR6JLXnwHlIP2dNaQqdpcbqf X-Developer-Key: i=demiobenour@gmail.com; a=ed25519; pk=X57Q4/YQDj9t4SBeKaDwvXYKB6quZJVx/DE2Ly2out0= Message-ID-Hash: 3OSCJNZQ6NQEUSJHIYSPND4JXRSX52SA X-Message-ID-Hash: 3OSCJNZQ6NQEUSJHIYSPND4JXRSX52SA X-MailFrom: demiobenour@gmail.com X-Mailman-Rule-Hits: member-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; emergency CC: Demi Marie Obenour , Alyssa Ross , Valentin Gagarin X-Mailman-Version: 3.3.10 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Signed-off-by: Demi Marie Obenour --- Cc: Valentin Gagarin --- Documentation/doc/development/control-groups.adoc | 108 ++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/Documentation/doc/development/control-groups.adoc b/Documentation/doc/development/control-groups.adoc new file mode 100644 index 0000000000000000000000000000000000000000..93b8c823e1d1c5a3692faa51ad41d3291d3fb743 --- /dev/null +++ b/Documentation/doc/development/control-groups.adoc @@ -0,0 +1,108 @@ += Control groups in Spectrum + +// SPDX-FileCopyrightText: 2026 Demi Marie Obenour +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html[Linux control groups] (cgroups) can be used for several purposes: + +1. Waiting for a group of processes to exit. +2. Terminating a group of processes. +3. Limiting a group of processes' access to resources. + +Spectrum currently uses the first two. The third is not yet used, +but will be in the future. + +== Control Group Hierarchy + +Spectrum uses the following cgroup hierarchy: + +1. There is a `+/vm-services.slice+` cgroup that contains all the per-VM + services on the system. +2. The per-VM services for each VM are under `+/vm-services.slice/vm-${VM}.slice+`, + where `+${VM}+` is replaced by the VM's ID. +3. Each per-VM service is under `+/vm-services.slice/vm-${VM}.slice/${SERVICE_NAME}+`, + where `+${VM}+` is replaced by the VM's ID and `+${SERVICE_NAME}+` is replaced by + the name of the service. +4. The virtual machine monitor (VMM) runs under `+/vm-services.slice/vm-${VM}.slice/vmm+`. + +If a cgroup contains child cgroups, it likely contains a `$inner.service` +cgroup. This is where programs that would otherwise run in the cgroup itself +are placed. Generally, these programs are instances of +https://skarnet.org/software/s6/s6-svscan.html[`s6-svscan`] or +https://skarnet.org/software/s6/s6-supervise.html[`s6-supervise`]. + +== Using Control Groups + +Most services should be placed in a control group, with only a few +exceptions: + +1. Services, such as `getty`, that spawn background processes. +2. Loggers. +3. Trivial services where cgroups are unnecessary. + +=== Setting Up Control Groups + +Generally, it's best to set the control group up as the first thing +the service does using Spectrum's https://spectrum-os.org/git/spectrum/tree/tools/cgroup-setup[`cgroup-setup`] tool. + +The most common case is: + +1. The service runs in a cgroup that is a direct child of the parent + cgroup. + +2. If there is a program running in the cgroup, the cgroup manager + should wait for it to exit before spawning a new instance. + +In this case, use `+cgroup-setup -- $1 COMMAND_LINE+`, where `+$1+` +should be the service name and `+COMMAND_LINE+` is the program +to run in a cgroup. + +If you need to put the program in a different cgroup, you can use +a different name. If the control group starts with `+/+`, it is +interpreted as a path relative to `+/sys/fs/cgroup+`. Otherwise, +it is relative to the current control group the program is in. + +If you don't want to wait for existing programs in the cgroup to +exit, use `+--no-wait+`. This is rare. + +For example, using https://skarnet.org/software/execline/[execline] for your run script: + +[source,execline] +.... +#!/bin/execlineb -WS1 + +# Often, your cgroup is just the parent cgroup +# with the name of the service ($1) appended. +cgroup-setup -- $1 +# The rest of the script goes here. +.... + +=== Purging Control Groups + +If the service exits, it's usually best to terminate any programs left +behind with `SIGKILL` and remove the control group. To do this, +make the `finish` script invoke `cgroup-purge`. Its sole command-line +argument is the cgroup to remove. + +[source,execline] +.... +#!/bin/execlineb -WS3 + +# Use the same path you used in the run script. +cgroup-purge $3 +# The rest of the script goes here. +.... + +One can also use `cgroup-purge` to purge a cgroup explicitly. This is +used to stop the VMM and all per-VM services when a VM is shut down. + +== Future plans + +Control groups are designed around a single writer process controlling each +of them. Many Linux distros use systemd for this, but Spectrum doesn't use +systemd. The only persistent per-service process is s6-supervise, but that +doesn't have control group support. + +Instead, the plan is to have a database containing this information. +Whether this will be in the `data/` subdirectory of the service directory +or a separate system-wide database has not yet been determined. -- 2.55.0