From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id EFFA89F04; Thu, 25 Jun 2026 09:55:35 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 74DB69EBE; Thu, 25 Jun 2026 09:55:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_MISSING,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=4.0.1 Received: from fhigh-b7-smtp.messagingengine.com (fhigh-b7-smtp.messagingengine.com [202.12.124.158]) by atuin.qyliss.net (Postfix) with ESMTPS id 9649A9E73 for ; Thu, 25 Jun 2026 09:55:32 +0000 (UTC) Received: from phl-compute-05.internal (phl-compute-05.internal [10.202.2.45]) by mailfhigh.stl.internal (Postfix) with ESMTP id 4490A7A0102; Thu, 25 Jun 2026 05:55:31 -0400 (EDT) Received: from phl-frontend-03 ([10.202.2.162]) by phl-compute-05.internal (MEProxy); Thu, 25 Jun 2026 05:55:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alyssa.is; h=cc :cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1782381331; x=1782467731; bh=mS9neymORT 3/LYOTjW556FEZvOb6odxXHu9tMbamBk0=; b=wlR9jQ9WYPGUKfu7fS5fJUIzim tQpFirQBVCm4eUhdVvAMs8wWek8mcQc37TUCdIhNkS2xy2QZXhuFiSCjC9l85uHh 46PmCVJ2LPqwNf9FC15kYT+Wpnpb6C8UZoAgIXt8LepT35pO5KNSO2mXMJv+0BNR H1Flj0v4Z3ZBEdZ6XkAAnLsSp+feye0NuHwM9Z3x6SmYoV2tiGd80hQkoQc2MZv4 tf67lx8AscALzCIIzbKYmRS607UiVKpYyR4HHAXRW1DE1D/LxsVdaiXh5/QIlNNF Y/0QIWteQr/6Y1LjWurkZ3xYiBCpaRMjfO+PStflvMhjSKwPEcqM25UGDRQg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1782381331; x=1782467731; bh=mS9neymORT3/LYOTjW556FEZvOb6odxXHu9 tMbamBk0=; b=X8aTWzcBqK8EBQ91NJ6HPOysL0Zg67S53nNL1+G4Qd2uWKxCCIW QJ4nOc/cVrwIQntKG4Z5Yc69V2Is3yBKmGD9gLOKMJPzDfE9hGtvUJZJmpdPy3T2 vU71bs7CRJF6W9asWYn8p1eVzWhlwtQV4N/NLUl9ee8j2l04pr9xIYy2YRjhx0hq ndPs9wYPTrhgQnvFhAml3Z1lYnJjXVQPDnHWKGETkm/72oWz8QXgpe4njggR5wRp +/E82TW5kyQP8AtcexhoxZgk2DhQid+ndwaRbhmdhVBtekrJfNMJ+AV5oYi4w0io nhYZAJUtssTqOKf8AmGvBM7o2Y5WM76Li9g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTEGBfRaxtZiODNyEJLEUS+TOPahMdot/UA86rhgE0unirSQtshupmmVJGZoXgad7Z Quqlwaku6/3f3GYvidRwetFgzvnqedTYle4E0xLF+Cx8lHDBwOLjsRQkcZmXcs3pDC2j6C grA6TErxGD6CsuXp/io20F51QOn7S1V2RysepZ1LLUxhaXFobU1ip2vULfwBgNlj0O4Fwd KhEdhhg/TndWXVLcmlbUY1qZ7Ok4RwCeoSiSLsmBvspXHqPy600xxXSwQk7WepZlwP84C6 7q0HyAaCwFgzv7p0XG+kAyKiUZArZS2xRF3ClT94+VvP9sWNiTTyiSUxI8k3DL6Skq5uEV tK66n4hrlsCZLAeGoyUoSdiLJv0CuU8tK3zTyB237RANgt4LJhOO070rXpckGMnX5w+oHn TLU0Mq5WH81n+2gsz6gD8xiRkZHRCOR4VB6NdCYrh/k3HxpGhE+z7bPuie5Kx+YzUmrL4y ETqZJzlp7LG7jlN6ZEEtDqewJVnntdcIfcQVCRfhOUNOMGgAmORN0/ulUChiBg+KCVhdrX PfbgepANvgzEXtGjY3ZxI+kDkaFJdovRfCOPV/lNYGo4LtvP/SrEFYYyvHg8q6LI9Dy+d9 8jcc/TKrrv+jmEbXcgBk2Iurg76O0F4MfHUwL4JW2W2iYt0xRfesRfSKqNfg X-ME-Proxy: Feedback-ID: i12284293:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 25 Jun 2026 05:55:30 -0400 (EDT) Received: by mbp.qyliss.net (Postfix, from userid 1000) id 50F878369220; Thu, 25 Jun 2026 11:55:29 +0200 (CEST) From: Alyssa Ross To: Demi Marie Obenour Subject: Re: [PATCH v2] Set up control groups for most services In-Reply-To: <17ab104b-e6a5-4b33-b941-25ba35bf9b0a@gmail.com> References: <20260620-cgroups-v1-1-0e5abf35101b@gmail.com> <20260620-cgroups-v2-1-ccae224b6c85@gmail.com> <87qzlw5e65.fsf@alyssa.is> <17ab104b-e6a5-4b33-b941-25ba35bf9b0a@gmail.com> Date: Thu, 25 Jun 2026 11:55:28 +0200 Message-ID: <87ik7754hb.fsf@alyssa.is> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Message-ID-Hash: KSGYIEBDTRH62NUKTHXOJYS2Y66X6KEM X-Message-ID-Hash: KSGYIEBDTRH62NUKTHXOJYS2Y66X6KEM X-MailFrom: hi@alyssa.is X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Spectrum OS Development X-Mailman-Version: 3.3.10 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Demi Marie Obenour writes: > On 6/24/26 22:03, Demi Marie Obenour wrote: >> On 6/24/26 08:13, Alyssa Ross wrote: >>> Demi Marie Obenour writes: >>> >>>> The cgroups are handled by a Rust tool. The name of the cgroup is >>>> autogenerated from the service name. >>>> >>>> Using cgroups for process control is not implemented yet. >>>> >>> >>> Perhaps you could include some more details of how the cgroup hierarchy >>> is supposed to be set up and why, either in the patch body or in >>> documentation? >>=20 >> Will fix in v3. >>=20 >>> As I understood it, the point of using cgroups was that we could bundle >>> all services for a VM, including the VMM, into a single cgroup, but I >>> don't see that here, just a pure translation of the service hierarchy >>> (which the VMM might not even be part of). Are per-VM cgroups coming >>> later? >>=20 >> I think it would be significantly simpler to have the VMM be >> just another VM service. This would naturally put it under the >> vm-services cgroup, solving this problem. Since this would mostly >> involve changes to execline scripting, I think it would be better if >> you wrote this code. >>=20 >>>> diff --git a/host/rootfs/image/etc/fstab b/host/rootfs/image/etc/fstab >>>> index 18bb5e45abafeaf43871306ce8233239e5c07f76..d92364d83f4d26f766fcd5= b494614c06a630d2fe 100644 >>>> --- a/host/rootfs/image/etc/fstab >>>> +++ b/host/rootfs/image/etc/fstab >>>> @@ -6,3 +6,4 @@ tmpfs /dev/shm tmpfs nosuid,nodev 0 0 >>>> tmpfs /media tmpfs nosuid,nodev,noexec,nosymfollow,mode=3D755 0 0 >>>> sysfs /sys sysfs nosuid,nodev,noexec 0 0 >>>> tmpfs /tmp tmpfs nosuid,nodev 0 0 >>>> +cgroup2 /sys/fs/cgroup cgroup2 nosuid,nodev,noexec,nosymfollow 0 0 >>> >>> Alignment is off here. The rest of the file uses tabs at 8 characters. >>> (I know, I know, but it's traditional.) >>> >>>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/ser= ial-getty-generator/run b/host/rootfs/image/etc/s6-linux-init/run-image/ser= vice/serial-getty-generator/run >>>> index bf66ab0878fc6d8e77ca71a4a89e50c139a6bb11..5c9988c994ed9b94232905= 5673b332d0aa918957 100755 >>>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-get= ty-generator/run >>>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-get= ty-generator/run >>>> @@ -1,7 +1,8 @@ >>>> -#!/bin/execlineb -WP >>>> +#!/bin/execlineb -WS1 >>>> # SPDX-License-Identifier: EUPL-1.2+ >>>> # SPDX-FileCopyrightText: 2024-2025 Alyssa Ross >>>>=20=20 >>>> +cgroup-setup -- $1 >>>> piperw 3 4 >>>> background { >>>> fdclose 3 >>> >>> What benefit does this cgroup provide? >>=20 >> None, I'll remove it. >>=20 >> The only requirement of the current code is that if a service is under >> a cgroup, its parent should *also* be under a cgroup. Otherwise you >> can get conflicting cgroup paths. >>=20 >>>> diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/ser= ial-getty/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/seria= l-getty/run >>>> index 78f794202bf174f3c036f3e20755ac087a988277..ed0808b9e45b077023f237= a0f9c0e5c830015ac2 100755 >>>> --- a/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-get= ty/run >>>> +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/serial-get= ty/run >>>> @@ -1,5 +1,6 @@ >>>> -#!/bin/execlineb -WP >>>> +#!/bin/execlineb -WS1 >>>> # SPDX-License-Identifier: EUPL-1.2+ >>>> # SPDX-FileCopyrightText: 2023 Alyssa Ross >>>>=20=20 >>>> +cgroup-setup -- $1 >>>> s6-svscan -d3 instance >>> >>> We're putting supervisors of instances services in a cgroup? Can you >>> explain to me why that's useful? >>=20 >> This allows configuring limits that apply to the whole instance, and >> means that the whole instance can be reliably killed. Services spawned >> by the instance will be in their own sub-cgroups, so one can apply >> separate limits to them so long as those limits do not exceed those >> of the containing cgroup. >>=20 >> The hierarchy works like this: >>=20 >> /sys/fs/cgroup >> service1.service >> @inner.service # processes go here >> service2.service >> @inner.service # and here >> service3.service >> @inner.service # and here >>=20 >> You can apply resource limits at any level of the hierarchy. Limits in >> nested cgroups can be smaller but not larger. Terminating processes >> in a cgroup with cgroup.kill also terminates all processes in child >> cgroups. After terminating the processes in a cgroup, cgroup-setup >> will delete everything in the hierarchy recursively. > > More generally: (I already replied to the previous email before seeing this, but I'll try to respond in depth here too.) > The design I implemented is based on a model where cgroups and services > are very tightly coupled: most services have an associated cgroup and > every cgroup is attached to a service. This is like systemd service > units, except that it can be nested many layers deep. > > Since a VM may have many processes that serve it, this model requires > that all of these processes be under a single service. In this case, > that service would be the vm-services service. > > That said, I'm not sure which of these service, if any, should be > automatically restarted if they crash. Automatic restart of services > is actually very useful for attackers, as it lets them retry an exploit > over and over until they get the memory layout they need for success. This is exactly why this model doesn't fit. It does not make sense to use an s6 service for something that doesn't get restarted. Oneshot services are not a concept that exist in s6. For a clean, repeatable environment etc. one can use s6-sudod, but even if we had an s6-sudo service for starting VMMs, they still wouldn't be under a vm-services instance. The service=E2=86=94cgroup equivalence from systemd does not make sense here. > If you restart the VMM, you might as well restart every other > service as well. Unless Cloud Hypervisor automatically reconnects to > vhost-user devices (and I don't think it does), there's no point in > restarting virtiofsd or crosvm without either re-adding the device > or restarting the VMM. Even if you did re-add the device, I don't > know if the guest would be able to recover. Guests are able to recover from e.g. dbus crashing trivially, and I've seen virtio-fs working fine after a virtiofsd restart too. That's why we run them as services. It's set up the way it is for a reason! > Without automatic restart, there's no real reason to use s6 to manage > the per-VM services. s6 assumes you do want to restart services > that crash. Which we do, generally. > An alternative approach is to have cgroups be freestanding objects > that exist independently of whether there are processes in them. > This corresponds to systemd scope units. It's not hard to implement > this: just disable automatic purging and pass the full path to the > cgroup. But it requires each VM service to have identical resource > limits, whereas one typically wants per-process limits. For instance, > none of the services should be able to create child processes, and > per-service cgroups make this easy: just set the maximum amount of > processes in the cgroup to 1. Why would you be restricted to identical resource limits. > Incidentally, this works very well with PID namespaces. One can have > the "main" process of the service run in a PID namespace, and have > all the others enter that namespace. Then if the main process dies, > so do all of the others. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQRV/neXydHjZma5XLJbRZGEIw/wogUCajz7EAAKCRBbRZGEIw/w opqTAQCDeO7Ml/T1f/jl/YlQFarmUHP+PukICH3H6QFhwinrlgD/WCABWNScqSXJ 8Z98cm1Q359wVg9SP93jPqDgtN51DA8= =cRcJ -----END PGP SIGNATURE----- --=-=-=--