From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id B8BF31FE15; Fri, 26 Sep 2025 16:33:36 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 484291FD79; Fri, 26 Sep 2025 16:33:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_PASS,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=4.0.1 Received: from mail-yw1-x1133.google.com (mail-yw1-x1133.google.com [IPv6:2607:f8b0:4864:20::1133]) by atuin.qyliss.net (Postfix) with ESMTPS id 5F8C81FDC1 for ; Fri, 26 Sep 2025 16:33:32 +0000 (UTC) Received: by mail-yw1-x1133.google.com with SMTP id 00721157ae682-71d6051aeafso23682247b3.2 for ; Fri, 26 Sep 2025 09:33:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1758904411; x=1759509211; darn=spectrum-os.org; h=cc:to:references:in-reply-to:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=kC2tezDttsUd6tUlAMh0Q0exLrTntJsLLJchNEyOqWY=; b=DhFdoIA7xcxdrbAS28uacdGNIGBDMZjl2PhBvrd3/5QjY5+sbc+tebGs/pKP8W3I60 R4cUGuZQHR5Xev08hh1aiVMoO4+JwQIsffX2o1UD0wBO9EAZubl03QMtE7CiREfA3vsb dIK/b7VnRn01TTO8satZWaXy0HQuRMz/lpW51JyAe5izS+9Y1IdCjHgscHbiCuW/7wC0 oGLhUIcFZInyknqzpm5PMSyOnzPW6ejs0wz8zlkuUFDJZuQFJCAN7kFE3oW6OZfLr3Ay kVJ8O/iXnMHEjnViX+ofXFFA5oDbjpol5nRLuThc9gFCmEXzqvrJacLuk1zxLIfpNoWq FgNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758904411; x=1759509211; h=cc:to:references:in-reply-to:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kC2tezDttsUd6tUlAMh0Q0exLrTntJsLLJchNEyOqWY=; b=pktKqEih9PH7YM0Bp3bX0tTjhbR7luxFbhu0Wli2ummX3DzpItB+LSee2e0d8B1mw7 yxG86kF2cC1poAstsogIDM46ynOpBI94u3mxMwDIin5NtE+gORnM9McSoCIToh/O0fhB hC3BgBXrY9PP0pt4zmxLgdwCTYuS5dYiGW31HBB6Pxd/ufbauumuH0GtKKC4FBawElgP deXQ3X1KkkeaQlaabHHsNfqsIoyn6+heBQsNmfSRx4K7+pa+a49gI1xcWzHzlfAzI1yU D3oTQdiw1JNL2roUglzMN9ARaGW35eMSHH4+7q5P3U42bn3HGURuzd239Uu1QcvukNIu ZtVw== X-Gm-Message-State: AOJu0YxYIUzEOQPL9lXZn6ua/ASxMM4nXDPzGEZXt7JaAKpRIsdkbFkp LziJAyLwQHyupOxBKwCAr9URFk3IG1Uk30IQPk2u/mhcUQeXLOa09FpuVKM7MP0Z X-Gm-Gg: ASbGnct6tLgO1rCA+zQQC4XUZWgvVDNwCza1nopfglgn9MfdDIAnhZQGTLlq1iRvDO5 j455rqBDxxiB/ag51EHOF7Q69mq4uiWQlmo34cwklMjeIs2yHnNUr/LicCWyTyr4ZKXSnbvHIf8 w3A+xvTuG2yMjkEsufHlQksMN1PBOAMWudoqtz3wgWYk4ABemfS1Pxv/iKBFV7STPBx+9kkLFq8 +jtdHJ2E/wVkjMwoLKmKYULnSU5wXT59/gTTKvTU4z7JRvF1mlrnO6Nc9z7jphG4GqxD2KkuIL6 E3tNuu/H50ebmf8TMC6QCDmbjmcDRvJ3ndIY4v9yAcP17hFgrrfFQ859r6bMm8yLl1OnqPXmiK/ 5s8t6HKsAgVDDFa9NlQEe0KhnBkFj9bvIAGPqBZfKh6QCE1avy3IsEPe646h0RpXm36lGbfVq8Q PRmtUkEp51LWh2nHe8F7g8a7zM8A+ppJYV5we08zG3f6FtPPG6qZSozQ== X-Google-Smtp-Source: AGHT+IFYgm4KImuAuF1kVMbbHylHHfA6WSwHzioOwMJlV7cJWwX3yxPfsp/kEoDZVTI4URy5YZr4aw== X-Received: by 2002:a05:690c:4802:b0:721:5b31:54c0 with SMTP id 00721157ae682-764034232e2mr87777587b3.39.1758904409555; Fri, 26 Sep 2025 09:33:29 -0700 (PDT) Received: from localhost.localdomain (h96-60-249-169.cncrtn.broadband.dynamic.tds.net. [96.60.249.169]) by smtp.gmail.com with UTF8SMTPSA id 956f58d0204a3-636d5b1d8f2sm619389d50.23.2025.09.26.09.33.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Sep 2025 09:33:28 -0700 (PDT) From: Demi Marie Obenour Date: Fri, 26 Sep 2025 12:31:17 -0400 Subject: [PATCH v5] Generate file lists from a script MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20250926-genfiles-v5-1-3394dfb3b330@gmail.com> X-B4-Tracking: v=1; b=H4sIANS/1mgC/2XPTQ6CMBCG4auQrq3pHy248h7GBR0GaIJgWtNoC He34MJGl1/T501mIQG9w0BOxUI8RhfcPKVRHgoCQzP1SF2bNhFMlKySnPY4dW7EQJU2lemAs0r XJH2/e+zcc09drmkPLjxm/9rLkW+vn0jN5DcSOWUUoK5lB5ajgnN/a9x4hPlGtkgUGeQsgyJBa dAiM42E9g/KDIocSsppq0FYbbSxSvxClcPs3qgSVNKUtm3S5czkcF3XN0+EPOdLAQAA X-Change-ID: 20250831-genfiles-46787fc10869 In-Reply-To: <20250921-genfiles-v4-1-4375bda78707@gmail.com> References: <20250921-genfiles-v4-1-4375bda78707@gmail.com> To: Spectrum OS Development X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1758904276; l=31285; i=demiobenour@gmail.com; s=20250729; h=from:subject:message-id; bh=1B6DA7vZ9VypeTYcZRrTZJ8vqyNcNv5vNcDzBPJw7zU=; b=bnNiT3w3ro4CJ4dEQz+5HtXsI8J1ai6rAYPBH1dN126/GLZvZauZAPxW7PeZBGZZ5G7wGd/kt E4tJU+/QeaWAwu7Ds/ZglDbgXLimRLIBEDIqdCWtW9/8MCSF3ite5X2 X-Developer-Key: i=demiobenour@gmail.com; a=ed25519; pk=X57Q4/YQDj9t4SBeKaDwvXYKB6quZJVx/DE2Ly2out0= Message-ID-Hash: A3X2WVL2F7VPENT6NBG6GP6Y5V3NRW2A X-Message-ID-Hash: A3X2WVL2F7VPENT6NBG6GP6Y5V3NRW2A X-MailFrom: demiobenour@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Alyssa Ross , Demi Marie Obenour X-Mailman-Version: 3.3.9 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Right now, the makefiles in host/rootfs, vm/sys/net, and img/app have manually-maintained lists of files and symlinks. These duplicate the information in the git repository and can easily get out of sync or cause unnecessary merge conflicts. Fix all of these issues by having the git repository be the source of truth, and using a script to generate the file lists. Developers can regenerate the lists before every commit, or even add a git hook to do that. Signed-off-by: Demi Marie Obenour --- This actually reduces the amount of code that has to be written by hand. --- Changes in v5: - Use 'print ""' instead of 'print' in awk to print a newline. 'print' with no arguments implicitly prints $0 instead. This caused the generated makefiles to be incorrect. - Use S6_RC_FILES instead of VM_S6_RC_FILES in vm/sys/net/Makefile and img/app/Makefile. This prevented the image from being built. - Do not check for git repository being in a directory with a name ending in a newline. - Use shell redirection instead of awk redirection. - Do not include trailing DONE line in input to awk. - Link to v4: https://lore.kernel.org/r/20250921-genfiles-v4-1-4375bda78707@gmail.com Changes in v4: - Use /bin/sh instead of bash. - Do not assume that negated awk character classes match all bytes. - Do not check the mode of license files. - Use implicit awk variable initialization. - Use 'git rev-parse --show-toplevel' to find the repository root. - Remove wrongly added copyright header. - Improve documentation. - Remove git hooks. - Add missing copyright header. - Avoid non-portable /usr/bin/env -S. - Avoid assuming that awk is GNU awk. - Avoid non-portable awk -E. - Do not check for git bugs. - Fix link in v3 changelog. - Link to v3: https://spectrum-os.org/lists/archives/spectrum-devel/20250920-genfiles-v3-1-d6c2b6767b42@gmail.com Changes in v3: - Only include the file list generator. Move the rest to separate patch series. - Remove the update-file-list make targets from img/app/Makefile and vm/sys/net/Makefile. - Link to v2: https://spectrum-os.org/lists/archives/spectrum-devel/20250910-genfiles-v2-0-37ebe07a3cdc@gmail.com Changes in v2: - Drop the last patch (switching to /etc/s6-rc/compiled) as it is controversial and should be reviewed separately. - Add missing copyright notices. - Use a wrapper shell script to make the awk code easier to read. - Improve documentation. - Add helper scripts for use in git hooks and rebasing. - Link to v1: https://spectrum-os.org/lists/archives/spectrum-devel/20250903-genfiles-v1-0-cc993fcb1e4c@gmail.com/ --- Documentation/development/built-in-vms.adoc | 7 ++ host/rootfs/Makefile | 102 +--------------------------- host/rootfs/file-list.mk | 99 +++++++++++++++++++++++++++ img/app/Makefile | 82 ++++------------------ img/app/file-list.mk | 65 ++++++++++++++++++ scripts/genfiles.awk | 77 +++++++++++++++++++++ scripts/genfiles.sh | 24 +++++++ vm/sys/net/Makefile | 52 +++----------- vm/sys/net/file-list.mk | 42 ++++++++++++ 9 files changed, 337 insertions(+), 213 deletions(-) diff --git a/Documentation/development/built-in-vms.adoc b/Documentation/development/built-in-vms.adoc index e90009ee5a3c2c254a7ae11e36121576b819eee7..d044e7509a43df4b015f312dac61be6c74cc7feb 100644 --- a/Documentation/development/built-in-vms.adoc +++ b/Documentation/development/built-in-vms.adoc @@ -3,6 +3,7 @@ :page-nav_order: 3 // SPDX-FileCopyrightText: 2023-2025 Alyssa Ross +// SPDX-FileCopyrightText: 2025 Demi Marie Obenour // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 Spectrum comes with some built-in application VMs, to demonstrate @@ -44,6 +45,12 @@ NOTE: As a special convenience, it's not necessary to run `make clean` if the only change to the Nix files is modifying the packages installed in the VM. +The list of files used for images is stored in a separate file, +file-list.mk. To update it, run scripts/genfiles.sh, which will +regenerate it from the output of `git ls-files`. This script uses +Git's index to generate the list, so only staged changes will be +reflected in its output. + By default, `make run` runs the VM in Cloud Hypervisor, to match the Spectrum environment. You can also add `VMM=crosvm` or `VMM=qemu` to your command line to run the VM in a different VMM. One reason to do diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index 322d75026dbcba4301f26abfbf74efa8cc68cdfd..da868557328fa84082e7b4cd968a1c2f7dd99af6 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -4,67 +4,10 @@ .POSIX: include ../../lib/common.mk +include file-list.mk dest = build/rootfs.erofs -FILES = \ - image/etc/fonts/fonts.conf \ - image/etc/fstab \ - image/etc/group \ - image/etc/init \ - image/etc/login \ - image/etc/mdev.conf \ - image/etc/mdev/listen \ - image/etc/mdev/net/add \ - image/etc/mdev/wait \ - image/etc/parse-devname \ - image/etc/passwd \ - image/etc/s6-linux-init/env/WAYLAND_DISPLAY \ - image/etc/s6-linux-init/env/XDG_RUNTIME_DIR \ - image/etc/s6-linux-init/run-image/service/dbus/notification-fd \ - image/etc/s6-linux-init/run-image/service/dbus/run \ - image/etc/s6-linux-init/run-image/service/dbus/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/dbus/template/run \ - image/etc/s6-linux-init/run-image/service/getty-tty1/run \ - image/etc/s6-linux-init/run-image/service/getty-tty2/run \ - image/etc/s6-linux-init/run-image/service/getty-tty3/run \ - image/etc/s6-linux-init/run-image/service/getty-tty4/run \ - image/etc/s6-linux-init/run-image/service/s6-svscan-log/notification-fd \ - image/etc/s6-linux-init/run-image/service/s6-svscan-log/run \ - image/etc/s6-linux-init/run-image/service/serial-getty-generator/run \ - image/etc/s6-linux-init/run-image/service/serial-getty/notification-fd \ - image/etc/s6-linux-init/run-image/service/serial-getty/run \ - image/etc/s6-linux-init/run-image/service/serial-getty/template/run \ - image/etc/s6-linux-init/run-image/service/vhost-user-fs/notification-fd \ - image/etc/s6-linux-init/run-image/service/vhost-user-fs/run \ - image/etc/s6-linux-init/run-image/service/vhost-user-fs/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/vhost-user-fs/template/run \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/notification-fd \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/run \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/data/check \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/run \ - image/etc/s6-linux-init/run-image/service/vmm/notification-fd \ - image/etc/s6-linux-init/run-image/service/vmm/run \ - image/etc/s6-linux-init/run-image/service/vmm/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/notification-fd \ - image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/run \ - image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/template/run \ - image/etc/s6-linux-init/scripts/rc.init \ - image/etc/xdg/weston/autolaunch \ - image/etc/xdg/weston/weston.ini \ - image/usr/share/dbus-1/services/org.freedesktop.portal.Documents.service \ - image/usr/bin/assign-devices \ - image/usr/bin/create-vm-dependencies \ - image/usr/bin/run-appimage \ - image/usr/bin/run-vmm \ - image/usr/bin/vm-console \ - image/usr/bin/vm-import \ - image/usr/bin/vm-start \ - image/usr/bin/vm-stop \ - image/usr/bin/xdg-open - DIRS = \ dev \ etc/s6-linux-init/env \ @@ -101,18 +44,9 @@ DIRS = \ FIFOS = etc/s6-linux-init/run-image/service/s6-svscan-log/fifo -# These are separate because they need to be included, but putting -# them as make dependencies would confuse make. -LINKS = \ - image/bin \ - image/etc/s6-linux-init/run-image/opengl-driver \ - image/etc/s6-linux-init/run-image/service/vmm/template/run \ - image/lib \ - image/sbin - BUILD_FILES = build/etc/s6-rc -$(dest): ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty build/fifo +$(dest): ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty build/fifo file-list.mk ( \ cat $(PACKAGES_FILE) ;\ for file in $(FILES) $(LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ @@ -128,36 +62,6 @@ build/fifo: build/empty: mkdir -p $@ -S6_RC_FILES = \ - image/etc/s6-rc/card0/type \ - image/etc/s6-rc/card0/up \ - image/etc/s6-rc/core/type \ - image/etc/s6-rc/core/up \ - image/etc/s6-rc/kvm/timeout-up \ - image/etc/s6-rc/kvm/type \ - image/etc/s6-rc/kvm/up \ - image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ - image/etc/s6-rc/mdevd-coldplug/type \ - image/etc/s6-rc/mdevd-coldplug/up \ - image/etc/s6-rc/mdevd/notification-fd \ - image/etc/s6-rc/mdevd/run \ - image/etc/s6-rc/mdevd/type \ - image/etc/s6-rc/ok-all/contents \ - image/etc/s6-rc/ok-all/type \ - image/etc/s6-rc/static-nodes/type \ - image/etc/s6-rc/static-nodes/up \ - image/etc/s6-rc/sys-vmms/dependencies.d/vmm-env \ - image/etc/s6-rc/sys-vmms/type \ - image/etc/s6-rc/sys-vmms/up \ - image/etc/s6-rc/vm-env/contents \ - image/etc/s6-rc/vm-env/type \ - image/etc/s6-rc/vmm-env/contents \ - image/etc/s6-rc/vmm-env/type \ - image/etc/s6-rc/weston/dependencies.d/card0 \ - image/etc/s6-rc/weston/notification-fd \ - image/etc/s6-rc/weston/run \ - image/etc/s6-rc/weston/type - # s6-rc-compile's input is a directory, but that doesn't play nice # with Make, because it won't know to update if some file in the # directory is changed, or a file is created or removed in a @@ -165,7 +69,7 @@ S6_RC_FILES = \ # including files that aren't intended to be part of the input, like # temporary editor files or .license files. So for all these reasons, # only explicitly listed files are made available to s6-rc-compile. -build/etc/s6-rc: $(S6_RC_FILES) +build/etc/s6-rc: $(S6_RC_FILES) file-list.mk mkdir -p $$(dirname $@) rm -rf $@ set -uo pipefail && dir=$$(mktemp -d) && \ diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk new file mode 100644 index 0000000000000000000000000000000000000000..58cda39f85f720ab46f025bc72f1a98f108f1c25 --- /dev/null +++ b/host/rootfs/file-list.mk @@ -0,0 +1,99 @@ +# SPDX-License-Identifier: EUPL-1.2+ +# SPDX-FileCopyrightText: 2021-2024 Alyssa Ross +# Generated by scripts/genfile.sh. Any changes will be overwritten. +FILES = \ + image/etc/fonts/fonts.conf \ + image/etc/fstab \ + image/etc/group \ + image/etc/init \ + image/etc/login \ + image/etc/mdev.conf \ + image/etc/mdev/listen \ + image/etc/mdev/net/add \ + image/etc/mdev/wait \ + image/etc/parse-devname \ + image/etc/passwd \ + image/etc/s6-linux-init/env/WAYLAND_DISPLAY \ + image/etc/s6-linux-init/env/XDG_RUNTIME_DIR \ + image/etc/s6-linux-init/run-image/service/dbus/notification-fd \ + image/etc/s6-linux-init/run-image/service/dbus/run \ + image/etc/s6-linux-init/run-image/service/dbus/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/dbus/template/run \ + image/etc/s6-linux-init/run-image/service/getty-tty1/run \ + image/etc/s6-linux-init/run-image/service/getty-tty2/run \ + image/etc/s6-linux-init/run-image/service/getty-tty3/run \ + image/etc/s6-linux-init/run-image/service/getty-tty4/run \ + image/etc/s6-linux-init/run-image/service/s6-svscan-log/notification-fd \ + image/etc/s6-linux-init/run-image/service/s6-svscan-log/run \ + image/etc/s6-linux-init/run-image/service/serial-getty-generator/run \ + image/etc/s6-linux-init/run-image/service/serial-getty/notification-fd \ + image/etc/s6-linux-init/run-image/service/serial-getty/run \ + image/etc/s6-linux-init/run-image/service/serial-getty/template/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-fs/notification-fd \ + image/etc/s6-linux-init/run-image/service/vhost-user-fs/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-fs/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/vhost-user-fs/template/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/notification-fd \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/data/check \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/run \ + image/etc/s6-linux-init/run-image/service/vmm/notification-fd \ + image/etc/s6-linux-init/run-image/service/vmm/run \ + image/etc/s6-linux-init/run-image/service/vmm/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/notification-fd \ + image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/run \ + image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/template/run \ + image/etc/s6-linux-init/scripts/rc.init \ + image/etc/xdg/weston/autolaunch \ + image/etc/xdg/weston/weston.ini \ + image/usr/bin/assign-devices \ + image/usr/bin/create-vm-dependencies \ + image/usr/bin/run-appimage \ + image/usr/bin/run-vmm \ + image/usr/bin/vm-console \ + image/usr/bin/vm-import \ + image/usr/bin/vm-start \ + image/usr/bin/vm-stop \ + image/usr/bin/xdg-open \ + image/usr/share/dbus-1/services/org.freedesktop.portal.Documents.service + +# These are separate because they need to be included, but putting +# them as make dependencies would confuse make. +LINKS = \ + image/bin \ + image/etc/s6-linux-init/run-image/opengl-driver \ + image/etc/s6-linux-init/run-image/service/vmm/template/run \ + image/lib \ + image/sbin + +S6_RC_FILES = \ + image/etc/s6-rc/card0/type \ + image/etc/s6-rc/card0/up \ + image/etc/s6-rc/core/type \ + image/etc/s6-rc/core/up \ + image/etc/s6-rc/kvm/timeout-up \ + image/etc/s6-rc/kvm/type \ + image/etc/s6-rc/kvm/up \ + image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ + image/etc/s6-rc/mdevd-coldplug/type \ + image/etc/s6-rc/mdevd-coldplug/up \ + image/etc/s6-rc/mdevd/notification-fd \ + image/etc/s6-rc/mdevd/run \ + image/etc/s6-rc/mdevd/type \ + image/etc/s6-rc/ok-all/contents \ + image/etc/s6-rc/ok-all/type \ + image/etc/s6-rc/static-nodes/type \ + image/etc/s6-rc/static-nodes/up \ + image/etc/s6-rc/sys-vmms/dependencies.d/vmm-env \ + image/etc/s6-rc/sys-vmms/type \ + image/etc/s6-rc/sys-vmms/up \ + image/etc/s6-rc/vm-env/contents \ + image/etc/s6-rc/vm-env/type \ + image/etc/s6-rc/vmm-env/contents \ + image/etc/s6-rc/vmm-env/type \ + image/etc/s6-rc/weston/dependencies.d/card0 \ + image/etc/s6-rc/weston/notification-fd \ + image/etc/s6-rc/weston/run \ + image/etc/s6-rc/weston/type diff --git a/img/app/Makefile b/img/app/Makefile index 8b9ecf990fe10adb7aa19b46935826d684f10686..df54cb52bbb21e2e0ad6d7933edf84f8448e1f04 100644 --- a/img/app/Makefile +++ b/img/app/Makefile @@ -5,6 +5,7 @@ .POSIX: include ../../lib/common.mk +include file-list.mk prefix = build/host libdir = $(prefix)/lib @@ -29,45 +30,13 @@ $(imgdir)/appvm/blk/root.img: ../../scripts/make-gpt.sh ../../scripts/sfdisk-fie build/rootfs.erofs:root:5460386f-2203-4911-8694-91400125c604:root mv $@.tmp $@ -VM_FILES = \ - image/etc/dbus-1/session.conf \ - image/etc/fonts/fonts.conf \ - image/etc/fstab \ - image/etc/init \ - image/etc/mdev.conf \ - image/etc/mdev/iface \ - image/etc/mdev/listen \ - image/etc/mdev/virtiofs \ - image/etc/mdev/wait \ - image/etc/passwd \ - image/etc/pipewire/pipewire.conf \ - image/etc/resolv.conf \ - image/etc/s6-linux-init/env/DBUS_SESSION_BUS_ADDRESS \ - image/etc/s6-linux-init/env/DISPLAY \ - image/etc/s6-linux-init/env/GTK_USE_PORTAL \ - image/etc/s6-linux-init/env/NIX_XDG_DESKTOP_PORTAL_DIR \ - image/etc/s6-linux-init/env/WAYLAND_DISPLAY \ - image/etc/s6-linux-init/env/XDG_RUNTIME_DIR \ - image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ - image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/notification-fd \ - image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run \ - image/etc/s6-linux-init/scripts/rc.init \ - image/etc/s6-linux-init/scripts/rc.shutdown \ - image/etc/s6-linux-init/scripts/rc.shutdown.final \ - image/etc/wireplumber/wireplumber.conf.d/99_spectrum.conf \ - image/etc/xdg/xdg-desktop-portal/portals.conf - -VM_DIRS = dev run proc sys tmp \ +DIRS = dev run proc sys tmp \ etc/s6-linux-init/run-image/service \ etc/s6-linux-init/run-image/user \ etc/s6-linux-init/run-image/wait -VM_FIFOS = etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/fifo +FIFOS = etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/fifo -# These are separate because they need to be included, but putting -# them as make dependencies would confuse make. -VM_LINKS = image/bin image/etc/ssl/certs/ca-certificates.crt image/sbin - -VM_BUILD_FILES = build/etc/s6-rc +BUILD_FILES = build/etc/s6-rc build/fifo: mkdir -p build @@ -76,48 +45,21 @@ build/fifo: build/empty: mkdir -p $@ -build/rootfs.erofs: ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(VM_FILES) $(VM_BUILD_FILES) build/empty build/fifo +build/rootfs.erofs: ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty build/fifo file-list.mk ( \ cat $(PACKAGES_FILE) ;\ - for file in $(VM_FILES) $(VM_LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ - for file in $(VM_BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ - printf 'build/empty\n%s\n' $(VM_DIRS) ;\ - printf 'build/fifo\n%s\n' $(VM_FIFOS) ;\ + for file in $(FILES) $(LINKS); do printf '%s\n%s\n' $$file $${file#image/}; done ;\ + for file in $(BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ + printf 'build/empty\n%s\n' $(DIRS) ;\ + printf 'build/fifo\n%s\n' $(FIFOS) ;\ ) | ../../scripts/make-erofs.sh $@ -VM_S6_RC_FILES = \ - image/etc/s6-rc/app/dependencies.d/dbus \ - image/etc/s6-rc/app/dependencies.d/pipewire \ - image/etc/s6-rc/app/dependencies.d/wayland-proxy-virtwl \ - image/etc/s6-rc/app/run \ - image/etc/s6-rc/app/type \ - image/etc/s6-rc/dbus/notification-fd \ - image/etc/s6-rc/dbus/run \ - image/etc/s6-rc/dbus/type \ - image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ - image/etc/s6-rc/mdevd-coldplug/type \ - image/etc/s6-rc/mdevd-coldplug/up \ - image/etc/s6-rc/mdevd/notification-fd \ - image/etc/s6-rc/mdevd/run \ - image/etc/s6-rc/mdevd/type \ - image/etc/s6-rc/ok-all/contents \ - image/etc/s6-rc/ok-all/type \ - image/etc/s6-rc/pipewire/notification-fd \ - image/etc/s6-rc/pipewire/run \ - image/etc/s6-rc/pipewire/type \ - image/etc/s6-rc/wayland-proxy-virtwl/notification-fd \ - image/etc/s6-rc/wayland-proxy-virtwl/run \ - image/etc/s6-rc/wayland-proxy-virtwl/type \ - image/etc/s6-rc/wireplumber/dependencies.d/dbus \ - image/etc/s6-rc/wireplumber/dependencies.d/pipewire \ - image/etc/s6-rc/wireplumber/run \ - image/etc/s6-rc/wireplumber/type - -build/etc/s6-rc: $(VM_S6_RC_FILES) + +build/etc/s6-rc: $(S6_RC_FILES) file-list.mk mkdir -p $$(dirname $@) rm -rf $@ set -uo pipefail && dir=$$(mktemp -d) && \ - { tar -c $(VM_S6_RC_FILES) | tar -C $$dir -x --strip-components 3; } && \ + { tar -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 3; } && \ s6-rc-compile $@ $$dir; \ exit=$$?; rm -r $$dir; exit $$exit diff --git a/img/app/file-list.mk b/img/app/file-list.mk new file mode 100644 index 0000000000000000000000000000000000000000..2afe652bbc87cacb7b823a5edf340d861e9bb9ba --- /dev/null +++ b/img/app/file-list.mk @@ -0,0 +1,65 @@ +# SPDX-License-Identifier: EUPL-1.2+ +# SPDX-FileCopyrightText: 2021-2024 Alyssa Ross +# Generated by scripts/genfile.sh. Any changes will be overwritten. +FILES = \ + image/etc/dbus-1/session.conf \ + image/etc/fonts/fonts.conf \ + image/etc/fstab \ + image/etc/init \ + image/etc/mdev.conf \ + image/etc/mdev/iface \ + image/etc/mdev/listen \ + image/etc/mdev/virtiofs \ + image/etc/mdev/wait \ + image/etc/passwd \ + image/etc/pipewire/pipewire.conf \ + image/etc/resolv.conf \ + image/etc/s6-linux-init/env/DBUS_SESSION_BUS_ADDRESS \ + image/etc/s6-linux-init/env/DISPLAY \ + image/etc/s6-linux-init/env/GTK_USE_PORTAL \ + image/etc/s6-linux-init/env/NIX_XDG_DESKTOP_PORTAL_DIR \ + image/etc/s6-linux-init/env/WAYLAND_DISPLAY \ + image/etc/s6-linux-init/env/XDG_RUNTIME_DIR \ + image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ + image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/notification-fd \ + image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run \ + image/etc/s6-linux-init/scripts/rc.init \ + image/etc/s6-linux-init/scripts/rc.shutdown \ + image/etc/s6-linux-init/scripts/rc.shutdown.final \ + image/etc/wireplumber/wireplumber.conf.d/99_spectrum.conf \ + image/etc/xdg/xdg-desktop-portal/portals.conf + +# These are separate because they need to be included, but putting +# them as make dependencies would confuse make. +LINKS = \ + image/bin \ + image/etc/ssl/certs/ca-certificates.crt \ + image/sbin + +S6_RC_FILES = \ + image/etc/s6-rc/app/dependencies.d/dbus \ + image/etc/s6-rc/app/dependencies.d/pipewire \ + image/etc/s6-rc/app/dependencies.d/wayland-proxy-virtwl \ + image/etc/s6-rc/app/run \ + image/etc/s6-rc/app/type \ + image/etc/s6-rc/dbus/notification-fd \ + image/etc/s6-rc/dbus/run \ + image/etc/s6-rc/dbus/type \ + image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ + image/etc/s6-rc/mdevd-coldplug/type \ + image/etc/s6-rc/mdevd-coldplug/up \ + image/etc/s6-rc/mdevd/notification-fd \ + image/etc/s6-rc/mdevd/run \ + image/etc/s6-rc/mdevd/type \ + image/etc/s6-rc/ok-all/contents \ + image/etc/s6-rc/ok-all/type \ + image/etc/s6-rc/pipewire/notification-fd \ + image/etc/s6-rc/pipewire/run \ + image/etc/s6-rc/pipewire/type \ + image/etc/s6-rc/wayland-proxy-virtwl/notification-fd \ + image/etc/s6-rc/wayland-proxy-virtwl/run \ + image/etc/s6-rc/wayland-proxy-virtwl/type \ + image/etc/s6-rc/wireplumber/dependencies.d/dbus \ + image/etc/s6-rc/wireplumber/dependencies.d/pipewire \ + image/etc/s6-rc/wireplumber/run \ + image/etc/s6-rc/wireplumber/type diff --git a/scripts/genfiles.awk b/scripts/genfiles.awk new file mode 100644 index 0000000000000000000000000000000000000000..935eebbdc7f0e2aa07b0b6439ab53d1f50940929 --- /dev/null +++ b/scripts/genfiles.awk @@ -0,0 +1,77 @@ +# SPDX-License-Identifier: EUPL-1.2+ +# SPDX-FileCopyrightText: 2021-2024 Alyssa Ross +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour +BEGIN { + RS = "\n"; + FS = "\t"; + modes["120000"] = "symlink"; + modes["100644"] = "regular"; + modes["100755"] = "regular"; +} + +function fail(msg) { + exit_code = 1; + print msg > "/dev/stderr"; + exit 1; +} + +# Extract data from built-in variables. +{ + filename = $2; + raw_mode = $1; + # awk autocreates empty string entries if the key is invalid, + # but the code exits in this case so that is okay. + mode = modes[raw_mode]; +} + +filename !~ /^[[:alnum:]_./-]+$/ { + fail("filename '" filename "' has forbidden characters"); +} + +# Skip license files +/\.license$/ { next } + +filename ~ /^image\/etc\/s6-rc\// { + if (mode != "regular") { + fail("s6-rc-compile input '" filename "' isn't a regular file"); + } + rc_files[rc_count++] = filename; + next; +} + +mode == "symlink" { + symlinks[symlink_count++] = filename; + next; +} + +mode == "regular" { + files[file_count++] = filename; + next; +} + +{ fail("File '" filename "' is not regular file or symlink (mode " raw_mode ")"); } + +END { + if (exit_code) { + exit exit_code; + } + printf ("# SPDX-License-Identifier: EUPL-1.2+\n" \ +"# SPDX-FileCopyrightText: 2021-2024 Alyssa Ross \n" \ +"# Generated by scripts/genfile.sh. Any changes will be overwritten.\n" \ +"FILES ="); + for (array_index = 0; array_index < file_count; array_index += 1) { + printf " \\\n\t%s", files[array_index]; + } + printf ("\n\n" \ +"# These are separate because they need to be included, but putting\n" \ +"# them as make dependencies would confuse make.\n" \ +"LINKS ="); + for (array_index = 0; array_index < symlink_count; array_index += 1) { + printf " \\\n\t%s", symlinks[array_index]; + } + printf "\n\nS6_RC_FILES ="; + for (array_index = 0; array_index < rc_count; array_index += 1) { + printf " \\\n\t%s", rc_files[array_index]; + } + print ""; +} diff --git a/scripts/genfiles.sh b/scripts/genfiles.sh new file mode 100755 index 0000000000000000000000000000000000000000..a595d9effd64fe110ed0c2875bbd30f4428ee3f7 --- /dev/null +++ b/scripts/genfiles.sh @@ -0,0 +1,24 @@ +#!/bin/sh -- +set -euo pipefail +export LC_ALL=C LANGUAGE=C +dir=$(git rev-parse --show-toplevel) +for i in host/rootfs img/app vm/sys/net; do + output_file=$i/file-list.mk + { + git -C "$i" -c core.quotePath=true ls-files $'--format=%(objectmode)\t%(path)' -- image | + sort -t $'\t' -k 2 + } | + awk -f scripts/genfiles.awk > "$output_file.tmp" + if [ -f "$output_file" ]; then + # Avoid changing output file if it is up to date, as that + # would cause unnecessary rebuilds. + if cmp -s -- "$output_file.tmp" "$output_file"; then + rm -- "$output_file.tmp" + continue + else + astatus=$? + if [ "$astatus" != 1 ]; then exit "$astatus"; fi + fi + fi + mv -- "$output_file.tmp" "$output_file" +done diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile index 0193cc8c09f895a6021de0d02fde20362818e185..6f8664aea3b2b9bfdf569c4b089695d2a575668a 100644 --- a/vm/sys/net/Makefile +++ b/vm/sys/net/Makefile @@ -4,6 +4,7 @@ .POSIX: include ../../../lib/common.mk +include file-list.mk prefix = build/host libdir = $(prefix)/lib @@ -28,61 +29,24 @@ $(vmdir)/netvm/blk/root.img: ../../../scripts/make-gpt.sh ../../../scripts/sfdis build/rootfs.erofs:root:ea21da27-0391-48da-9235-9d2ab2ca7844:root mv $@.tmp $@ -VM_FILES = \ - image/etc/dbus-1/system.conf \ - image/etc/fstab \ - image/etc/init \ - image/etc/mdev.conf \ - image/etc/mdev/iface \ - image/etc/nftables.conf \ - image/etc/passwd \ - image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ - image/etc/s6-linux-init/scripts/rc.init \ - image/etc/sysctl.conf -VM_DIRS = dev etc/s6-linux-init/env run proc sys var/lib/connman - -# These are separate because they need to be included, but putting -# them as make dependencies would confuse make. -VM_LINKS = image/bin image/lib image/sbin image/var/run - -VM_BUILD_FILES = build/etc/s6-rc +BUILD_FILES = build/etc/s6-rc build/empty: mkdir -p $@ -build/rootfs.erofs: ../../../scripts/make-erofs.sh $(PACKAGES_FILE) $(VM_FILES) $(VM_BUILD_FILES) build/empty +build/rootfs.erofs: ../../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty file-list.mk ( \ cat $(PACKAGES_FILE) ;\ - for file in $(VM_FILES) $(VM_LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ - for file in $(VM_BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ - printf 'build/empty\n%s\n' $(VM_DIRS) ;\ + for file in $(FILES) $(LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ + for file in $(BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ + printf 'build/empty\n%s\n' $(DIRS) ;\ ) | ../../../scripts/make-erofs.sh $@ -VM_S6_RC_FILES = \ - image/etc/s6-rc/connman/dependencies.d/dbus \ - image/etc/s6-rc/connman/run \ - image/etc/s6-rc/connman/type \ - image/etc/s6-rc/dbus/notification-fd \ - image/etc/s6-rc/dbus/run \ - image/etc/s6-rc/dbus/type \ - image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ - image/etc/s6-rc/mdevd-coldplug/type \ - image/etc/s6-rc/mdevd-coldplug/up \ - image/etc/s6-rc/mdevd/notification-fd \ - image/etc/s6-rc/mdevd/run \ - image/etc/s6-rc/mdevd/type \ - image/etc/s6-rc/nftables/type \ - image/etc/s6-rc/nftables/up \ - image/etc/s6-rc/ok-all/contents \ - image/etc/s6-rc/ok-all/type \ - image/etc/s6-rc/sysctl/type \ - image/etc/s6-rc/sysctl/up - -build/etc/s6-rc: $(VM_S6_RC_FILES) +build/etc/s6-rc: $(S6_RC_FILES) file-list.mk mkdir -p $$(dirname $@) rm -rf $@ set -uo pipefail && dir=$$(mktemp -d) && \ - { tar -c $(VM_S6_RC_FILES) | tar -C $$dir -x --strip-components 3; } && \ + { tar -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 3; } && \ s6-rc-compile $@ $$dir; \ exit=$$?; rm -r $$dir; exit $$exit diff --git a/vm/sys/net/file-list.mk b/vm/sys/net/file-list.mk new file mode 100644 index 0000000000000000000000000000000000000000..384b370fc84a279fff31162a571398cc0507b010 --- /dev/null +++ b/vm/sys/net/file-list.mk @@ -0,0 +1,42 @@ +# SPDX-License-Identifier: EUPL-1.2+ +# SPDX-FileCopyrightText: 2021-2024 Alyssa Ross +# Generated by scripts/genfile.sh. Any changes will be overwritten. +FILES = \ + image/etc/dbus-1/system.conf \ + image/etc/fstab \ + image/etc/init \ + image/etc/mdev.conf \ + image/etc/mdev/iface \ + image/etc/nftables.conf \ + image/etc/passwd \ + image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ + image/etc/s6-linux-init/scripts/rc.init \ + image/etc/sysctl.conf + +# These are separate because they need to be included, but putting +# them as make dependencies would confuse make. +LINKS = \ + image/bin \ + image/lib \ + image/sbin \ + image/var/run + +S6_RC_FILES = \ + image/etc/s6-rc/connman/dependencies.d/dbus \ + image/etc/s6-rc/connman/run \ + image/etc/s6-rc/connman/type \ + image/etc/s6-rc/dbus/notification-fd \ + image/etc/s6-rc/dbus/run \ + image/etc/s6-rc/dbus/type \ + image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ + image/etc/s6-rc/mdevd-coldplug/type \ + image/etc/s6-rc/mdevd-coldplug/up \ + image/etc/s6-rc/mdevd/notification-fd \ + image/etc/s6-rc/mdevd/run \ + image/etc/s6-rc/mdevd/type \ + image/etc/s6-rc/nftables/type \ + image/etc/s6-rc/nftables/up \ + image/etc/s6-rc/ok-all/contents \ + image/etc/s6-rc/ok-all/type \ + image/etc/s6-rc/sysctl/type \ + image/etc/s6-rc/sysctl/up --- base-commit: 2e9d63d3e213939615d3dcc44bcb3aa2e8dfa77f change-id: 20250831-genfiles-46787fc10869 -- Sincerely, Demi Marie Obenour (she/her/hers)