From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id 80473F8E2; Wed, 01 Oct 2025 18:32:20 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 41183F8D8; Wed, 01 Oct 2025 18:32:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_PASS,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=4.0.1 Received: from mail-yw1-x112d.google.com (mail-yw1-x112d.google.com [IPv6:2607:f8b0:4864:20::112d]) by atuin.qyliss.net (Postfix) with ESMTPS id 6A42AF8D4 for ; Wed, 01 Oct 2025 18:32:14 +0000 (UTC) Received: by mail-yw1-x112d.google.com with SMTP id 00721157ae682-749399349ddso3153387b3.3 for ; Wed, 01 Oct 2025 11:32:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1759343533; x=1759948333; darn=spectrum-os.org; h=cc:to:references:in-reply-to:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=8zSZb5XBmxkAyKHviPyUcS8YoQMvExRLvRT7Eh6wv6A=; b=BnQi72T6Q8aZMhwaV87G0KajpHlEZbz3Q23zaJvfv0btdLHRkQh9oAGEj1f7GbUtKk 0b2H2cV0s9URoaGQgQPClU9EE5sU14tMTTUaAHgosOxwl9WBQjBFwyIQJz6IO3p/0tM6 cWsWZegjCfn3jMa7ra1z+TKV+qagJB4cUdq7vch4YrHu+iS4H6LsukmQgflpCKLvQIIL TaHd3Iv8G0WSLyth0yAHmPXkrDjQCIiHKfGzhg3OukO31/I9HHzqmOLE6R1pz5XCuaci WtTyE3oJDX8Maz5Vcledbt2mJ3X6kge3vEaRCkctBGLCjEZPkZJwARhUZw9A+zM9PBOP 5neQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1759343533; x=1759948333; h=cc:to:references:in-reply-to:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8zSZb5XBmxkAyKHviPyUcS8YoQMvExRLvRT7Eh6wv6A=; b=NCLs/MWIQL7smTQLxVyk+K8Q4JMUC4lvOBWpo64+WqVzz1CwDrWuG9KcY4UnlXadZU kYIJ77pVb/8zCpc4MEDC8no4gmJrGic+jXu2jmSMylFus7v6gNZxo2oz+XzQcvsg2DfX DVVKMrLv8i3fINPtWVENUjwdkh3/v2g3kCJ1jRHmv8RiQPNcUvK80sOcAOah+EG3JgjB 4SN5nSa6ACedoV76AQ0gFraqKbY1QssPlCpwuzMBn6Jt2rHIoA8kLUvB9WGgXHdfecFH ptabssrFycpEFdVi9YRUs+Xadgu7x42C/rouMvOiG+iv9jYovlSt9QMDW7KLwKJFkG6I HRLA== X-Gm-Message-State: AOJu0YzKvjsL50CvurLMelvjV1tcglFxaFmBfSngUVygjaJILpifzSju kczLimkU1w12TUIJlCrQ5HSSx0sPisdyKygQGnlktt34ecLmHnc/nQuHkCtG0g== X-Gm-Gg: ASbGnctQKd5s9VvuMdnzY2vWvS4Ms2J1o2NBHkWQTVLTJ70XgbJkd3MHRxXij14Jdv3 h4LEMFcBd5kCYiAgbstCQlM3WjAUG5Noc/2VOJ2aDQ87HEKVMKtaiFS4eitK9ZyXa1TSTJ1J7Hg n5zam8OX2pFJcAQSLd+KFA0VSBwZzD5ng2LZ3Iqob2Y47aGcUIqkHYSTiTVoxoEjcx2uHytchk5 NlWYf6T+N4PyChnugvqLqxFAWy2c/3fknCcktFJVgeHlTV8QBwkuMyWtAZ+4lL7+WvOVdaJjdeu F9eYQgpvNHOdeyy+wJ7+PnEJRYJPY34yK5tNl9gv3nOk4k/MkOv0uae7s1o235ohzBlyJ7ShIZ4 hJkn7fl0XDyg0+9zJQsaZqsOFJVmhabkvJHu1gFZj5p5WFUSj1+sKnA3SrmCEU8BJlXcGVqKM2w /pA3E09Dk746OZAeRoMpFUyRY+5YC50wR5B7d309izcz1D9I0EN6s5IxxokDTvBS6wd4dfqb762 9Bszw== X-Google-Smtp-Source: AGHT+IHvfDN66tPPzRV13icRAjxP9EN7YMBluEgJa+ptgqCAaFXKriN7E/xCGnb2bQVEP5lilWjddw== X-Received: by 2002:a05:690c:3341:b0:76d:5713:7d65 with SMTP id 00721157ae682-77f6f382a32mr57565947b3.43.1759343532697; Wed, 01 Oct 2025 11:32:12 -0700 (PDT) Received: from localhost.localdomain (h96-60-249-169.cncrtn.broadband.dynamic.tds.net. [96.60.249.169]) by smtp.gmail.com with UTF8SMTPSA id 00721157ae682-77f81d338d3sm1209557b3.45.2025.10.01.11.32.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Oct 2025 11:32:12 -0700 (PDT) From: Demi Marie Obenour Date: Wed, 01 Oct 2025 14:30:54 -0400 Subject: [PATCH v8] Generate file lists from a script MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20251001-genfiles-v8-1-171a29081ab7@gmail.com> X-B4-Tracking: v=1; b=H4sIAF1z3WgC/2XPwWrEIBDG8VdZPNfiOOrEnvoepYeok6ywm5Skh JYl7153e+iQHj/x94e5qZWXyqt6Od3Uwltd6zy10T2dVD7308i6lraVNdabDkGPPA31wqt2gTo aMpguRNW+fyw81K9H6u297XNdP+fl+1He4P76G4kG/yIbaKNzjhGHnIBdfh2vfb085/mq7pHNC ghGQNsgEic21GMu/yAKaCVEDbqEbFOgQMnZI3QSins316BD8qn07XJDR+glDAL6BhGjK0PChGi OMEhIAoYGLQD0sQMfQzlCkjAKSA36noxldt4mkHDf9x/ZRGJY9gEAAA== X-Change-ID: 20250831-genfiles-46787fc10869 In-Reply-To: <20250929-genfiles-v7-1-5a702ee452b1@gmail.com> References: <20250929-genfiles-v7-1-5a702ee452b1@gmail.com> To: Spectrum OS Development X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1759343453; l=33182; i=demiobenour@gmail.com; s=20250729; h=from:subject:message-id; bh=yoxIg/EirhCMGzFa8MroLIqvaU8vonOS1u3OaoZfOmA=; b=6yvJOVkOpMs+XeeKyA0siiWVFjIwVPpndYiFwa9ZiWabiNYCcTaIxypSRXigpmB+zegspzWiT isnoWzr5DuDANmEKvoH4ZtV4ERq0XcNVgBYPJG+Krl6wNOSVZ4EHOho X-Developer-Key: i=demiobenour@gmail.com; a=ed25519; pk=X57Q4/YQDj9t4SBeKaDwvXYKB6quZJVx/DE2Ly2out0= Message-ID-Hash: T6T6HBSK7UXX3VIRYKZPY2GLZW7KU626 X-Message-ID-Hash: T6T6HBSK7UXX3VIRYKZPY2GLZW7KU626 X-MailFrom: demiobenour@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Alyssa Ross , Demi Marie Obenour X-Mailman-Version: 3.3.9 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Right now, the makefiles in host/rootfs, vm/sys/net, and img/app have manually-maintained lists of files and symlinks. These duplicate the information in the git repository and can easily get out of sync or cause unnecessary merge conflicts. Fix all of these issues by having the git repository be the source of truth, and using a script to generate the file lists. Developers can regenerate the lists before every commit, or even add a git hook to do that. Signed-off-by: Demi Marie Obenour --- This actually reduces the amount of code that has to be written by hand. --- Changes in v8: - Re-add missing DIRS in vm/sys/net/Makefile. - Re-add missing "$dir" in scripts/genfiles.sh. - Link to v7: https://spectrum-os.org/lists/archives/spectrum-devel/20250929-genfiles-v7-1-5a702ee452b1@gmail.com Changes in v7: - Do not break sorting of directories in host/rootfs/Makefile. - Add newline after generated copyright headers. - Add comment to scripts/genfiles.awk explaining use of exit_code variable. - Check filename instead of $0 to see if a file is a license file. This has no functional impact. - Add missing copyright to scripts/genfiles.sh. - Prevent reuse from tripping over scripts/genfiles.awk. - Link to v6: https://spectrum-os.org/lists/archives/spectrum-devel/20250927-genfiles-v6-1-2111a981596d@gmail.com Changes in v6: - Do not include comment in generated makefiles - Change makefile license to CC0. - Link to v5: https://spectrum-os.org/lists/archives/spectrum-devel/20250926-genfiles-v5-1-3394dfb3b330@gmail.com Changes in v5: - Use 'print ""' instead of 'print' in awk to print a newline. 'print' with no arguments implicitly prints $0 instead. This caused the generated makefiles to be incorrect. - Use S6_RC_FILES instead of VM_S6_RC_FILES in vm/sys/net/Makefile and img/app/Makefile. This prevented the image from being built. - Do not check for git repository being in a directory with a name ending in a newline. - Use shell redirection instead of awk redirection. - Do not include trailing DONE line in input to awk. - Link to v4: https://spectrum-os.org/lists/archives/spectrum-devel/20250921-genfiles-v4-1-4375bda78707@gmail.com Changes in v4: - Use /bin/sh instead of bash. - Do not assume that negated awk character classes match all bytes. - Do not check the mode of license files. - Use implicit awk variable initialization. - Use 'git rev-parse --show-toplevel' to find the repository root. - Remove wrongly added copyright header. - Improve documentation. - Remove git hooks. - Add missing copyright header. - Avoid non-portable /usr/bin/env -S. - Avoid assuming that awk is GNU awk. - Avoid non-portable awk -E. - Do not check for git bugs. - Fix link in v3 changelog. - Link to v3: https://spectrum-os.org/lists/archives/spectrum-devel/20250920-genfiles-v3-1-d6c2b6767b42@gmail.com Changes in v3: - Only include the file list generator. Move the rest to separate patch series. - Remove the update-file-list make targets from img/app/Makefile and vm/sys/net/Makefile. - Link to v2: https://spectrum-os.org/lists/archives/spectrum-devel/20250910-genfiles-v2-0-37ebe07a3cdc@gmail.com Changes in v2: - Drop the last patch (switching to /etc/s6-rc/compiled) as it is controversial and should be reviewed separately. - Add missing copyright notices. - Use a wrapper shell script to make the awk code easier to read. - Improve documentation. - Add helper scripts for use in git hooks and rebasing. - Link to v1: https://spectrum-os.org/lists/archives/spectrum-devel/20250903-genfiles-v1-0-cc993fcb1e4c@gmail.com/ --- Documentation/development/built-in-vms.adoc | 7 ++ host/rootfs/Makefile | 107 +--------------------------- host/rootfs/file-list.mk | 102 ++++++++++++++++++++++++++ img/app/Makefile | 84 ++++------------------ img/app/file-list.mk | 65 +++++++++++++++++ scripts/genfiles.awk | 85 ++++++++++++++++++++++ scripts/genfiles.sh | 26 +++++++ vm/sys/net/Makefile | 55 +++----------- vm/sys/net/file-list.mk | 41 +++++++++++ 9 files changed, 351 insertions(+), 221 deletions(-) diff --git a/Documentation/development/built-in-vms.adoc b/Documentation/development/built-in-vms.adoc index e90009ee5a3c2c254a7ae11e36121576b819eee7..d044e7509a43df4b015f312dac61be6c74cc7feb 100644 --- a/Documentation/development/built-in-vms.adoc +++ b/Documentation/development/built-in-vms.adoc @@ -3,6 +3,7 @@ :page-nav_order: 3 // SPDX-FileCopyrightText: 2023-2025 Alyssa Ross +// SPDX-FileCopyrightText: 2025 Demi Marie Obenour // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 Spectrum comes with some built-in application VMs, to demonstrate @@ -44,6 +45,12 @@ NOTE: As a special convenience, it's not necessary to run `make clean` if the only change to the Nix files is modifying the packages installed in the VM. +The list of files used for images is stored in a separate file, +file-list.mk. To update it, run scripts/genfiles.sh, which will +regenerate it from the output of `git ls-files`. This script uses +Git's index to generate the list, so only staged changes will be +reflected in its output. + By default, `make run` runs the VM in Cloud Hypervisor, to match the Spectrum environment. You can also add `VMM=crosvm` or `VMM=qemu` to your command line to run the VM in a different VMM. One reason to do diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index a0b32dc6af554d4122974e214e5ba2b648608bae..aa45ca1d5c18d0dfb78d19267f263cc4222e8e84 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -4,67 +4,10 @@ .POSIX: include ../../lib/common.mk +include file-list.mk dest = build/rootfs.erofs -FILES = \ - image/etc/fonts/fonts.conf \ - image/etc/fstab \ - image/etc/group \ - image/etc/init \ - image/etc/login \ - image/etc/mdev.conf \ - image/etc/mdev/listen \ - image/etc/mdev/net/add \ - image/etc/mdev/wait \ - image/etc/parse-devname \ - image/etc/passwd \ - image/etc/s6-linux-init/env/WAYLAND_DISPLAY \ - image/etc/s6-linux-init/env/XDG_RUNTIME_DIR \ - image/etc/s6-linux-init/run-image/service/dbus/notification-fd \ - image/etc/s6-linux-init/run-image/service/dbus/run \ - image/etc/s6-linux-init/run-image/service/dbus/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/dbus/template/run \ - image/etc/s6-linux-init/run-image/service/getty-tty1/run \ - image/etc/s6-linux-init/run-image/service/getty-tty2/run \ - image/etc/s6-linux-init/run-image/service/getty-tty3/run \ - image/etc/s6-linux-init/run-image/service/getty-tty4/run \ - image/etc/s6-linux-init/run-image/service/s6-svscan-log/notification-fd \ - image/etc/s6-linux-init/run-image/service/s6-svscan-log/run \ - image/etc/s6-linux-init/run-image/service/serial-getty-generator/run \ - image/etc/s6-linux-init/run-image/service/serial-getty/notification-fd \ - image/etc/s6-linux-init/run-image/service/serial-getty/run \ - image/etc/s6-linux-init/run-image/service/serial-getty/template/run \ - image/etc/s6-linux-init/run-image/service/vhost-user-fs/notification-fd \ - image/etc/s6-linux-init/run-image/service/vhost-user-fs/run \ - image/etc/s6-linux-init/run-image/service/vhost-user-fs/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/vhost-user-fs/template/run \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/notification-fd \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/run \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/data/check \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/run \ - image/etc/s6-linux-init/run-image/service/vmm/notification-fd \ - image/etc/s6-linux-init/run-image/service/vmm/run \ - image/etc/s6-linux-init/run-image/service/vmm/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/notification-fd \ - image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/run \ - image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/template/run \ - image/etc/s6-linux-init/scripts/rc.init \ - image/etc/xdg/weston/autolaunch \ - image/etc/xdg/weston/weston.ini \ - image/usr/bin/assign-devices \ - image/usr/bin/create-vm-dependencies \ - image/usr/bin/run-appimage \ - image/usr/bin/run-vmm \ - image/usr/bin/vm-console \ - image/usr/bin/vm-import \ - image/usr/bin/vm-start \ - image/usr/bin/vm-stop \ - image/usr/bin/xdg-open \ - image/usr/share/dbus-1/services/org.freedesktop.portal.Documents.service - DIRS = \ dev \ etc/s6-linux-init/env \ @@ -101,18 +44,9 @@ DIRS = \ FIFOS = etc/s6-linux-init/run-image/service/s6-svscan-log/fifo -# These are separate because they need to be included, but putting -# them as make dependencies would confuse make. -LINKS = \ - image/bin \ - image/etc/s6-linux-init/run-image/opengl-driver \ - image/etc/s6-linux-init/run-image/service/vmm/template/run \ - image/lib \ - image/sbin - BUILD_FILES = build/etc/s6-rc -$(dest): ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty build/fifo +$(dest): ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty build/fifo file-list.mk { \ cat $(PACKAGES_FILE) ;\ for file in $(FILES) $(LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ @@ -128,41 +62,6 @@ build/fifo: build/empty: mkdir -p $@ -S6_RC_FILES = \ - image/etc/s6-rc/card0/type \ - image/etc/s6-rc/card0/up \ - image/etc/s6-rc/core/type \ - image/etc/s6-rc/core/up \ - image/etc/s6-rc/kvm/timeout-up \ - image/etc/s6-rc/kvm/type \ - image/etc/s6-rc/kvm/up \ - image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ - image/etc/s6-rc/mdevd-coldplug/type \ - image/etc/s6-rc/mdevd-coldplug/up \ - image/etc/s6-rc/mdevd/notification-fd \ - image/etc/s6-rc/mdevd/run \ - image/etc/s6-rc/mdevd/type \ - image/etc/s6-rc/ok-all/contents.d/mdevd-coldplug \ - image/etc/s6-rc/ok-all/contents.d/sys-vmms \ - image/etc/s6-rc/ok-all/contents.d/vm-env \ - image/etc/s6-rc/ok-all/type \ - image/etc/s6-rc/static-nodes/type \ - image/etc/s6-rc/static-nodes/up \ - image/etc/s6-rc/sys-vmms/dependencies.d/vmm-env \ - image/etc/s6-rc/sys-vmms/type \ - image/etc/s6-rc/sys-vmms/up \ - image/etc/s6-rc/vm-env/contents.d/static-nodes \ - image/etc/s6-rc/vm-env/contents.d/weston \ - image/etc/s6-rc/vm-env/type \ - image/etc/s6-rc/vmm-env/contents.d/core \ - image/etc/s6-rc/vmm-env/contents.d/kvm \ - image/etc/s6-rc/vmm-env/contents.d/static-nodes \ - image/etc/s6-rc/vmm-env/type \ - image/etc/s6-rc/weston/dependencies.d/card0 \ - image/etc/s6-rc/weston/notification-fd \ - image/etc/s6-rc/weston/run \ - image/etc/s6-rc/weston/type - # s6-rc-compile's input is a directory, but that doesn't play nice # with Make, because it won't know to update if some file in the # directory is changed, or a file is created or removed in a @@ -170,7 +69,7 @@ S6_RC_FILES = \ # including files that aren't intended to be part of the input, like # temporary editor files or .license files. So for all these reasons, # only explicitly listed files are made available to s6-rc-compile. -build/etc/s6-rc: $(S6_RC_FILES) +build/etc/s6-rc: $(S6_RC_FILES) file-list.mk mkdir -p $$(dirname $@) rm -rf $@ set -uo pipefail && dir=$$(mktemp -d) && \ diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk new file mode 100644 index 0000000000000000000000000000000000000000..3694dc472a013fff931847171145465329fb800f --- /dev/null +++ b/host/rootfs/file-list.mk @@ -0,0 +1,102 @@ +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour + +FILES = \ + image/etc/fonts/fonts.conf \ + image/etc/fstab \ + image/etc/group \ + image/etc/init \ + image/etc/login \ + image/etc/mdev.conf \ + image/etc/mdev/listen \ + image/etc/mdev/net/add \ + image/etc/mdev/wait \ + image/etc/parse-devname \ + image/etc/passwd \ + image/etc/s6-linux-init/env/WAYLAND_DISPLAY \ + image/etc/s6-linux-init/env/XDG_RUNTIME_DIR \ + image/etc/s6-linux-init/run-image/service/dbus/notification-fd \ + image/etc/s6-linux-init/run-image/service/dbus/run \ + image/etc/s6-linux-init/run-image/service/dbus/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/dbus/template/run \ + image/etc/s6-linux-init/run-image/service/getty-tty1/run \ + image/etc/s6-linux-init/run-image/service/getty-tty2/run \ + image/etc/s6-linux-init/run-image/service/getty-tty3/run \ + image/etc/s6-linux-init/run-image/service/getty-tty4/run \ + image/etc/s6-linux-init/run-image/service/s6-svscan-log/notification-fd \ + image/etc/s6-linux-init/run-image/service/s6-svscan-log/run \ + image/etc/s6-linux-init/run-image/service/serial-getty-generator/run \ + image/etc/s6-linux-init/run-image/service/serial-getty/notification-fd \ + image/etc/s6-linux-init/run-image/service/serial-getty/run \ + image/etc/s6-linux-init/run-image/service/serial-getty/template/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-fs/notification-fd \ + image/etc/s6-linux-init/run-image/service/vhost-user-fs/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-fs/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/vhost-user-fs/template/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/notification-fd \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/data/check \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/run \ + image/etc/s6-linux-init/run-image/service/vmm/notification-fd \ + image/etc/s6-linux-init/run-image/service/vmm/run \ + image/etc/s6-linux-init/run-image/service/vmm/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/notification-fd \ + image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/run \ + image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/template/run \ + image/etc/s6-linux-init/scripts/rc.init \ + image/etc/xdg/weston/autolaunch \ + image/etc/xdg/weston/weston.ini \ + image/usr/bin/assign-devices \ + image/usr/bin/create-vm-dependencies \ + image/usr/bin/run-appimage \ + image/usr/bin/run-vmm \ + image/usr/bin/vm-console \ + image/usr/bin/vm-import \ + image/usr/bin/vm-start \ + image/usr/bin/vm-stop \ + image/usr/bin/xdg-open \ + image/usr/share/dbus-1/services/org.freedesktop.portal.Documents.service + +LINKS = \ + image/bin \ + image/etc/s6-linux-init/run-image/opengl-driver \ + image/etc/s6-linux-init/run-image/service/vmm/template/run \ + image/lib \ + image/sbin + +S6_RC_FILES = \ + image/etc/s6-rc/card0/type \ + image/etc/s6-rc/card0/up \ + image/etc/s6-rc/core/type \ + image/etc/s6-rc/core/up \ + image/etc/s6-rc/kvm/timeout-up \ + image/etc/s6-rc/kvm/type \ + image/etc/s6-rc/kvm/up \ + image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ + image/etc/s6-rc/mdevd-coldplug/type \ + image/etc/s6-rc/mdevd-coldplug/up \ + image/etc/s6-rc/mdevd/notification-fd \ + image/etc/s6-rc/mdevd/run \ + image/etc/s6-rc/mdevd/type \ + image/etc/s6-rc/ok-all/contents.d/mdevd-coldplug \ + image/etc/s6-rc/ok-all/contents.d/sys-vmms \ + image/etc/s6-rc/ok-all/contents.d/vm-env \ + image/etc/s6-rc/ok-all/type \ + image/etc/s6-rc/static-nodes/type \ + image/etc/s6-rc/static-nodes/up \ + image/etc/s6-rc/sys-vmms/dependencies.d/vmm-env \ + image/etc/s6-rc/sys-vmms/type \ + image/etc/s6-rc/sys-vmms/up \ + image/etc/s6-rc/vm-env/contents.d/static-nodes \ + image/etc/s6-rc/vm-env/contents.d/weston \ + image/etc/s6-rc/vm-env/type \ + image/etc/s6-rc/vmm-env/contents.d/core \ + image/etc/s6-rc/vmm-env/contents.d/kvm \ + image/etc/s6-rc/vmm-env/contents.d/static-nodes \ + image/etc/s6-rc/vmm-env/type \ + image/etc/s6-rc/weston/dependencies.d/card0 \ + image/etc/s6-rc/weston/notification-fd \ + image/etc/s6-rc/weston/run \ + image/etc/s6-rc/weston/type diff --git a/img/app/Makefile b/img/app/Makefile index 93eeefb53e23f250812039a4d28a7ccd59e79446..981889ebe55d9ba03228977f3dc0ea3f26d5c4fb 100644 --- a/img/app/Makefile +++ b/img/app/Makefile @@ -5,6 +5,7 @@ .POSIX: include ../../lib/common.mk +include file-list.mk prefix = build/host libdir = $(prefix)/lib @@ -29,45 +30,13 @@ $(imgdir)/appvm/blk/root.img: ../../scripts/make-gpt.sh ../../scripts/sfdisk-fie build/rootfs.erofs:root:5460386f-2203-4911-8694-91400125c604:root mv $@.tmp $@ -VM_FILES = \ - image/etc/dbus-1/session.conf \ - image/etc/fonts/fonts.conf \ - image/etc/fstab \ - image/etc/init \ - image/etc/mdev.conf \ - image/etc/mdev/iface \ - image/etc/mdev/listen \ - image/etc/mdev/virtiofs \ - image/etc/mdev/wait \ - image/etc/passwd \ - image/etc/pipewire/pipewire.conf \ - image/etc/resolv.conf \ - image/etc/s6-linux-init/env/DBUS_SESSION_BUS_ADDRESS \ - image/etc/s6-linux-init/env/DISPLAY \ - image/etc/s6-linux-init/env/GTK_USE_PORTAL \ - image/etc/s6-linux-init/env/NIX_XDG_DESKTOP_PORTAL_DIR \ - image/etc/s6-linux-init/env/WAYLAND_DISPLAY \ - image/etc/s6-linux-init/env/XDG_RUNTIME_DIR \ - image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ - image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/notification-fd \ - image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run \ - image/etc/s6-linux-init/scripts/rc.init \ - image/etc/s6-linux-init/scripts/rc.shutdown \ - image/etc/s6-linux-init/scripts/rc.shutdown.final \ - image/etc/wireplumber/wireplumber.conf.d/99_spectrum.conf \ - image/etc/xdg/xdg-desktop-portal/portals.conf - -VM_DIRS = dev run proc sys tmp \ +DIRS = dev run proc sys tmp \ etc/s6-linux-init/run-image/service \ etc/s6-linux-init/run-image/user \ etc/s6-linux-init/run-image/wait -VM_FIFOS = etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/fifo +FIFOS = etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/fifo -# These are separate because they need to be included, but putting -# them as make dependencies would confuse make. -VM_LINKS = image/bin image/etc/ssl/certs/ca-certificates.crt image/sbin - -VM_BUILD_FILES = build/etc/s6-rc +BUILD_FILES = build/etc/s6-rc build/fifo: mkdir -p build @@ -76,50 +45,21 @@ build/fifo: build/empty: mkdir -p $@ -build/rootfs.erofs: ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(VM_FILES) $(VM_BUILD_FILES) build/empty build/fifo +build/rootfs.erofs: ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty build/fifo file-list.mk { \ cat $(PACKAGES_FILE) ;\ - for file in $(VM_FILES) $(VM_LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ - for file in $(VM_BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ - printf 'build/empty\n%s\n' $(VM_DIRS) ;\ - printf 'build/fifo\n%s\n' $(VM_FIFOS) ;\ + for file in $(FILES) $(LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ + for file in $(BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ + printf 'build/empty\n%s\n' $(DIRS) ;\ + printf 'build/fifo\n%s\n' $(FIFOS) ;\ } | ../../scripts/make-erofs.sh $@ -VM_S6_RC_FILES = \ - image/etc/s6-rc/app/dependencies.d/dbus \ - image/etc/s6-rc/app/dependencies.d/pipewire \ - image/etc/s6-rc/app/dependencies.d/wayland-proxy-virtwl \ - image/etc/s6-rc/app/run \ - image/etc/s6-rc/app/type \ - image/etc/s6-rc/dbus/notification-fd \ - image/etc/s6-rc/dbus/run \ - image/etc/s6-rc/dbus/type \ - image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ - image/etc/s6-rc/mdevd-coldplug/type \ - image/etc/s6-rc/mdevd-coldplug/up \ - image/etc/s6-rc/mdevd/notification-fd \ - image/etc/s6-rc/mdevd/run \ - image/etc/s6-rc/mdevd/type \ - image/etc/s6-rc/ok-all/contents.d/app \ - image/etc/s6-rc/ok-all/contents.d/mdevd-coldplug \ - image/etc/s6-rc/ok-all/contents.d/wireplumber \ - image/etc/s6-rc/ok-all/type \ - image/etc/s6-rc/pipewire/notification-fd \ - image/etc/s6-rc/pipewire/run \ - image/etc/s6-rc/pipewire/type \ - image/etc/s6-rc/wayland-proxy-virtwl/notification-fd \ - image/etc/s6-rc/wayland-proxy-virtwl/run \ - image/etc/s6-rc/wayland-proxy-virtwl/type \ - image/etc/s6-rc/wireplumber/dependencies.d/dbus \ - image/etc/s6-rc/wireplumber/dependencies.d/pipewire \ - image/etc/s6-rc/wireplumber/run \ - image/etc/s6-rc/wireplumber/type - -build/etc/s6-rc: $(VM_S6_RC_FILES) + +build/etc/s6-rc: $(S6_RC_FILES) file-list.mk mkdir -p $$(dirname $@) rm -rf $@ set -uo pipefail && dir=$$(mktemp -d) && \ - { tar -c $(VM_S6_RC_FILES) | tar -C $$dir -x --strip-components 3; } && \ + { tar -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 3; } && \ s6-rc-compile $@ $$dir; \ exit=$$?; rm -r $$dir; exit $$exit diff --git a/img/app/file-list.mk b/img/app/file-list.mk new file mode 100644 index 0000000000000000000000000000000000000000..9967285ca9972b675713de36859850fed609f15f --- /dev/null +++ b/img/app/file-list.mk @@ -0,0 +1,65 @@ +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour + +FILES = \ + image/etc/dbus-1/session.conf \ + image/etc/fonts/fonts.conf \ + image/etc/fstab \ + image/etc/init \ + image/etc/mdev.conf \ + image/etc/mdev/iface \ + image/etc/mdev/listen \ + image/etc/mdev/virtiofs \ + image/etc/mdev/wait \ + image/etc/passwd \ + image/etc/pipewire/pipewire.conf \ + image/etc/resolv.conf \ + image/etc/s6-linux-init/env/DBUS_SESSION_BUS_ADDRESS \ + image/etc/s6-linux-init/env/DISPLAY \ + image/etc/s6-linux-init/env/GTK_USE_PORTAL \ + image/etc/s6-linux-init/env/NIX_XDG_DESKTOP_PORTAL_DIR \ + image/etc/s6-linux-init/env/WAYLAND_DISPLAY \ + image/etc/s6-linux-init/env/XDG_RUNTIME_DIR \ + image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ + image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/notification-fd \ + image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run \ + image/etc/s6-linux-init/scripts/rc.init \ + image/etc/s6-linux-init/scripts/rc.shutdown \ + image/etc/s6-linux-init/scripts/rc.shutdown.final \ + image/etc/wireplumber/wireplumber.conf.d/99_spectrum.conf \ + image/etc/xdg/xdg-desktop-portal/portals.conf + +LINKS = \ + image/bin \ + image/etc/ssl/certs/ca-certificates.crt \ + image/sbin + +S6_RC_FILES = \ + image/etc/s6-rc/app/dependencies.d/dbus \ + image/etc/s6-rc/app/dependencies.d/pipewire \ + image/etc/s6-rc/app/dependencies.d/wayland-proxy-virtwl \ + image/etc/s6-rc/app/run \ + image/etc/s6-rc/app/type \ + image/etc/s6-rc/dbus/notification-fd \ + image/etc/s6-rc/dbus/run \ + image/etc/s6-rc/dbus/type \ + image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ + image/etc/s6-rc/mdevd-coldplug/type \ + image/etc/s6-rc/mdevd-coldplug/up \ + image/etc/s6-rc/mdevd/notification-fd \ + image/etc/s6-rc/mdevd/run \ + image/etc/s6-rc/mdevd/type \ + image/etc/s6-rc/ok-all/contents.d/app \ + image/etc/s6-rc/ok-all/contents.d/mdevd-coldplug \ + image/etc/s6-rc/ok-all/contents.d/wireplumber \ + image/etc/s6-rc/ok-all/type \ + image/etc/s6-rc/pipewire/notification-fd \ + image/etc/s6-rc/pipewire/run \ + image/etc/s6-rc/pipewire/type \ + image/etc/s6-rc/wayland-proxy-virtwl/notification-fd \ + image/etc/s6-rc/wayland-proxy-virtwl/run \ + image/etc/s6-rc/wayland-proxy-virtwl/type \ + image/etc/s6-rc/wireplumber/dependencies.d/dbus \ + image/etc/s6-rc/wireplumber/dependencies.d/pipewire \ + image/etc/s6-rc/wireplumber/run \ + image/etc/s6-rc/wireplumber/type diff --git a/scripts/genfiles.awk b/scripts/genfiles.awk new file mode 100644 index 0000000000000000000000000000000000000000..2d32016b5e6d6773685b71cc04e49cf24e842ba9 --- /dev/null +++ b/scripts/genfiles.awk @@ -0,0 +1,85 @@ +# SPDX-License-Identifier: EUPL-1.2+ +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour +BEGIN { + RS = "\n"; + FS = "\t"; + modes["120000"] = "symlink"; + modes["100644"] = "regular"; + modes["100755"] = "regular"; +} + +function fail(msg) { + # Awk will run END blocks even after exit. + # The END block checks this variable and + # immediately exits if it is nonzero. + exit_code = 1; + print msg > "/dev/stderr"; + exit 1; +} + +# Extract data from built-in variables. +{ + filename = $2; + raw_mode = $1; + # Awk autocreates empty string entries if the key is invalid, + # but the code exits in this case so that is okay. + mode = modes[raw_mode]; +} + +filename !~ /^[[:alnum:]_./-]+$/ { + fail("filename '" filename "' has forbidden characters"); +} + +# Skip license files +filename ~ /\.license$/ { next } + +filename ~ /^image\/etc\/s6-rc\// { + if (mode != "regular") { + fail("s6-rc-compile input '" filename "' isn't a regular file"); + } + rc_files[rc_count++] = filename; + next; +} + +mode == "symlink" { + symlinks[symlink_count++] = filename; + next; +} + +mode == "regular" { + files[file_count++] = filename; + next; +} + +{ fail("File '" filename "' is not regular file or symlink (mode " raw_mode ")"); } + +END { + if (exit_code) { + exit exit_code; + } + printf ("# SPDX-" \ +"License-Identifier: CC0-1.0\n" \ +"# SPDX-" \ +"FileCopyrightText: 2025 Demi Marie Obenour \n" \ +"\n" \ +"FILES ="); + for (array_index = 0; array_index < file_count; array_index += 1) { + printf " \\\n\t%s", files[array_index]; + } + # GNU Make uses the modification time of the *target* of a symlink, + # rather than the modification time of the symlink itself. It can be told + # to *also* use the symlink's modification time, but not to *only* use + # the symlink's modification time. However, these symlinks will generally + # be broken, so make will not be able to dereference the symlink. + # Therefore, using these symlinks as make dependencies won't work. + printf ("\n\n" \ +"LINKS ="); + for (array_index = 0; array_index < symlink_count; array_index += 1) { + printf " \\\n\t%s", symlinks[array_index]; + } + printf "\n\nS6_RC_FILES ="; + for (array_index = 0; array_index < rc_count; array_index += 1) { + printf " \\\n\t%s", rc_files[array_index]; + } + print ""; +} diff --git a/scripts/genfiles.sh b/scripts/genfiles.sh new file mode 100755 index 0000000000000000000000000000000000000000..5bee72ec72bc3aeb5b2e1db26721664858661d35 --- /dev/null +++ b/scripts/genfiles.sh @@ -0,0 +1,26 @@ +#!/bin/sh -- +# SPDX-License-Identifier: EUPL-1.2+ +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour +set -euo pipefail +export LC_ALL=C LANGUAGE=C +dir=$(git rev-parse --show-toplevel) +for i in host/rootfs img/app vm/sys/net; do + output_file=$i/file-list.mk + { + git -C "$i" -c core.quotePath=true ls-files $'--format=%(objectmode)\t%(path)' -- image | + sort -t $'\t' -k 2 + } | + awk -f scripts/genfiles.awk > "$output_file.tmp" + if [ -f "$output_file" ]; then + # Avoid changing output file if it is up to date, as that + # would cause unnecessary rebuilds. + if cmp -s -- "$output_file.tmp" "$output_file"; then + rm -- "$output_file.tmp" + continue + else + astatus=$? + if [ "$astatus" != 1 ]; then exit "$astatus"; fi + fi + fi + mv -- "$output_file.tmp" "$output_file" +done diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile index 8a59faca2451ca3d404a62d76def15dec67add51..b377e12bba8f062026e997de18e19c9af8e07cb5 100644 --- a/vm/sys/net/Makefile +++ b/vm/sys/net/Makefile @@ -4,6 +4,7 @@ .POSIX: include ../../../lib/common.mk +include file-list.mk prefix = build/host libdir = $(prefix)/lib @@ -28,62 +29,26 @@ $(vmdir)/netvm/blk/root.img: ../../../scripts/make-gpt.sh ../../../scripts/sfdis build/rootfs.erofs:root:ea21da27-0391-48da-9235-9d2ab2ca7844:root mv $@.tmp $@ -VM_FILES = \ - image/etc/dbus-1/system.conf \ - image/etc/fstab \ - image/etc/init \ - image/etc/mdev.conf \ - image/etc/mdev/iface \ - image/etc/nftables.conf \ - image/etc/passwd \ - image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ - image/etc/s6-linux-init/scripts/rc.init \ - image/etc/sysctl.conf -VM_DIRS = dev etc/s6-linux-init/env proc run sys var/lib/connman - -# These are separate because they need to be included, but putting -# them as make dependencies would confuse make. -VM_LINKS = image/bin image/lib image/sbin image/var/run - -VM_BUILD_FILES = build/etc/s6-rc +DIRS = dev etc/s6-linux-init/env proc run sys var/lib/connman + +BUILD_FILES = build/etc/s6-rc build/empty: mkdir -p $@ -build/rootfs.erofs: ../../../scripts/make-erofs.sh $(PACKAGES_FILE) $(VM_FILES) $(VM_BUILD_FILES) build/empty +build/rootfs.erofs: ../../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty file-list.mk { \ cat $(PACKAGES_FILE) ;\ - for file in $(VM_FILES) $(VM_LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ - for file in $(VM_BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ - printf 'build/empty\n%s\n' $(VM_DIRS) ;\ + for file in $(FILES) $(LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ + for file in $(BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ + printf 'build/empty\n%s\n' $(DIRS) ;\ } | ../../../scripts/make-erofs.sh $@ -VM_S6_RC_FILES = \ - image/etc/s6-rc/connman/dependencies.d/dbus \ - image/etc/s6-rc/connman/run \ - image/etc/s6-rc/connman/type \ - image/etc/s6-rc/dbus/notification-fd \ - image/etc/s6-rc/dbus/run \ - image/etc/s6-rc/dbus/type \ - image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ - image/etc/s6-rc/mdevd-coldplug/type \ - image/etc/s6-rc/mdevd-coldplug/up \ - image/etc/s6-rc/mdevd/notification-fd \ - image/etc/s6-rc/mdevd/run \ - image/etc/s6-rc/mdevd/type \ - image/etc/s6-rc/nftables/type \ - image/etc/s6-rc/nftables/up \ - image/etc/s6-rc/ok-all/contents.d/mdevd-coldplug \ - image/etc/s6-rc/ok-all/contents.d/sysctl \ - image/etc/s6-rc/ok-all/type \ - image/etc/s6-rc/sysctl/type \ - image/etc/s6-rc/sysctl/up - -build/etc/s6-rc: $(VM_S6_RC_FILES) +build/etc/s6-rc: $(S6_RC_FILES) file-list.mk mkdir -p $$(dirname $@) rm -rf $@ set -uo pipefail && dir=$$(mktemp -d) && \ - { tar -c $(VM_S6_RC_FILES) | tar -C $$dir -x --strip-components 3; } && \ + { tar -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 3; } && \ s6-rc-compile $@ $$dir; \ exit=$$?; rm -r $$dir; exit $$exit diff --git a/vm/sys/net/file-list.mk b/vm/sys/net/file-list.mk new file mode 100644 index 0000000000000000000000000000000000000000..a6f1a41c9a30b769a508f6c486cced0c8daaa833 --- /dev/null +++ b/vm/sys/net/file-list.mk @@ -0,0 +1,41 @@ +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour + +FILES = \ + image/etc/dbus-1/system.conf \ + image/etc/fstab \ + image/etc/init \ + image/etc/mdev.conf \ + image/etc/mdev/iface \ + image/etc/nftables.conf \ + image/etc/passwd \ + image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ + image/etc/s6-linux-init/scripts/rc.init \ + image/etc/sysctl.conf + +LINKS = \ + image/bin \ + image/lib \ + image/sbin \ + image/var/run + +S6_RC_FILES = \ + image/etc/s6-rc/connman/dependencies.d/dbus \ + image/etc/s6-rc/connman/run \ + image/etc/s6-rc/connman/type \ + image/etc/s6-rc/dbus/notification-fd \ + image/etc/s6-rc/dbus/run \ + image/etc/s6-rc/dbus/type \ + image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ + image/etc/s6-rc/mdevd-coldplug/type \ + image/etc/s6-rc/mdevd-coldplug/up \ + image/etc/s6-rc/mdevd/notification-fd \ + image/etc/s6-rc/mdevd/run \ + image/etc/s6-rc/mdevd/type \ + image/etc/s6-rc/nftables/type \ + image/etc/s6-rc/nftables/up \ + image/etc/s6-rc/ok-all/contents.d/mdevd-coldplug \ + image/etc/s6-rc/ok-all/contents.d/sysctl \ + image/etc/s6-rc/ok-all/type \ + image/etc/s6-rc/sysctl/type \ + image/etc/s6-rc/sysctl/up --- base-commit: 6e0a84ec0380f9861bea12370160f0f20348c1dc change-id: 20250831-genfiles-46787fc10869 -- Sincerely, Demi Marie Obenour (she/her/hers)