From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id DE14765B0; Wed, 10 Sep 2025 05:37:28 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id EDB5665A9; Wed, 10 Sep 2025 05:37:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_PASS,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=4.0.1 Received: from mail-yw1-x1133.google.com (mail-yw1-x1133.google.com [IPv6:2607:f8b0:4864:20::1133]) by atuin.qyliss.net (Postfix) with ESMTPS id 5FAFB655B for ; Wed, 10 Sep 2025 05:37:25 +0000 (UTC) Received: by mail-yw1-x1133.google.com with SMTP id 00721157ae682-71d71bcab69so51641987b3.0 for ; Tue, 09 Sep 2025 22:37:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1757482644; x=1758087444; darn=spectrum-os.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=3yHfxKObyNi4EHQmiHslwbu8Gi5AQ96Waqq/navAlmM=; b=WkffKnBFIT58ugFSPlCZi3kI9bAkHN6MmDcbTXWCfqDo1mJuvEtqYdFmxHir3JGc8L PkIug5F3lcEqXwSAB4ehKi7QcUdjz+pHSWrdY3UKPOIsLWRYHgoiC54Gc+yhGfR88tHg 9fFRXKqb4mcqaaXDK08VokwbQZuZXrjedn9DkwbVcG0wjNC5P0iYJBGPIHsHrLIf13bj gzTISjODV5W3CQ/zqyZ/30b3ABBIU7k5BYoXZpnOzHB9/fymaj4bn92oU6FpoRQ0TauB RaxRZnn9XpqfrmPZxoyorgEaLE7y9JR1EGrfhRnJqVG7zdp2522HiPz5bQ/umNCXZWJe 1cEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1757482644; x=1758087444; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3yHfxKObyNi4EHQmiHslwbu8Gi5AQ96Waqq/navAlmM=; b=IVZW9lH/9pAIYtdpQ64DdfFFWKhuqnIMS4xgPKWyISZL5lLk0+X6a/wfZyIXS+WgoP aLoQS6LV8aMfHEZtLy3AFw65XiQ38ERb+ug0pwmKprs8XeaTDZU3ROHi3uioVnd1i4nJ ChVdjhGv/q9KM84L8QJ9weN+SACa0Zxg4lOETxXPxp16pMOdiWiHe5+XCWqVX6fNkkeT z1uB8qTKwgN+JcVkX3XAVignajXGLGg352U9WYMF3XBcZ55sN4yde3deXS2+stqlaTKE 1A3iVD5Axhhsfthiomnjbmxq4Rp5gnGUEkXCgghIAb54EEDzLWR+SsWkxXKEICB+V5KZ GIww== X-Gm-Message-State: AOJu0YxoT23sXly29RhfuoK1XwczUt8w0oE14Eme/hTL7K0ubNo5+MaF +/E0GEtFpsfjcPl4yePgxJ/1PjY3bWClwwdhpzQ/cEmD6d3t6cBWJhMbrMinZQ== X-Gm-Gg: ASbGnctlYu/PrYGJ8v3dOhhBG28oh4DbdDXh/aDOMkPK9j97F8TWi2scWJAuGa1aboD GSqipILMQsclW3H0IhgmgvVjV7+Xv+9W+MRZFtb5/pHJn+gLt3vDn9SHDuizz5G2jmTy3c8mw9k a+2tSNUJI5+6XxbbgWxE67x+tak+etZxuAHDZTs9jgUO5gqZEW/qr97kDas5yARE6H7BF2XJxS0 Vz/Mt2m4pdW0UN69Ns8/XiOMM3mRPCzFvLeZsYbEQAydovLBMwE3rXK/yWS0ew1ejIZtlXjqXXb ZRU7784GprC2zt1b9tl18TcA/jX2RfgeJRee0hRrNMFMK0vY3ryNed+d3rW2HcA35S54/MRv8Qu dEh/q+4l4U7wMniZKcxrvJRaZOKw5qbIudpU866uILMnh26+WyYW35OXzXjlNK34hjPP7aPXFj4 MF5DeBYCO8R7qTc4pXfnyNO20wut6MXp493CHjaaaPpyT54oSQg/PbrQ== X-Google-Smtp-Source: AGHT+IHXsSx7O06DxtjoElFl/FHTUQZfXrgb3h4N4AnKsSxH4i7TqFuJtEt6b6kQ/CjYxbkDEuTOyA== X-Received: by 2002:a05:690c:9a03:b0:71f:ff0c:c96a with SMTP id 00721157ae682-727f339af46mr130754607b3.24.1757482643599; Tue, 09 Sep 2025 22:37:23 -0700 (PDT) Received: from localhost.localdomain (h96-60-249-169.cncrtn.broadband.dynamic.tds.net. [96.60.249.169]) by smtp.gmail.com with UTF8SMTPSA id 00721157ae682-723a8356bcfsm67236477b3.29.2025.09.09.22.37.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Sep 2025 22:37:23 -0700 (PDT) From: Demi Marie Obenour Date: Wed, 10 Sep 2025 01:29:55 -0400 Subject: [PATCH v2 2/3] Generate makefile file lists from a script MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20250910-genfiles-v2-2-37ebe07a3cdc@gmail.com> References: <20250910-genfiles-v2-0-37ebe07a3cdc@gmail.com> In-Reply-To: <20250910-genfiles-v2-0-37ebe07a3cdc@gmail.com> To: Spectrum OS Development X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1757482194; l=32378; i=demiobenour@gmail.com; s=20250729; h=from:subject:message-id; bh=0wt8aGeRXVUR3TlKA/IjnQcpozT/UFYLvNps2NWHYvI=; b=F7Vdba7jeNmGZLodNrhS/61TgyUkISMTezHCkok1QSJYR1ZWjtOhxEMQrVF6wzvq+yj0oD3ro R+GTM4l0rwgBryrmmVnJN97gdrQHJ0P7ykQB9RiH+Z7wU74zzAxKXfD X-Developer-Key: i=demiobenour@gmail.com; a=ed25519; pk=X57Q4/YQDj9t4SBeKaDwvXYKB6quZJVx/DE2Ly2out0= Message-ID-Hash: SRXTSP56U7F3ONMPD6H5N6B346ABP5N2 X-Message-ID-Hash: SRXTSP56U7F3ONMPD6H5N6B346ABP5N2 X-MailFrom: demiobenour@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Demi Marie Obenour , Alyssa Ross X-Mailman-Version: 3.3.9 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: The script will always get them right, whereas humans (the author of this commit included) generally will not. Signed-off-by: Demi Marie Obenour --- Documentation/development/built-in-vms.adoc | 18 +++++ host/rootfs/Makefile | 102 +---------------------- host/rootfs/file-list.mk | 100 +++++++++++++++++++++++ img/app/Makefile | 87 ++++---------------- img/app/file-list.mk | 65 +++++++++++++++ lib/common.mk | 1 + scripts/genfiles.awk | 120 ++++++++++++++++++++++++++++ scripts/genfiles.sh | 26 ++++++ scripts/git-rebase | 17 ++++ scripts/pre-commit.sh | 11 +++ vm/sys/net/Makefile | 58 ++++---------- vm/sys/net/file-list.mk | 42 ++++++++++ 12 files changed, 434 insertions(+), 213 deletions(-) diff --git a/Documentation/development/built-in-vms.adoc b/Documentation/development/built-in-vms.adoc index e90009ee5a3c2c254a7ae11e36121576b819eee7..66b53f414c8d0101ff5dc9dff1c39e0f28ba9237 100644 --- a/Documentation/development/built-in-vms.adoc +++ b/Documentation/development/built-in-vms.adoc @@ -44,6 +44,24 @@ NOTE: As a special convenience, it's not necessary to run `make clean` if the only change to the Nix files is modifying the packages installed in the VM. +The list of files used for the VM image is stored in a separate file, +`file-lists.mk`. To update it, run `scripts/genfiles.sh` from the +same directory as the `file-lists.mk` file you want to update. +which will regenerate it from the output of `git ls-files`. Any +changes you made will be lost. This script uses uses Git's index to +generate the list, so you need to use `git add`, `git rm`, and `git mv` +to ensure that Git knows about your changes. It is not necessary to +commit the changes. + +It is recommended that you add a Git pre-commit hook that calls +`scripts/pre-commit.sh`. This updates the `file-list.mk` files +for you, so that each commit always includes the correct +`file-list.mk`. Sadly, Git does not run the pre-commit hook +during rebases, so if you rebase you should use the script +`scripts/git-rebase` instead of `git rebase`. It tells +Git to run `scripts/pre-commit.sh` after each commit and then +calls `git commit --amend` to fixup the commit if needed. + By default, `make run` runs the VM in Cloud Hypervisor, to match the Spectrum environment. You can also add `VMM=crosvm` or `VMM=qemu` to your command line to run the VM in a different VMM. One reason to do diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index 4872699d88107feab3e695c6be0b1503a6566520..795d0018ea2858c4d21a9ec42370597fd102473a 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -4,67 +4,10 @@ .POSIX: include ../../lib/common.mk +include file-list.mk dest = build/rootfs.erofs -FILES = \ - image/etc/fonts/fonts.conf \ - image/etc/fstab \ - image/etc/group \ - image/etc/init \ - image/etc/login \ - image/etc/mdev.conf \ - image/etc/mdev/listen \ - image/etc/mdev/net/add \ - image/etc/mdev/wait \ - image/etc/parse-devname \ - image/etc/passwd \ - image/etc/s6-linux-init/env/WAYLAND_DISPLAY \ - image/etc/s6-linux-init/env/XDG_RUNTIME_DIR \ - image/etc/s6-linux-init/run-image/service/dbus/notification-fd \ - image/etc/s6-linux-init/run-image/service/dbus/run \ - image/etc/s6-linux-init/run-image/service/dbus/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/dbus/template/run \ - image/etc/s6-linux-init/run-image/service/getty-tty1/run \ - image/etc/s6-linux-init/run-image/service/getty-tty2/run \ - image/etc/s6-linux-init/run-image/service/getty-tty3/run \ - image/etc/s6-linux-init/run-image/service/getty-tty4/run \ - image/etc/s6-linux-init/run-image/service/s6-svscan-log/notification-fd \ - image/etc/s6-linux-init/run-image/service/s6-svscan-log/run \ - image/etc/s6-linux-init/run-image/service/serial-getty-generator/run \ - image/etc/s6-linux-init/run-image/service/serial-getty/notification-fd \ - image/etc/s6-linux-init/run-image/service/serial-getty/run \ - image/etc/s6-linux-init/run-image/service/serial-getty/template/run \ - image/etc/s6-linux-init/run-image/service/vhost-user-fs/notification-fd \ - image/etc/s6-linux-init/run-image/service/vhost-user-fs/run \ - image/etc/s6-linux-init/run-image/service/vhost-user-fs/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/vhost-user-fs/template/run \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/notification-fd \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/run \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/data/check \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/run \ - image/etc/s6-linux-init/run-image/service/vmm/notification-fd \ - image/etc/s6-linux-init/run-image/service/vmm/run \ - image/etc/s6-linux-init/run-image/service/vmm/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/notification-fd \ - image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/run \ - image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/template/notification-fd \ - image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/template/run \ - image/etc/s6-linux-init/scripts/rc.init \ - image/etc/xdg/weston/autolaunch \ - image/etc/xdg/weston/weston.ini \ - image/usr/share/dbus-1/services/org.freedesktop.portal.Documents.service \ - image/usr/bin/assign-devices \ - image/usr/bin/create-vm-dependencies \ - image/usr/bin/run-appimage \ - image/usr/bin/run-vmm \ - image/usr/bin/vm-console \ - image/usr/bin/vm-import \ - image/usr/bin/vm-start \ - image/usr/bin/vm-stop \ - image/usr/bin/xdg-open - DIRS = \ dev \ etc/s6-linux-init/env \ @@ -101,18 +44,9 @@ DIRS = \ FIFOS = etc/s6-linux-init/run-image/service/s6-svscan-log/fifo -# These are separate because they need to be included, but putting -# them as make dependencies would confuse make. -LINKS = \ - image/bin \ - image/etc/s6-linux-init/run-image/opengl-driver \ - image/etc/s6-linux-init/run-image/service/vmm/template/run \ - image/lib \ - image/sbin - BUILD_FILES = build/etc/s6-rc -$(dest): ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty build/fifo +$(dest): ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty build/fifo file-list.mk ( \ cat $(PACKAGES_FILE) ;\ for file in $(FILES) $(LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ @@ -128,36 +62,6 @@ build/fifo: build/empty: mkdir -p $@ -S6_RC_FILES = \ - image/etc/s6-rc/card0/type \ - image/etc/s6-rc/card0/up \ - image/etc/s6-rc/core/type \ - image/etc/s6-rc/core/up \ - image/etc/s6-rc/kvm/timeout-up \ - image/etc/s6-rc/kvm/type \ - image/etc/s6-rc/kvm/up \ - image/etc/s6-rc/mdevd-coldplug/dependencies \ - image/etc/s6-rc/mdevd-coldplug/type \ - image/etc/s6-rc/mdevd-coldplug/up \ - image/etc/s6-rc/mdevd/notification-fd \ - image/etc/s6-rc/mdevd/run \ - image/etc/s6-rc/mdevd/type \ - image/etc/s6-rc/ok-all/contents \ - image/etc/s6-rc/ok-all/type \ - image/etc/s6-rc/static-nodes/type \ - image/etc/s6-rc/static-nodes/up \ - image/etc/s6-rc/sys-vmms/dependencies \ - image/etc/s6-rc/sys-vmms/type \ - image/etc/s6-rc/sys-vmms/up \ - image/etc/s6-rc/vm-env/contents \ - image/etc/s6-rc/vm-env/type \ - image/etc/s6-rc/vmm-env/contents \ - image/etc/s6-rc/vmm-env/type \ - image/etc/s6-rc/weston/dependencies \ - image/etc/s6-rc/weston/notification-fd \ - image/etc/s6-rc/weston/type \ - image/etc/s6-rc/weston/run - # s6-rc-compile's input is a directory, but that doesn't play nice # with Make, because it won't know to update if some file in the # directory is changed, or a file is created or removed in a @@ -165,7 +69,7 @@ S6_RC_FILES = \ # including files that aren't intended to be part of the input, like # temporary editor files or .license files. So for all these reasons, # only explicitly listed files are made available to s6-rc-compile. -build/etc/s6-rc: $(S6_RC_FILES) +build/etc/s6-rc: $(S6_RC_FILES) file-list.mk mkdir -p $$(dirname $@) rm -rf $@ diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk new file mode 100644 index 0000000000000000000000000000000000000000..8e5385752d5927d7c67fe4e91812f44e8c9ec033 --- /dev/null +++ b/host/rootfs/file-list.mk @@ -0,0 +1,100 @@ +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour +# Generated by scripts/genfile.sh. Any changes will be overwritten. +FILES := \ + image/etc/fonts/fonts.conf \ + image/etc/fstab \ + image/etc/group \ + image/etc/init \ + image/etc/login \ + image/etc/mdev.conf \ + image/etc/mdev/listen \ + image/etc/mdev/net/add \ + image/etc/mdev/wait \ + image/etc/parse-devname \ + image/etc/passwd \ + image/etc/s6-linux-init/env/WAYLAND_DISPLAY \ + image/etc/s6-linux-init/env/XDG_RUNTIME_DIR \ + image/etc/s6-linux-init/run-image/service/dbus/notification-fd \ + image/etc/s6-linux-init/run-image/service/dbus/run \ + image/etc/s6-linux-init/run-image/service/dbus/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/dbus/template/run \ + image/etc/s6-linux-init/run-image/service/getty-tty1/run \ + image/etc/s6-linux-init/run-image/service/getty-tty2/run \ + image/etc/s6-linux-init/run-image/service/getty-tty3/run \ + image/etc/s6-linux-init/run-image/service/getty-tty4/run \ + image/etc/s6-linux-init/run-image/service/s6-svscan-log/notification-fd \ + image/etc/s6-linux-init/run-image/service/s6-svscan-log/run \ + image/etc/s6-linux-init/run-image/service/serial-getty-generator/run \ + image/etc/s6-linux-init/run-image/service/serial-getty/notification-fd \ + image/etc/s6-linux-init/run-image/service/serial-getty/run \ + image/etc/s6-linux-init/run-image/service/serial-getty/template/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-fs/notification-fd \ + image/etc/s6-linux-init/run-image/service/vhost-user-fs/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-fs/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/vhost-user-fs/template/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/notification-fd \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/data/check \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/run \ + image/etc/s6-linux-init/run-image/service/vhost-user-gpu/template/type \ + image/etc/s6-linux-init/run-image/service/vmm/notification-fd \ + image/etc/s6-linux-init/run-image/service/vmm/run \ + image/etc/s6-linux-init/run-image/service/vmm/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/notification-fd \ + image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/run \ + image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/template/notification-fd \ + image/etc/s6-linux-init/run-image/service/xdg-desktop-portal-spectrum-host/template/run \ + image/etc/s6-linux-init/scripts/rc.init \ + image/etc/xdg/weston/autolaunch \ + image/etc/xdg/weston/weston.ini \ + image/usr/bin/assign-devices \ + image/usr/bin/create-vm-dependencies \ + image/usr/bin/run-appimage \ + image/usr/bin/run-vmm \ + image/usr/bin/vm-console \ + image/usr/bin/vm-import \ + image/usr/bin/vm-start \ + image/usr/bin/vm-stop \ + image/usr/bin/xdg-open \ + image/usr/share/dbus-1/services/org.freedesktop.portal.Documents.service + +# These are separate because they need to be included, but putting +# them as make dependencies would confuse make. +LINKS := \ + image/bin \ + image/etc/s6-linux-init/run-image/opengl-driver \ + image/etc/s6-linux-init/run-image/service/vmm/template/run \ + image/lib \ + image/sbin + +S6_RC_FILES := \ + image/etc/s6-rc/card0/type \ + image/etc/s6-rc/card0/up \ + image/etc/s6-rc/core/type \ + image/etc/s6-rc/core/up \ + image/etc/s6-rc/kvm/timeout-up \ + image/etc/s6-rc/kvm/type \ + image/etc/s6-rc/kvm/up \ + image/etc/s6-rc/mdevd-coldplug/dependencies \ + image/etc/s6-rc/mdevd-coldplug/type \ + image/etc/s6-rc/mdevd-coldplug/up \ + image/etc/s6-rc/mdevd/notification-fd \ + image/etc/s6-rc/mdevd/run \ + image/etc/s6-rc/mdevd/type \ + image/etc/s6-rc/ok-all/contents \ + image/etc/s6-rc/ok-all/type \ + image/etc/s6-rc/static-nodes/type \ + image/etc/s6-rc/static-nodes/up \ + image/etc/s6-rc/sys-vmms/dependencies \ + image/etc/s6-rc/sys-vmms/type \ + image/etc/s6-rc/sys-vmms/up \ + image/etc/s6-rc/vm-env/contents \ + image/etc/s6-rc/vm-env/type \ + image/etc/s6-rc/vmm-env/contents \ + image/etc/s6-rc/vmm-env/type \ + image/etc/s6-rc/weston/dependencies \ + image/etc/s6-rc/weston/notification-fd \ + image/etc/s6-rc/weston/run \ + image/etc/s6-rc/weston/type diff --git a/img/app/Makefile b/img/app/Makefile index cc209710cb3d521110ce560ce956cdab228db178..04fbf99c7d2e30527aa54e5e549043d03ce6362b 100644 --- a/img/app/Makefile +++ b/img/app/Makefile @@ -5,6 +5,7 @@ .POSIX: include ../../lib/common.mk +include file-list.mk prefix = build/host libdir = $(prefix)/lib @@ -29,45 +30,13 @@ $(imgdir)/appvm/blk/root.img: ../../scripts/make-gpt.sh ../../scripts/sfdisk-fie build/rootfs.erofs:root:5460386f-2203-4911-8694-91400125c604:root mv $@.tmp $@ -VM_FILES = \ - image/etc/dbus-1/session.conf \ - image/etc/fonts/fonts.conf \ - image/etc/fstab \ - image/etc/init \ - image/etc/mdev.conf \ - image/etc/mdev/iface \ - image/etc/mdev/listen \ - image/etc/mdev/virtiofs \ - image/etc/mdev/wait \ - image/etc/passwd \ - image/etc/pipewire/pipewire.conf \ - image/etc/resolv.conf \ - image/etc/s6-linux-init/env/DBUS_SESSION_BUS_ADDRESS \ - image/etc/s6-linux-init/env/DISPLAY \ - image/etc/s6-linux-init/env/GTK_USE_PORTAL \ - image/etc/s6-linux-init/env/NIX_XDG_DESKTOP_PORTAL_DIR \ - image/etc/s6-linux-init/env/WAYLAND_DISPLAY \ - image/etc/s6-linux-init/env/XDG_RUNTIME_DIR \ - image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ - image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/notification-fd \ - image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run \ - image/etc/s6-linux-init/scripts/rc.init \ - image/etc/s6-linux-init/scripts/rc.shutdown \ - image/etc/s6-linux-init/scripts/rc.shutdown.final \ - image/etc/wireplumber/wireplumber.conf.d/99_spectrum.conf \ - image/etc/xdg/xdg-desktop-portal/portals.conf - -VM_DIRS = dev run proc sys tmp \ +DIRS = dev run proc sys tmp \ etc/s6-linux-init/run-image/service \ etc/s6-linux-init/run-image/user \ etc/s6-linux-init/run-image/wait -VM_FIFOS = etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/fifo +FIFOS = etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/fifo -# These are separate because they need to be included, but putting -# them as make dependencies would confuse make. -VM_LINKS = image/bin image/etc/ssl/certs/ca-certificates.crt image/sbin - -VM_BUILD_FILES = build/etc/s6-rc +BUILD_FILES = build/etc/s6-rc build/fifo: mkdir -p build @@ -76,49 +45,22 @@ build/fifo: build/empty: mkdir -p $@ -build/rootfs.erofs: ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(VM_FILES) $(VM_BUILD_FILES) build/empty build/fifo +build/rootfs.erofs: ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(addprefix image/,$(FILES)) $(BUILD_FILES) build/empty build/fifo file-list.mk ( \ cat $(PACKAGES_FILE) ;\ - for file in $(VM_FILES) $(VM_LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ - for file in $(VM_BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ - printf 'build/empty\n%s\n' $(VM_DIRS) ;\ - printf 'build/fifo\n%s\n' $(VM_FIFOS) ;\ + for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' $$file $$file; done ;\ + for file in $(BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ + printf 'build/empty\n%s\n' $(DIRS) ;\ + printf 'build/fifo\n%s\n' $(FIFOS) ;\ ) | ../../scripts/make-erofs.sh $@ -VM_S6_RC_FILES = \ - image/etc/s6-rc/app/dependencies.d/dbus \ - image/etc/s6-rc/app/dependencies.d/pipewire \ - image/etc/s6-rc/app/dependencies.d/wayland-proxy-virtwl \ - image/etc/s6-rc/app/run \ - image/etc/s6-rc/app/type \ - image/etc/s6-rc/dbus/notification-fd \ - image/etc/s6-rc/dbus/run \ - image/etc/s6-rc/dbus/type \ - image/etc/s6-rc/mdevd-coldplug/dependencies \ - image/etc/s6-rc/mdevd-coldplug/type \ - image/etc/s6-rc/mdevd-coldplug/up \ - image/etc/s6-rc/mdevd/notification-fd \ - image/etc/s6-rc/mdevd/run \ - image/etc/s6-rc/mdevd/type \ - image/etc/s6-rc/ok-all/contents \ - image/etc/s6-rc/ok-all/type \ - image/etc/s6-rc/pipewire/notification-fd \ - image/etc/s6-rc/pipewire/run \ - image/etc/s6-rc/pipewire/type \ - image/etc/s6-rc/wayland-proxy-virtwl/notification-fd \ - image/etc/s6-rc/wayland-proxy-virtwl/run \ - image/etc/s6-rc/wayland-proxy-virtwl/type \ - image/etc/s6-rc/wireplumber/dependencies.d/dbus \ - image/etc/s6-rc/wireplumber/dependencies.d/pipewire \ - image/etc/s6-rc/wireplumber/run \ - image/etc/s6-rc/wireplumber/type - -build/etc/s6-rc: $(VM_S6_RC_FILES) + +build/etc/s6-rc: $(S6_RC_FILES) file-list.mk mkdir -p $$(dirname $@) rm -rf $@ dir=$$(mktemp -d) && \ - tar -C image -c $(VM_S6_RC_FILES) | tar -C $$dir -x --strip-components 3 && \ + tar -C image -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 3 && \ s6-rc-compile $@ $$dir; \ exit=$$?; rm -r $$dir; exit $$exit @@ -197,6 +139,11 @@ run-crosvm: $(imgdir)/appvm/blk/root.img start-vhost-user-gpu start-virtiofsd $(KERNEL) .PHONY: run-crosvm +update-file-list: + ../../scripts/genfiles.awk image > file-list.mk + +.PHONY: update-file-list + run: run-$(VMM) .PHONY: run diff --git a/img/app/file-list.mk b/img/app/file-list.mk new file mode 100644 index 0000000000000000000000000000000000000000..2a249c27af3c5c7ab5158e3acf52400878ac6b4d --- /dev/null +++ b/img/app/file-list.mk @@ -0,0 +1,65 @@ +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour +# Generated by scripts/genfile.sh. Any changes will be overwritten. +FILES := \ + image/etc/dbus-1/session.conf \ + image/etc/fonts/fonts.conf \ + image/etc/fstab \ + image/etc/init \ + image/etc/mdev.conf \ + image/etc/mdev/iface \ + image/etc/mdev/listen \ + image/etc/mdev/virtiofs \ + image/etc/mdev/wait \ + image/etc/passwd \ + image/etc/pipewire/pipewire.conf \ + image/etc/resolv.conf \ + image/etc/s6-linux-init/env/DBUS_SESSION_BUS_ADDRESS \ + image/etc/s6-linux-init/env/DISPLAY \ + image/etc/s6-linux-init/env/GTK_USE_PORTAL \ + image/etc/s6-linux-init/env/NIX_XDG_DESKTOP_PORTAL_DIR \ + image/etc/s6-linux-init/env/WAYLAND_DISPLAY \ + image/etc/s6-linux-init/env/XDG_RUNTIME_DIR \ + image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ + image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/notification-fd \ + image/etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/run \ + image/etc/s6-linux-init/scripts/rc.init \ + image/etc/s6-linux-init/scripts/rc.shutdown \ + image/etc/s6-linux-init/scripts/rc.shutdown.final \ + image/etc/wireplumber/wireplumber.conf.d/99_spectrum.conf \ + image/etc/xdg/xdg-desktop-portal/portals.conf + +# These are separate because they need to be included, but putting +# them as make dependencies would confuse make. +LINKS := \ + image/bin \ + image/etc/ssl/certs/ca-certificates.crt \ + image/sbin + +S6_RC_FILES := \ + image/etc/s6-rc/app/dependencies.d/dbus \ + image/etc/s6-rc/app/dependencies.d/pipewire \ + image/etc/s6-rc/app/dependencies.d/wayland-proxy-virtwl \ + image/etc/s6-rc/app/run \ + image/etc/s6-rc/app/type \ + image/etc/s6-rc/dbus/notification-fd \ + image/etc/s6-rc/dbus/run \ + image/etc/s6-rc/dbus/type \ + image/etc/s6-rc/mdevd-coldplug/dependencies \ + image/etc/s6-rc/mdevd-coldplug/type \ + image/etc/s6-rc/mdevd-coldplug/up \ + image/etc/s6-rc/mdevd/notification-fd \ + image/etc/s6-rc/mdevd/run \ + image/etc/s6-rc/mdevd/type \ + image/etc/s6-rc/ok-all/contents \ + image/etc/s6-rc/ok-all/type \ + image/etc/s6-rc/pipewire/notification-fd \ + image/etc/s6-rc/pipewire/run \ + image/etc/s6-rc/pipewire/type \ + image/etc/s6-rc/wayland-proxy-virtwl/notification-fd \ + image/etc/s6-rc/wayland-proxy-virtwl/run \ + image/etc/s6-rc/wayland-proxy-virtwl/type \ + image/etc/s6-rc/wireplumber/dependencies.d/dbus \ + image/etc/s6-rc/wireplumber/dependencies.d/pipewire \ + image/etc/s6-rc/wireplumber/run \ + image/etc/s6-rc/wireplumber/type diff --git a/lib/common.mk b/lib/common.mk index 277c3544036d9a9057f8ba4ad37fe2207548cc59..0a03ff440cc671264d2b859a2ae048db9252d047 100644 --- a/lib/common.mk +++ b/lib/common.mk @@ -1,5 +1,6 @@ # SPDX-License-Identifier: EUPL-1.2+ # SPDX-FileCopyrightText: 2021, 2023, 2025 Alyssa Ross +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour BACKGROUND = background CPIO = cpio diff --git a/scripts/genfiles.awk b/scripts/genfiles.awk new file mode 100644 index 0000000000000000000000000000000000000000..88a2824c671dbf1fabca184a9e29c28d564d1cee --- /dev/null +++ b/scripts/genfiles.awk @@ -0,0 +1,120 @@ +#!/usr/bin/env -S LC_ALL=C LANGUAGE=C awk -E +# SPDX-License-Identifier: EUPL-1.2+ +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour +BEGIN { + RS = "\n"; + FS = "\t"; + file_count = 0; + symlink_count = 0; + rc_count = 0; + is_rc = 0; + exit_code = 0; + done = 0; + modes["120000"] = "symlink"; + modes["040755"] = "directory"; + modes["100644"] = "regular"; + modes["100755"] = "regular"; +} + +function fail(msg, status) { + if (status ~ /^([1-9][0-9]?|1[0-9]{2}|2[0-4][1-9]|25[1-5])$/) { + exit_code = status; + } else { + exit_code = 1; + status = 1; + } + print ("FATAL: " msg) > "/dev/stderr"; + exit status; +} +done { fail("Junk after DONE", 1); } +/^DONE$/ { + done = 1 + next +} + +# Make sure git produced valid output. +!/^[0-7]{6}\t[ -~]+$/ { + fail("git ls-files produced invalid output", 1); +} + +# Extract data from built-in variables. +{ + filename = $2; + raw_mode = $1; + # awk autocreates empty string entries if the key is invalid, + # but the code exits in this case so that is okay. + mode = modes[raw_mode]; +} + +# Another check for a git bug. +filename ~ /^\/|((^|\/)\.{0,2}($|\/))/ { + fail("git ls-files output non-canonical or absolute path '" filename "'", 1); +} + +filename ~ /[^[:alnum:]_.+@/-]/ { + fail("filename '" filename "' has forbidden characters", 1); +} + +/\.license$/ { + if (raw_mode != "100644") { + fail("License file '" filename "' is executable or not regular file", 1); + } + next; +} + +mode == "directory" { next } + +filename ~ /^image\/etc\/s6-rc\// { + if (mode != "regular") { + fail("s6-rc-compile input '" filename "' isn't a regular file"); + } + rc_count += 1; + rc_files[rc_count] = filename; + next; +} + +mode == "symlink" { + symlink_count += 1; + symlinks[symlink_count] = filename; + next; +} + +mode == "regular" { + file_count += 1; + files[file_count] = filename; + next; +} + +{ fail("File '" filename "' is not regular file, directory, or symlink (mode " raw_mode ")"); } + +END { + if (exit_code) { + exit exit_code; + } + if (!done) { + fail("Did not receive DONE line", 1); + } + printf ("# SPDX-License-Identifier: CC0-1.0\n" \ + "# SPDX-FileCopyrightText: 2025 Demi Marie Obenour \n" \ + "# Generated by scripts/genfile.sh. Any changes will be overwritten.\n" \ + "FILES :=") > out_file; + for (array_index = 1; array_index <= file_count; array_index += 1) { + printf " \\\n\t%s", files[array_index] > out_file; + } + printf ("\n\n" \ +"# These are separate because they need to be included, but putting\n" \ +"# them as make dependencies would confuse make.\n" \ +"LINKS :=") > out_file; + for (array_index = 1; array_index <= symlink_count; array_index += 1) { + printf " \\\n\t%s", symlinks[array_index] > out_file; + } + printf "\n\nS6_RC_FILES :=" > out_file; + for (array_index = 1; array_index <= rc_count; array_index += 1) { + printf " \\\n\t%s", rc_files[array_index] > out_file; + } + printf "\n" > out_file; + if (close(out_file)) { + print ("Cannot close output file: " ERRNO "\n") > "/dev/stderr"; + exit 1; + } +} diff --git a/scripts/genfiles.sh b/scripts/genfiles.sh new file mode 100755 index 0000000000000000000000000000000000000000..cdd24c122e6223bc162612ffc86b1bbaa5503dfa --- /dev/null +++ b/scripts/genfiles.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env -S LC_ALL=C LANGUAGE=C bash -- +set -euo pipefail +script_file=genfiles.awk +case $0 in +(/*) script_file=${0%/*}/$script_file;; +(*/*) script_file=./${0%/*}/$script_file;; +(*) script_file=./$script_file;; +esac +case ${1-.} in +('') echo 'Directory name cannot be empty' >&2; exit 1;; +(*/) output_file=${1}file-list.mk;; +(*) output_file=${1-.}/file-list.mk;; +esac +{ + git ${1+-C} ${1+"$1"} -c core.quotePath=true ls-files $'--format=%(objectmode)\t%(path)' -- image | + sort -t $'\t' -k 2 + echo DONE +} | +gawk -v "out_file=$output_file.tmp" -E "$script_file" +status=0 +cmp -s -- "$output_file.tmp" "$output_file" || status=$? +case $status in +(0) rm -- "$output_file.tmp";; +(1) mv -- "$output_file.tmp" "$output_file";; +(*) exit "$?";; +esac diff --git a/scripts/git-rebase b/scripts/git-rebase new file mode 100755 index 0000000000000000000000000000000000000000..593f9299ddfd2c553ceacc68e738629ca8dec019 --- /dev/null +++ b/scripts/git-rebase @@ -0,0 +1,17 @@ +#!/usr/bin/env -S bash -- +set -eu +case $0 in +(/*) dir=${0%/*}/..;; +(*/*) dir=./${0%/*}/..;; +(*) dir=..;; +esac +exec git rebase -x "set -eu; \ +cd '${dir//\'/\'\\\'\'}'; \ +scripts/pre-commit.sh; \ +set +e;\ +git diff --quiet HEAD -- host/rootfs/gen/file-list.mk img/app/file-list.mk vm/sys/net/file-list.mk;\ +case \$? in \ +(0) :;;\ +(1) git commit --amend -- host/rootfs/gen/file-list.mk img/app/file-list.mk vm/sys/net/file-list.mk;;\ +(*) exit \"\$?\";;\ +esac" "$@" diff --git a/scripts/pre-commit.sh b/scripts/pre-commit.sh new file mode 100755 index 0000000000000000000000000000000000000000..517ea0c1f51abbbdf4553c3dcc65a6dbd5531b91 --- /dev/null +++ b/scripts/pre-commit.sh @@ -0,0 +1,11 @@ +#!/bin/sh -- +set -eu +case $0 in +(/*) cd ${0%/*}/..;; +(*/*) cd ./${0%/*}/..;; +(*) cd ..;; +esac +for i in host/rootfs img/app vm/sys/net; do + scripts/genfiles.sh "$i" + git add -- "$i/file-list.mk" +done diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile index 7e6da494db83e99042e2b50e72af2e9cd4e1190c..11f9215b58a89569ead0c5cc7a4a24b0cc4eec22 100644 --- a/vm/sys/net/Makefile +++ b/vm/sys/net/Makefile @@ -4,6 +4,7 @@ .POSIX: include ../../../lib/common.mk +include file-list.mk prefix = build/host libdir = $(prefix)/lib @@ -28,62 +29,26 @@ $(vmdir)/netvm/blk/root.img: ../../../scripts/make-gpt.sh ../../../scripts/sfdis build/rootfs.erofs:root:ea21da27-0391-48da-9235-9d2ab2ca7844:root mv $@.tmp $@ -VM_FILES = \ - image/etc/dbus-1/system.conf \ - image/etc/fstab \ - image/etc/init \ - image/etc/mdev.conf \ - image/etc/mdev/iface \ - image/etc/nftables.conf \ - image/etc/passwd \ - image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ - image/etc/s6-linux-init/scripts/rc.init \ - image/etc/sysctl.conf -VM_DIRS = dev etc/s6-linux-init/env run proc sys var/lib/connman - -# These are separate because they need to be included, but putting -# them as make dependencies would confuse make. -VM_LINKS = image/bin image/lib image/sbin image/var/run - -VM_BUILD_FILES = build/etc/s6-rc +BUILD_FILES = build/etc/s6-rc build/empty: mkdir -p $@ -build/rootfs.erofs: ../../../scripts/make-erofs.sh $(PACKAGES_FILE) $(VM_FILES) $(VM_BUILD_FILES) build/empty +build/rootfs.erofs: ../../../scripts/make-erofs.sh $(PACKAGES_FILE) $(FILES) $(BUILD_FILES) build/empty file-list.mk ( \ cat $(PACKAGES_FILE) ;\ - for file in $(VM_FILES) $(VM_LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ - for file in $(VM_BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ - printf 'build/empty\n%s\n' $(VM_DIRS) ;\ + for file in $(FILES) $(LINKS); do printf '%s\n%s\n' $$file "$${file#image/}"; done ;\ + for file in $(BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ + printf 'build/empty\n%s\n' $(DIRS) ;\ ) | ../../../scripts/make-erofs.sh $@ -VM_S6_RC_FILES = \ - image/etc/s6-rc/connman/dependencies \ - image/etc/s6-rc/connman/run \ - image/etc/s6-rc/connman/type \ - image/etc/s6-rc/dbus/notification-fd \ - image/etc/s6-rc/dbus/run \ - image/etc/s6-rc/dbus/type \ - image/etc/s6-rc/mdevd-coldplug/dependencies \ - image/etc/s6-rc/mdevd-coldplug/type \ - image/etc/s6-rc/mdevd-coldplug/up \ - image/etc/s6-rc/mdevd/notification-fd \ - image/etc/s6-rc/mdevd/run \ - image/etc/s6-rc/mdevd/type \ - image/etc/s6-rc/nftables/type \ - image/etc/s6-rc/nftables/up \ - image/etc/s6-rc/ok-all/contents \ - image/etc/s6-rc/ok-all/type \ - image/etc/s6-rc/sysctl/type \ - image/etc/s6-rc/sysctl/up - -build/etc/s6-rc: $(VM_S6_RC_FILES) + +build/etc/s6-rc: $(S6_RC_FILES) file-list.mk mkdir -p $$(dirname $@) rm -rf $@ dir=$$(mktemp -d) && \ - tar -c $(VM_S6_RC_FILES) | tar -C $$dir -x --strip-components 3 && \ + tar -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 3 && \ s6-rc-compile $@ $$dir; \ exit=$$?; rm -r $$dir; exit $$exit @@ -138,6 +103,11 @@ run-crosvm: $(vmdir)/netvm/blk/root.img $(KERNEL) .PHONY: run-crosvm +update-file-list: + ../../scripts/genfiles.awk image > file-list.mk + +.PHONY: update-file-list + run: run-$(VMM) .PHONY: run diff --git a/vm/sys/net/file-list.mk b/vm/sys/net/file-list.mk new file mode 100644 index 0000000000000000000000000000000000000000..a8df7e1e0d16b6854bfe716b1134947241916119 --- /dev/null +++ b/vm/sys/net/file-list.mk @@ -0,0 +1,42 @@ +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour +# Generated by scripts/genfile.sh. Any changes will be overwritten. +FILES := \ + image/etc/dbus-1/system.conf \ + image/etc/fstab \ + image/etc/init \ + image/etc/mdev.conf \ + image/etc/mdev/iface \ + image/etc/nftables.conf \ + image/etc/passwd \ + image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ + image/etc/s6-linux-init/scripts/rc.init \ + image/etc/sysctl.conf + +# These are separate because they need to be included, but putting +# them as make dependencies would confuse make. +LINKS := \ + image/bin \ + image/lib \ + image/sbin \ + image/var/run + +S6_RC_FILES := \ + image/etc/s6-rc/connman/dependencies \ + image/etc/s6-rc/connman/run \ + image/etc/s6-rc/connman/type \ + image/etc/s6-rc/dbus/notification-fd \ + image/etc/s6-rc/dbus/run \ + image/etc/s6-rc/dbus/type \ + image/etc/s6-rc/mdevd-coldplug/dependencies \ + image/etc/s6-rc/mdevd-coldplug/type \ + image/etc/s6-rc/mdevd-coldplug/up \ + image/etc/s6-rc/mdevd/notification-fd \ + image/etc/s6-rc/mdevd/run \ + image/etc/s6-rc/mdevd/type \ + image/etc/s6-rc/nftables/type \ + image/etc/s6-rc/nftables/up \ + image/etc/s6-rc/ok-all/contents \ + image/etc/s6-rc/ok-all/type \ + image/etc/s6-rc/sysctl/type \ + image/etc/s6-rc/sysctl/up -- 2.51.0