From: Demi Marie Obenour <demiobenour@gmail.com>
To: Spectrum OS Development <devel@spectrum-os.org>
Cc: Demi Marie Obenour <demiobenour@gmail.com>, Alyssa Ross <hi@alyssa.is>
Subject: [PATCH v2 0/3] Generate file lists from a script
Date: Wed, 10 Sep 2025 01:29:53 -0400 [thread overview]
Message-ID: <20250910-genfiles-v2-0-37ebe07a3cdc@gmail.com> (raw)
In-Reply-To: <20250903-genfiles-v1-0-cc993fcb1e4c@gmail.com>
Right now, the makefiles in host/rootfs, vm/sys/net, and img/app have
manually-maintained lists of files and symlinks. These duplicate the
information in the git repository and can easily get out of sync or
cause unnecessary merge conflicts.
Avoid this problem by moving the files used in the erofs images to an
image/ directory and using what Git knows about this directory to
generate the lists via a script. This makes maintenance easier and
avoids conflating code used in the build (anything not in image/) and
code for the image (anything that is in image/).
This reveals that there is a lot of duplicated make logic around image
generation, so consolidate it in a single makefile that is included by
all three of the makefiles.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Changes in v2:
- Drop the last patch (switching to /etc/s6-rc/compiled) as it is
controversial and should be reviewed separately.
- Add missing copyright notices.
- Use a wrapper shell script to make the awk code easier to read.
- Improve documentation.
- Add helper scripts for use in git hooks and rebasing.
- Link to v1: https://spectrum-os.org/lists/archives/spectrum-devel/20250903-genfiles-v1-0-cc993fcb1e4c@gmail.com/
---
Demi Marie Obenour (3):
Move all files for the image into a subdirectory
Generate makefile file lists from a script
Common make rules for building erofs images
Documentation/development/built-in-vms.adoc | 18 +++
host/rootfs/Makefile | 141 +--------------------
host/rootfs/default.nix | 1 +
host/rootfs/file-list.mk | 100 +++++++++++++++
host/rootfs/{ => image}/bin | 0
host/rootfs/{ => image}/etc/fonts/fonts.conf | 0
host/rootfs/{ => image}/etc/fstab | 0
host/rootfs/{ => image}/etc/group | 0
host/rootfs/{ => image}/etc/group.license | 0
host/rootfs/{ => image}/etc/init | 0
host/rootfs/{ => image}/etc/login | 0
host/rootfs/{ => image}/etc/mdev.conf | 0
host/rootfs/{ => image}/etc/mdev/listen | 0
host/rootfs/{ => image}/etc/mdev/net/add | 0
host/rootfs/{ => image}/etc/mdev/wait | 0
host/rootfs/{ => image}/etc/parse-devname | 0
host/rootfs/{ => image}/etc/passwd | 0
host/rootfs/{ => image}/etc/passwd.license | 0
.../etc/s6-linux-init/env/WAYLAND_DISPLAY | 0
.../etc/s6-linux-init/env/WAYLAND_DISPLAY.license | 0
.../etc/s6-linux-init/env/XDG_RUNTIME_DIR | 0
.../etc/s6-linux-init/env/XDG_RUNTIME_DIR.license | 0
.../etc/s6-linux-init/run-image/opengl-driver | 0
.../run-image/service/dbus/notification-fd | 0
.../run-image/service/dbus/notification-fd.license | 0
.../etc/s6-linux-init/run-image/service/dbus/run | 0
.../service/dbus/template/notification-fd | 0
.../service/dbus/template/notification-fd.license | 0
.../run-image/service/dbus/template/run | 0
.../s6-linux-init/run-image/service/getty-tty1/run | 0
.../s6-linux-init/run-image/service/getty-tty2/run | 0
.../s6-linux-init/run-image/service/getty-tty3/run | 0
.../s6-linux-init/run-image/service/getty-tty4/run | 0
.../service/s6-svscan-log/notification-fd | 0
.../service/s6-svscan-log/notification-fd.license | 0
.../run-image/service/s6-svscan-log/run | 0
.../run-image/service/serial-getty-generator/run | 0
.../run-image/service/serial-getty/notification-fd | 0
.../service/serial-getty/notification-fd.license | 0
.../run-image/service/serial-getty/run | 0
.../run-image/service/serial-getty/template/run | 0
.../service/vhost-user-fs/notification-fd | 0
.../service/vhost-user-fs/notification-fd.license | 0
.../run-image/service/vhost-user-fs/run | 0
.../service/vhost-user-fs/template/notification-fd | 0
.../vhost-user-fs/template/notification-fd.license | 0
.../run-image/service/vhost-user-fs/template/run | 0
.../service/vhost-user-gpu/notification-fd | 0
.../service/vhost-user-gpu/notification-fd.license | 0
.../run-image/service/vhost-user-gpu/run | 0
.../service/vhost-user-gpu/template/data/check | 0
.../vhost-user-gpu/template/notification-fd | 0
.../template/notification-fd.license | 0
.../run-image/service/vhost-user-gpu/template/run | 0
.../run-image/service/vhost-user-gpu/template/type | 0
.../service/vhost-user-gpu/template/type.license | 0
.../run-image/service/vmm/notification-fd | 0
.../run-image/service/vmm/notification-fd.license | 0
.../etc/s6-linux-init/run-image/service/vmm/run | 0
.../run-image/service/vmm/template/notification-fd | 0
.../service/vmm/template/notification-fd.license | 0
.../run-image/service/vmm/template/run | 0
.../notification-fd | 0
.../notification-fd.license | 0
.../service/xdg-desktop-portal-spectrum-host/run | 0
.../template/notification-fd | 0
.../template/notification-fd.license | 0
.../xdg-desktop-portal-spectrum-host/template/run | 0
.../{ => image}/etc/s6-linux-init/scripts/rc.init | 0
host/rootfs/{ => image}/etc/s6-rc/card0/type | 0
.../{ => image}/etc/s6-rc/card0/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/card0/up | 0
host/rootfs/{ => image}/etc/s6-rc/core/type | 0
.../rootfs/{ => image}/etc/s6-rc/core/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/core/up | 0
host/rootfs/{ => image}/etc/s6-rc/kvm/timeout-up | 0
.../{ => image}/etc/s6-rc/kvm/timeout-up.license | 0
host/rootfs/{ => image}/etc/s6-rc/kvm/type | 0
host/rootfs/{ => image}/etc/s6-rc/kvm/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/kvm/up | 0
.../etc/s6-rc/mdevd-coldplug/dependencies | 0
.../{ => image}/etc/s6-rc/mdevd-coldplug/type | 0
.../etc/s6-rc/mdevd-coldplug/type.license | 0
.../rootfs/{ => image}/etc/s6-rc/mdevd-coldplug/up | 0
.../{ => image}/etc/s6-rc/mdevd/notification-fd | 0
.../etc/s6-rc/mdevd/notification-fd.license | 0
host/rootfs/{ => image}/etc/s6-rc/mdevd/run | 0
host/rootfs/{ => image}/etc/s6-rc/mdevd/type | 0
.../{ => image}/etc/s6-rc/mdevd/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/ok-all/contents | 0
host/rootfs/{ => image}/etc/s6-rc/ok-all/type | 0
.../{ => image}/etc/s6-rc/ok-all/type.license | 0
.../rootfs/{ => image}/etc/s6-rc/static-nodes/type | 0
.../etc/s6-rc/static-nodes/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/static-nodes/up | 0
.../{ => image}/etc/s6-rc/sys-vmms/dependencies | 0
host/rootfs/{ => image}/etc/s6-rc/sys-vmms/type | 0
.../{ => image}/etc/s6-rc/sys-vmms/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/sys-vmms/up | 0
host/rootfs/{ => image}/etc/s6-rc/vm-env/contents | 0
host/rootfs/{ => image}/etc/s6-rc/vm-env/type | 0
.../{ => image}/etc/s6-rc/vm-env/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/vmm-env/contents | 0
host/rootfs/{ => image}/etc/s6-rc/vmm-env/type | 0
.../{ => image}/etc/s6-rc/vmm-env/type.license | 0
.../{ => image}/etc/s6-rc/weston/dependencies | 0
.../{ => image}/etc/s6-rc/weston/notification-fd | 0
.../etc/s6-rc/weston/notification-fd.license | 0
host/rootfs/{ => image}/etc/s6-rc/weston/run | 0
host/rootfs/{ => image}/etc/s6-rc/weston/type | 0
.../{ => image}/etc/s6-rc/weston/type.license | 0
host/rootfs/{ => image}/etc/xdg/weston/autolaunch | 0
host/rootfs/{ => image}/etc/xdg/weston/weston.ini | 0
host/rootfs/{ => image}/lib | 0
host/rootfs/{ => image}/sbin | 0
host/rootfs/{ => image}/usr/bin/assign-devices | 0
.../{ => image}/usr/bin/create-vm-dependencies | 0
host/rootfs/{ => image}/usr/bin/run-appimage | 0
host/rootfs/{ => image}/usr/bin/run-vmm | 0
host/rootfs/{ => image}/usr/bin/vm-console | 0
host/rootfs/{ => image}/usr/bin/vm-import | 0
host/rootfs/{ => image}/usr/bin/vm-start | 0
host/rootfs/{ => image}/usr/bin/vm-stop | 0
host/rootfs/{ => image}/usr/bin/xdg-open | 0
.../org.freedesktop.portal.Documents.service | 0
img/app/Makefile | 98 +-------------
img/app/default.nix | 1 +
img/app/file-list.mk | 65 ++++++++++
img/app/{ => image}/bin | 0
img/app/{ => image}/etc/dbus-1/session.conf | 0
img/app/{ => image}/etc/fonts/fonts.conf | 0
img/app/{ => image}/etc/fstab | 0
img/app/{ => image}/etc/init | 0
img/app/{ => image}/etc/mdev.conf | 0
img/app/{ => image}/etc/mdev/iface | 0
img/app/{ => image}/etc/mdev/listen | 0
img/app/{ => image}/etc/mdev/virtiofs | 0
img/app/{ => image}/etc/mdev/wait | 0
img/app/{ => image}/etc/passwd | 0
img/app/{ => image}/etc/passwd.license | 0
img/app/{ => image}/etc/pipewire/pipewire.conf | 0
img/app/{ => image}/etc/resolv.conf | 0
.../etc/s6-linux-init/env/DBUS_SESSION_BUS_ADDRESS | 0
.../env/DBUS_SESSION_BUS_ADDRESS.license | 0
img/app/{ => image}/etc/s6-linux-init/env/DISPLAY | 0
.../etc/s6-linux-init/env/DISPLAY.license | 0
.../etc/s6-linux-init/env/GTK_USE_PORTAL | 0
.../etc/s6-linux-init/env/GTK_USE_PORTAL.license | 0
.../s6-linux-init/env/NIX_XDG_DESKTOP_PORTAL_DIR | 0
.../env/NIX_XDG_DESKTOP_PORTAL_DIR.license | 0
.../etc/s6-linux-init/env/WAYLAND_DISPLAY | 0
.../etc/s6-linux-init/env/WAYLAND_DISPLAY.license | 0
.../etc/s6-linux-init/env/XDG_RUNTIME_DIR | 0
.../etc/s6-linux-init/env/XDG_RUNTIME_DIR.license | 0
.../s6-linux-init/run-image/service/getty-hvc0/run | 0
.../s6-linux-init-shutdownd/notification-fd | 0
.../notification-fd.license | 0
.../run-image/service/s6-linux-init-shutdownd/run | 0
.../{ => image}/etc/s6-linux-init/scripts/rc.init | 0
.../etc/s6-linux-init/scripts/rc.shutdown | 0
.../etc/s6-linux-init/scripts/rc.shutdown.final | 0
.../{ => image}/etc/s6-rc/app/dependencies.d/dbus | 0
.../etc/s6-rc/app/dependencies.d/pipewire | 0
.../s6-rc/app/dependencies.d/wayland-proxy-virtwl | 0
img/app/{ => image}/etc/s6-rc/app/run | 0
img/app/{ => image}/etc/s6-rc/app/type | 0
img/app/{ => image}/etc/s6-rc/app/type.license | 0
img/app/{ => image}/etc/s6-rc/dbus/notification-fd | 0
.../etc/s6-rc/dbus/notification-fd.license | 0
img/app/{ => image}/etc/s6-rc/dbus/run | 0
img/app/{ => image}/etc/s6-rc/dbus/type | 0
img/app/{ => image}/etc/s6-rc/dbus/type.license | 0
.../etc/s6-rc/mdevd-coldplug/dependencies | 0
img/app/{ => image}/etc/s6-rc/mdevd-coldplug/type | 0
.../etc/s6-rc/mdevd-coldplug/type.license | 0
img/app/{ => image}/etc/s6-rc/mdevd-coldplug/up | 0
.../{ => image}/etc/s6-rc/mdevd/notification-fd | 0
.../etc/s6-rc/mdevd/notification-fd.license | 0
img/app/{ => image}/etc/s6-rc/mdevd/run | 0
img/app/{ => image}/etc/s6-rc/mdevd/type | 0
img/app/{ => image}/etc/s6-rc/mdevd/type.license | 0
img/app/{ => image}/etc/s6-rc/ok-all/contents | 0
img/app/{ => image}/etc/s6-rc/ok-all/type | 0
img/app/{ => image}/etc/s6-rc/ok-all/type.license | 0
.../{ => image}/etc/s6-rc/pipewire/notification-fd | 0
.../etc/s6-rc/pipewire/notification-fd.license | 0
img/app/{ => image}/etc/s6-rc/pipewire/run | 0
img/app/{ => image}/etc/s6-rc/pipewire/type | 0
.../{ => image}/etc/s6-rc/pipewire/type.license | 0
.../etc/s6-rc/wayland-proxy-virtwl/notification-fd | 0
.../wayland-proxy-virtwl/notification-fd.license | 0
.../{ => image}/etc/s6-rc/wayland-proxy-virtwl/run | 0
.../etc/s6-rc/wayland-proxy-virtwl/type | 0
.../etc/s6-rc/wayland-proxy-virtwl/type.license | 0
.../etc/s6-rc/wireplumber/dependencies.d/dbus | 0
.../etc/s6-rc/wireplumber/dependencies.d/pipewire | 0
img/app/{ => image}/etc/s6-rc/wireplumber/run | 0
img/app/{ => image}/etc/s6-rc/wireplumber/type | 0
.../{ => image}/etc/s6-rc/wireplumber/type.license | 0
.../{ => image}/etc/ssl/certs/ca-certificates.crt | 0
.../wireplumber.conf.d/99_spectrum.conf | 0
.../etc/xdg/xdg-desktop-portal/portals.conf | 0
img/app/{ => image}/sbin | 0
lib/common.mk | 1 +
lib/erofs.mk | 49 +++++++
scripts/genfiles.awk | 120 ++++++++++++++++++
scripts/genfiles.sh | 26 ++++
scripts/git-rebase | 17 +++
scripts/pre-commit.sh | 11 ++
vm/sys/net/Makefile | 69 +---------
vm/sys/net/default.nix | 1 +
vm/sys/net/file-list.mk | 42 ++++++
vm/sys/net/{ => image}/bin | 0
vm/sys/net/{ => image}/etc/dbus-1/system.conf | 0
vm/sys/net/{ => image}/etc/fstab | 0
vm/sys/net/{ => image}/etc/init | 0
vm/sys/net/{ => image}/etc/mdev.conf | 0
vm/sys/net/{ => image}/etc/mdev/iface | 0
vm/sys/net/{ => image}/etc/nftables.conf | 0
vm/sys/net/{ => image}/etc/passwd | 0
vm/sys/net/{ => image}/etc/passwd.license | 0
.../s6-linux-init/run-image/service/getty-hvc0/run | 0
.../{ => image}/etc/s6-linux-init/scripts/rc.init | 0
.../net/{ => image}/etc/s6-rc/connman/dependencies | 0
vm/sys/net/{ => image}/etc/s6-rc/connman/run | 0
vm/sys/net/{ => image}/etc/s6-rc/connman/type | 0
.../net/{ => image}/etc/s6-rc/connman/type.license | 0
.../net/{ => image}/etc/s6-rc/dbus/notification-fd | 0
.../etc/s6-rc/dbus/notification-fd.license | 0
vm/sys/net/{ => image}/etc/s6-rc/dbus/run | 0
vm/sys/net/{ => image}/etc/s6-rc/dbus/type | 0
vm/sys/net/{ => image}/etc/s6-rc/dbus/type.license | 0
.../etc/s6-rc/mdevd-coldplug/dependencies | 0
.../net/{ => image}/etc/s6-rc/mdevd-coldplug/type | 0
.../etc/s6-rc/mdevd-coldplug/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/mdevd-coldplug/up | 0
.../{ => image}/etc/s6-rc/mdevd/notification-fd | 0
.../etc/s6-rc/mdevd/notification-fd.license | 0
vm/sys/net/{ => image}/etc/s6-rc/mdevd/run | 0
vm/sys/net/{ => image}/etc/s6-rc/mdevd/type | 0
.../net/{ => image}/etc/s6-rc/mdevd/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/nftables/type | 0
.../{ => image}/etc/s6-rc/nftables/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/nftables/up | 0
vm/sys/net/{ => image}/etc/s6-rc/ok-all/contents | 0
vm/sys/net/{ => image}/etc/s6-rc/ok-all/type | 0
.../net/{ => image}/etc/s6-rc/ok-all/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/sysctl/type | 0
.../net/{ => image}/etc/s6-rc/sysctl/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/sysctl/up | 0
vm/sys/net/{ => image}/etc/sysctl.conf | 0
vm/sys/net/{ => image}/lib | 0
vm/sys/net/{ => image}/sbin | 0
vm/sys/net/{ => image}/var/run | 0
254 files changed, 463 insertions(+), 297 deletions(-)
---
base-commit: cbb10b15f36a7ac35400e226b284b1cb091cb3c9
change-id: 20250831-genfiles-46787fc10869
--
Sincerely,
Demi Marie Obenour (she/her/hers)
next prev parent reply other threads:[~2025-09-10 5:31 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-04 1:56 [PATCH 0/4] Generate file lists from a script Demi Marie Obenour
2025-09-04 1:56 ` [PATCH 1/4] Move all files for the image into a subdirectory Demi Marie Obenour
2025-09-04 1:56 ` [PATCH 2/4] Generate makefile file lists from a script Demi Marie Obenour
2025-09-08 9:59 ` Alyssa Ross
2025-09-08 18:45 ` Demi Marie Obenour
2025-09-09 14:51 ` Alyssa Ross
2025-09-04 1:56 ` [PATCH 3/4] Common make rules for building erofs images Demi Marie Obenour
2025-09-08 10:01 ` Alyssa Ross
2025-09-08 18:53 ` Demi Marie Obenour
2025-09-09 14:56 ` Alyssa Ross
2025-09-04 1:56 ` [PATCH 4/4] Use /etc/s6-rc/compiled for compiled s6-rc directory Demi Marie Obenour
2025-09-10 5:29 ` Demi Marie Obenour [this message]
2025-09-10 5:29 ` [PATCH v2 1/3] Move all files for the image into a subdirectory Demi Marie Obenour
2025-09-10 18:58 ` Alyssa Ross
2025-09-11 12:21 ` Demi Marie Obenour
2025-09-10 5:29 ` [PATCH v2 2/3] Generate makefile file lists from a script Demi Marie Obenour
2025-09-10 5:29 ` [PATCH v2 3/3] Common make rules for building erofs images Demi Marie Obenour
2025-09-11 12:47 ` [PATCH v3 0/4] Generate file lists from a script Demi Marie Obenour
2025-09-11 12:47 ` [PATCH v3 1/4] Do not ignore errors from tar Demi Marie Obenour
2025-09-17 11:48 ` Alyssa Ross
2025-09-18 2:45 ` Demi Marie Obenour
2025-09-19 7:46 ` Alyssa Ross
2025-09-30 12:59 ` Alyssa Ross
2025-09-19 7:55 ` Alyssa Ross
2025-09-19 19:03 ` Demi Marie Obenour
2025-09-11 12:47 ` [PATCH v3 2/4] Move all files for the image into a subdirectory Demi Marie Obenour
2025-09-17 12:30 ` Alyssa Ross
2025-09-17 12:39 ` Alyssa Ross
2025-09-17 13:03 ` Alyssa Ross
2025-09-11 12:47 ` [PATCH v3 3/4] Generate makefile file lists from a script Demi Marie Obenour
2025-09-11 12:47 ` [PATCH v3 4/4] Common make rules for building erofs images Demi Marie Obenour
2025-09-21 2:23 ` [PATCH v3] Generate file lists from a script Demi Marie Obenour
2025-09-21 8:47 ` Alyssa Ross
2025-09-21 16:51 ` Demi Marie Obenour
2025-09-21 17:07 ` Alyssa Ross
2025-09-21 17:24 ` [PATCH v4] " Demi Marie Obenour
2025-09-25 11:22 ` Alyssa Ross
2025-09-26 16:31 ` [PATCH v5] " Demi Marie Obenour
2025-09-27 8:19 ` Alyssa Ross
2025-09-27 8:42 ` Demi Marie Obenour
2025-09-27 16:22 ` [PATCH v6] " Demi Marie Obenour
2025-09-29 8:12 ` Alyssa Ross
2025-09-29 17:20 ` Demi Marie Obenour
2025-09-29 17:18 ` [PATCH v7] " Demi Marie Obenour
2025-10-01 9:20 ` Alyssa Ross
2025-10-01 9:24 ` Demi Marie Obenour
2025-10-01 9:35 ` Alyssa Ross
2025-10-01 18:30 ` [PATCH v8] " Demi Marie Obenour
2025-10-02 9:46 ` Alyssa Ross
2025-10-02 17:37 ` [PATCH v9] " Demi Marie Obenour
2025-10-03 9:04 ` Alyssa Ross
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250910-genfiles-v2-0-37ebe07a3cdc@gmail.com \
--to=demiobenour@gmail.com \
--cc=devel@spectrum-os.org \
--cc=hi@alyssa.is \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://spectrum-os.org/git/crosvm
https://spectrum-os.org/git/doc
https://spectrum-os.org/git/mktuntap
https://spectrum-os.org/git/nixpkgs
https://spectrum-os.org/git/spectrum
https://spectrum-os.org/git/ucspi-vsock
https://spectrum-os.org/git/www
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).