[PATCH v2 0/3] Generate file lists from a script

Demi Marie Obenour
2025-09-10T05:31:19Z

Right now, the makefiles in host/rootfs, vm/sys/net, and img/app have
manually-maintained lists of files and symlinks. These duplicate the
information in the git repository and can easily get out of sync or
cause unnecessary merge conflicts.

Avoid this problem by moving the files used in the erofs images to an
image/ directory and using what Git knows about this directory to
generate the lists via a script. This makes maintenance easier and
avoids conflating code used in the build (anything not in image/) and
code for the image (anything that is in image/).

This reveals that there is a lot of duplicated make logic around image
generation, so consolidate it in a single makefile that is included by
all three of the makefiles.

Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Changes in v2:
- Drop the last patch (switching to /etc/s6-rc/compiled) as it is
controversial and should be reviewed separately.
- Add missing copyright notices.
- Use a wrapper shell script to make the awk code easier to read.
- Improve documentation.
- Add helper scripts for use in git hooks and rebasing.
- Link to v1: https://spectrum-os.org/lists/archives/spectrum-devel/20250903-genfiles-v1-0-cc993fcb1e4c@gmail.com/

---
Demi Marie Obenour (3):
Move all files for the image into a subdirectory
Generate makefile file lists from a script
Common make rules for building erofs images

Documentation/development/built-in-vms.adoc | 18 +++
host/rootfs/Makefile | 141 +--------------------
host/rootfs/default.nix | 1 +
host/rootfs/file-list.mk | 100 +++++++++++++++
host/rootfs/{ => image}/bin | 0
host/rootfs/{ => image}/etc/fonts/fonts.conf | 0
host/rootfs/{ => image}/etc/fstab | 0
host/rootfs/{ => image}/etc/group | 0
host/rootfs/{ => image}/etc/group.license | 0
host/rootfs/{ => image}/etc/init | 0
host/rootfs/{ => image}/etc/login | 0
host/rootfs/{ => image}/etc/mdev.conf | 0
host/rootfs/{ => image}/etc/mdev/listen | 0
host/rootfs/{ => image}/etc/mdev/net/add | 0
host/rootfs/{ => image}/etc/mdev/wait | 0
host/rootfs/{ => image}/etc/parse-devname | 0
host/rootfs/{ => image}/etc/passwd | 0
host/rootfs/{ => image}/etc/passwd.license | 0
.../etc/s6-linux-init/env/WAYLAND_DISPLAY | 0
.../etc/s6-linux-init/env/WAYLAND_DISPLAY.license | 0
.../etc/s6-linux-init/env/XDG_RUNTIME_DIR | 0
.../etc/s6-linux-init/env/XDG_RUNTIME_DIR.license | 0
.../etc/s6-linux-init/run-image/opengl-driver | 0
.../run-image/service/dbus/notification-fd | 0
.../run-image/service/dbus/notification-fd.license | 0
.../etc/s6-linux-init/run-image/service/dbus/run | 0
.../service/dbus/template/notification-fd | 0
.../service/dbus/template/notification-fd.license | 0
.../run-image/service/dbus/template/run | 0
.../s6-linux-init/run-image/service/getty-tty1/run | 0
.../s6-linux-init/run-image/service/getty-tty2/run | 0
.../s6-linux-init/run-image/service/getty-tty3/run | 0
.../s6-linux-init/run-image/service/getty-tty4/run | 0
.../service/s6-svscan-log/notification-fd | 0
.../service/s6-svscan-log/notification-fd.license | 0
.../run-image/service/s6-svscan-log/run | 0
.../run-image/service/serial-getty-generator/run | 0
.../run-image/service/serial-getty/notification-fd | 0
.../service/serial-getty/notification-fd.license | 0
.../run-image/service/serial-getty/run | 0
.../run-image/service/serial-getty/template/run | 0
.../service/vhost-user-fs/notification-fd | 0
.../service/vhost-user-fs/notification-fd.license | 0
.../run-image/service/vhost-user-fs/run | 0
.../service/vhost-user-fs/template/notification-fd | 0
.../vhost-user-fs/template/notification-fd.license | 0
.../run-image/service/vhost-user-fs/template/run | 0
.../service/vhost-user-gpu/notification-fd | 0
.../service/vhost-user-gpu/notification-fd.license | 0
.../run-image/service/vhost-user-gpu/run | 0
.../service/vhost-user-gpu/template/data/check | 0
.../vhost-user-gpu/template/notification-fd | 0
.../template/notification-fd.license | 0
.../run-image/service/vhost-user-gpu/template/run | 0
.../run-image/service/vhost-user-gpu/template/type | 0
.../service/vhost-user-gpu/template/type.license | 0
.../run-image/service/vmm/notification-fd | 0
.../run-image/service/vmm/notification-fd.license | 0
.../etc/s6-linux-init/run-image/service/vmm/run | 0
.../run-image/service/vmm/template/notification-fd | 0
.../service/vmm/template/notification-fd.license | 0
.../run-image/service/vmm/template/run | 0
.../notification-fd | 0
.../notification-fd.license | 0
.../service/xdg-desktop-portal-spectrum-host/run | 0
.../template/notification-fd | 0
.../template/notification-fd.license | 0
.../xdg-desktop-portal-spectrum-host/template/run | 0
.../{ => image}/etc/s6-linux-init/scripts/rc.init | 0
host/rootfs/{ => image}/etc/s6-rc/card0/type | 0
.../{ => image}/etc/s6-rc/card0/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/card0/up | 0
host/rootfs/{ => image}/etc/s6-rc/core/type | 0
.../rootfs/{ => image}/etc/s6-rc/core/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/core/up | 0
host/rootfs/{ => image}/etc/s6-rc/kvm/timeout-up | 0
.../{ => image}/etc/s6-rc/kvm/timeout-up.license | 0
host/rootfs/{ => image}/etc/s6-rc/kvm/type | 0
host/rootfs/{ => image}/etc/s6-rc/kvm/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/kvm/up | 0
.../etc/s6-rc/mdevd-coldplug/dependencies | 0
.../{ => image}/etc/s6-rc/mdevd-coldplug/type | 0
.../etc/s6-rc/mdevd-coldplug/type.license | 0
.../rootfs/{ => image}/etc/s6-rc/mdevd-coldplug/up | 0
.../{ => image}/etc/s6-rc/mdevd/notification-fd | 0
.../etc/s6-rc/mdevd/notification-fd.license | 0
host/rootfs/{ => image}/etc/s6-rc/mdevd/run | 0
host/rootfs/{ => image}/etc/s6-rc/mdevd/type | 0
.../{ => image}/etc/s6-rc/mdevd/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/ok-all/contents | 0
host/rootfs/{ => image}/etc/s6-rc/ok-all/type | 0
.../{ => image}/etc/s6-rc/ok-all/type.license | 0
.../rootfs/{ => image}/etc/s6-rc/static-nodes/type | 0
.../etc/s6-rc/static-nodes/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/static-nodes/up | 0
.../{ => image}/etc/s6-rc/sys-vmms/dependencies | 0
host/rootfs/{ => image}/etc/s6-rc/sys-vmms/type | 0
.../{ => image}/etc/s6-rc/sys-vmms/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/sys-vmms/up | 0
host/rootfs/{ => image}/etc/s6-rc/vm-env/contents | 0
host/rootfs/{ => image}/etc/s6-rc/vm-env/type | 0
.../{ => image}/etc/s6-rc/vm-env/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/vmm-env/contents | 0
host/rootfs/{ => image}/etc/s6-rc/vmm-env/type | 0
.../{ => image}/etc/s6-rc/vmm-env/type.license | 0
.../{ => image}/etc/s6-rc/weston/dependencies | 0
.../{ => image}/etc/s6-rc/weston/notification-fd | 0
.../etc/s6-rc/weston/notification-fd.license | 0
host/rootfs/{ => image}/etc/s6-rc/weston/run | 0
host/rootfs/{ => image}/etc/s6-rc/weston/type | 0
.../{ => image}/etc/s6-rc/weston/type.license | 0
host/rootfs/{ => image}/etc/xdg/weston/autolaunch | 0
host/rootfs/{ => image}/etc/xdg/weston/weston.ini | 0
host/rootfs/{ => image}/lib | 0
host/rootfs/{ => image}/sbin | 0
host/rootfs/{ => image}/usr/bin/assign-devices | 0
.../{ => image}/usr/bin/create-vm-dependencies | 0
host/rootfs/{ => image}/usr/bin/run-appimage | 0
host/rootfs/{ => image}/usr/bin/run-vmm | 0
host/rootfs/{ => image}/usr/bin/vm-console | 0
host/rootfs/{ => image}/usr/bin/vm-import | 0
host/rootfs/{ => image}/usr/bin/vm-start | 0
host/rootfs/{ => image}/usr/bin/vm-stop | 0
host/rootfs/{ => image}/usr/bin/xdg-open | 0
.../org.freedesktop.portal.Documents.service | 0
img/app/Makefile | 98 +-------------
img/app/default.nix | 1 +
img/app/file-list.mk | 65 ++++++++++
img/app/{ => image}/bin | 0
img/app/{ => image}/etc/dbus-1/session.conf | 0
img/app/{ => image}/etc/fonts/fonts.conf | 0
img/app/{ => image}/etc/fstab | 0
img/app/{ => image}/etc/init | 0
img/app/{ => image}/etc/mdev.conf | 0
img/app/{ => image}/etc/mdev/iface | 0
img/app/{ => image}/etc/mdev/listen | 0
img/app/{ => image}/etc/mdev/virtiofs | 0
img/app/{ => image}/etc/mdev/wait | 0
img/app/{ => image}/etc/passwd | 0
img/app/{ => image}/etc/passwd.license | 0
img/app/{ => image}/etc/pipewire/pipewire.conf | 0
img/app/{ => image}/etc/resolv.conf | 0
.../etc/s6-linux-init/env/DBUS_SESSION_BUS_ADDRESS | 0
.../env/DBUS_SESSION_BUS_ADDRESS.license | 0
img/app/{ => image}/etc/s6-linux-init/env/DISPLAY | 0
.../etc/s6-linux-init/env/DISPLAY.license | 0
.../etc/s6-linux-init/env/GTK_USE_PORTAL | 0
.../etc/s6-linux-init/env/GTK_USE_PORTAL.license | 0
.../s6-linux-init/env/NIX_XDG_DESKTOP_PORTAL_DIR | 0
.../env/NIX_XDG_DESKTOP_PORTAL_DIR.license | 0
.../etc/s6-linux-init/env/WAYLAND_DISPLAY | 0
.../etc/s6-linux-init/env/WAYLAND_DISPLAY.license | 0
.../etc/s6-linux-init/env/XDG_RUNTIME_DIR | 0
.../etc/s6-linux-init/env/XDG_RUNTIME_DIR.license | 0
.../s6-linux-init/run-image/service/getty-hvc0/run | 0
.../s6-linux-init-shutdownd/notification-fd | 0
.../notification-fd.license | 0
.../run-image/service/s6-linux-init-shutdownd/run | 0
.../{ => image}/etc/s6-linux-init/scripts/rc.init | 0
.../etc/s6-linux-init/scripts/rc.shutdown | 0
.../etc/s6-linux-init/scripts/rc.shutdown.final | 0
.../{ => image}/etc/s6-rc/app/dependencies.d/dbus | 0
.../etc/s6-rc/app/dependencies.d/pipewire | 0
.../s6-rc/app/dependencies.d/wayland-proxy-virtwl | 0
img/app/{ => image}/etc/s6-rc/app/run | 0
img/app/{ => image}/etc/s6-rc/app/type | 0
img/app/{ => image}/etc/s6-rc/app/type.license | 0
img/app/{ => image}/etc/s6-rc/dbus/notification-fd | 0
.../etc/s6-rc/dbus/notification-fd.license | 0
img/app/{ => image}/etc/s6-rc/dbus/run | 0
img/app/{ => image}/etc/s6-rc/dbus/type | 0
img/app/{ => image}/etc/s6-rc/dbus/type.license | 0
.../etc/s6-rc/mdevd-coldplug/dependencies | 0
img/app/{ => image}/etc/s6-rc/mdevd-coldplug/type | 0
.../etc/s6-rc/mdevd-coldplug/type.license | 0
img/app/{ => image}/etc/s6-rc/mdevd-coldplug/up | 0
.../{ => image}/etc/s6-rc/mdevd/notification-fd | 0
.../etc/s6-rc/mdevd/notification-fd.license | 0
img/app/{ => image}/etc/s6-rc/mdevd/run | 0
img/app/{ => image}/etc/s6-rc/mdevd/type | 0
img/app/{ => image}/etc/s6-rc/mdevd/type.license | 0
img/app/{ => image}/etc/s6-rc/ok-all/contents | 0
img/app/{ => image}/etc/s6-rc/ok-all/type | 0
img/app/{ => image}/etc/s6-rc/ok-all/type.license | 0
.../{ => image}/etc/s6-rc/pipewire/notification-fd | 0
.../etc/s6-rc/pipewire/notification-fd.license | 0
img/app/{ => image}/etc/s6-rc/pipewire/run | 0
img/app/{ => image}/etc/s6-rc/pipewire/type | 0
.../{ => image}/etc/s6-rc/pipewire/type.license | 0
.../etc/s6-rc/wayland-proxy-virtwl/notification-fd | 0
.../wayland-proxy-virtwl/notification-fd.license | 0
.../{ => image}/etc/s6-rc/wayland-proxy-virtwl/run | 0
.../etc/s6-rc/wayland-proxy-virtwl/type | 0
.../etc/s6-rc/wayland-proxy-virtwl/type.license | 0
.../etc/s6-rc/wireplumber/dependencies.d/dbus | 0
.../etc/s6-rc/wireplumber/dependencies.d/pipewire | 0
img/app/{ => image}/etc/s6-rc/wireplumber/run | 0
img/app/{ => image}/etc/s6-rc/wireplumber/type | 0
.../{ => image}/etc/s6-rc/wireplumber/type.license | 0
.../{ => image}/etc/ssl/certs/ca-certificates.crt | 0
.../wireplumber.conf.d/99_spectrum.conf | 0
.../etc/xdg/xdg-desktop-portal/portals.conf | 0
img/app/{ => image}/sbin | 0
lib/common.mk | 1 +
lib/erofs.mk | 49 +++++++
scripts/genfiles.awk | 120 ++++++++++++++++++
scripts/genfiles.sh | 26 ++++
scripts/git-rebase | 17 +++
scripts/pre-commit.sh | 11 ++
vm/sys/net/Makefile | 69 +---------
vm/sys/net/default.nix | 1 +
vm/sys/net/file-list.mk | 42 ++++++
vm/sys/net/{ => image}/bin | 0
vm/sys/net/{ => image}/etc/dbus-1/system.conf | 0
vm/sys/net/{ => image}/etc/fstab | 0
vm/sys/net/{ => image}/etc/init | 0
vm/sys/net/{ => image}/etc/mdev.conf | 0
vm/sys/net/{ => image}/etc/mdev/iface | 0
vm/sys/net/{ => image}/etc/nftables.conf | 0
vm/sys/net/{ => image}/etc/passwd | 0
vm/sys/net/{ => image}/etc/passwd.license | 0
.../s6-linux-init/run-image/service/getty-hvc0/run | 0
.../{ => image}/etc/s6-linux-init/scripts/rc.init | 0
.../net/{ => image}/etc/s6-rc/connman/dependencies | 0
vm/sys/net/{ => image}/etc/s6-rc/connman/run | 0
vm/sys/net/{ => image}/etc/s6-rc/connman/type | 0
.../net/{ => image}/etc/s6-rc/connman/type.license | 0
.../net/{ => image}/etc/s6-rc/dbus/notification-fd | 0
.../etc/s6-rc/dbus/notification-fd.license | 0
vm/sys/net/{ => image}/etc/s6-rc/dbus/run | 0
vm/sys/net/{ => image}/etc/s6-rc/dbus/type | 0
vm/sys/net/{ => image}/etc/s6-rc/dbus/type.license | 0
.../etc/s6-rc/mdevd-coldplug/dependencies | 0
.../net/{ => image}/etc/s6-rc/mdevd-coldplug/type | 0
.../etc/s6-rc/mdevd-coldplug/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/mdevd-coldplug/up | 0
.../{ => image}/etc/s6-rc/mdevd/notification-fd | 0
.../etc/s6-rc/mdevd/notification-fd.license | 0
vm/sys/net/{ => image}/etc/s6-rc/mdevd/run | 0
vm/sys/net/{ => image}/etc/s6-rc/mdevd/type | 0
.../net/{ => image}/etc/s6-rc/mdevd/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/nftables/type | 0
.../{ => image}/etc/s6-rc/nftables/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/nftables/up | 0
vm/sys/net/{ => image}/etc/s6-rc/ok-all/contents | 0
vm/sys/net/{ => image}/etc/s6-rc/ok-all/type | 0
.../net/{ => image}/etc/s6-rc/ok-all/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/sysctl/type | 0
.../net/{ => image}/etc/s6-rc/sysctl/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/sysctl/up | 0
vm/sys/net/{ => image}/etc/sysctl.conf | 0
vm/sys/net/{ => image}/lib | 0
vm/sys/net/{ => image}/sbin | 0
vm/sys/net/{ => image}/var/run | 0
254 files changed, 463 insertions(+), 297 deletions(-)
---
base-commit: cbb10b15f36a7ac35400e226b284b1cb091cb3c9
change-id: 20250831-genfiles-46787fc10869
--
Sincerely,
Demi Marie Obenour (she/her/hers)