From: Demi Marie Obenour <demiobenour@gmail.com>
To: Spectrum OS Development <devel@spectrum-os.org>
Cc: Demi Marie Obenour <demiobenour@gmail.com>, Alyssa Ross <hi@alyssa.is>
Subject: [PATCH 0/4] Generate file lists from a script
Date: Wed, 03 Sep 2025 21:56:51 -0400 [thread overview]
Message-ID: <20250903-genfiles-v1-0-cc993fcb1e4c@gmail.com> (raw)
Right now, the makefiles in host/rootfs, vm/sys/net, and img/app have
manually-maintained lists of files and symlinks. These duplicate the
information in the git repository and can easily get out of sync or
cause unnecessary merge conflicts.
Avoid this problem by moving the files used in the erofs images to an
image/ directory and using what Git knows about this directory to
generate the lists via a script. This makes maintenance easier and
avoids conflating code used in the build (anything not in image/) and
code for the image (anything that is in image/).
This reveals that there is a lot of duplicated make logic around image
generation, so consolidate it in a single makefile that is included by
all three of the makefiles. This, in turn, makes tree-wide changes
easier, as shown by the final patch in this series.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Demi Marie Obenour (4):
Move all files for the image into a subdirectory
Generate makefile file lists from a script
Common make rules for building erofs images
Use /etc/s6-rc/compiled for compiled s6-rc directory
Documentation/development/built-in-vms.adoc | 7 +
host/rootfs/Makefile | 144 +--------------------
host/rootfs/default.nix | 1 +
host/rootfs/file-list.mk | 100 ++++++++++++++
host/rootfs/{ => image}/bin | 0
host/rootfs/{ => image}/etc/fonts/fonts.conf | 0
host/rootfs/{ => image}/etc/fstab | 0
host/rootfs/{ => image}/etc/group | 0
host/rootfs/{ => image}/etc/group.license | 0
host/rootfs/{ => image}/etc/init | 0
host/rootfs/{ => image}/etc/login | 0
host/rootfs/{ => image}/etc/mdev.conf | 0
host/rootfs/{ => image}/etc/mdev/listen | 0
host/rootfs/{ => image}/etc/mdev/net/add | 0
host/rootfs/{ => image}/etc/mdev/wait | 0
host/rootfs/{ => image}/etc/parse-devname | 0
host/rootfs/{ => image}/etc/passwd | 0
host/rootfs/{ => image}/etc/passwd.license | 0
.../etc/s6-linux-init/env/WAYLAND_DISPLAY | 0
.../etc/s6-linux-init/env/WAYLAND_DISPLAY.license | 0
.../etc/s6-linux-init/env/XDG_RUNTIME_DIR | 0
.../etc/s6-linux-init/env/XDG_RUNTIME_DIR.license | 0
.../etc/s6-linux-init/run-image/opengl-driver | 0
.../run-image/service/dbus/notification-fd | 0
.../run-image/service/dbus/notification-fd.license | 0
.../etc/s6-linux-init/run-image/service/dbus/run | 0
.../service/dbus/template/notification-fd | 0
.../service/dbus/template/notification-fd.license | 0
.../run-image/service/dbus/template/run | 0
.../s6-linux-init/run-image/service/getty-tty1/run | 0
.../s6-linux-init/run-image/service/getty-tty2/run | 0
.../s6-linux-init/run-image/service/getty-tty3/run | 0
.../s6-linux-init/run-image/service/getty-tty4/run | 0
.../service/s6-svscan-log/notification-fd | 0
.../service/s6-svscan-log/notification-fd.license | 0
.../run-image/service/s6-svscan-log/run | 0
.../run-image/service/serial-getty-generator/run | 0
.../run-image/service/serial-getty/notification-fd | 0
.../service/serial-getty/notification-fd.license | 0
.../run-image/service/serial-getty/run | 0
.../run-image/service/serial-getty/template/run | 0
.../service/vhost-user-fs/notification-fd | 0
.../service/vhost-user-fs/notification-fd.license | 0
.../run-image/service/vhost-user-fs/run | 0
.../service/vhost-user-fs/template/notification-fd | 0
.../vhost-user-fs/template/notification-fd.license | 0
.../run-image/service/vhost-user-fs/template/run | 0
.../service/vhost-user-gpu/notification-fd | 0
.../service/vhost-user-gpu/notification-fd.license | 0
.../run-image/service/vhost-user-gpu/run | 0
.../service/vhost-user-gpu/template/data/check | 0
.../vhost-user-gpu/template/notification-fd | 0
.../template/notification-fd.license | 0
.../run-image/service/vhost-user-gpu/template/run | 0
.../run-image/service/vhost-user-gpu/template/type | 0
.../service/vhost-user-gpu/template/type.license | 0
.../run-image/service/vmm/notification-fd | 0
.../run-image/service/vmm/notification-fd.license | 0
.../etc/s6-linux-init/run-image/service/vmm/run | 0
.../run-image/service/vmm/template/notification-fd | 0
.../service/vmm/template/notification-fd.license | 0
.../run-image/service/vmm/template/run | 0
.../notification-fd | 0
.../notification-fd.license | 0
.../service/xdg-desktop-portal-spectrum-host/run | 0
.../template/notification-fd | 0
.../template/notification-fd.license | 0
.../xdg-desktop-portal-spectrum-host/template/run | 0
.../{ => image}/etc/s6-linux-init/scripts/rc.init | 2 +-
host/rootfs/{ => image}/etc/s6-rc/card0/type | 0
.../{ => image}/etc/s6-rc/card0/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/card0/up | 0
host/rootfs/{ => image}/etc/s6-rc/core/type | 0
.../rootfs/{ => image}/etc/s6-rc/core/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/core/up | 0
host/rootfs/{ => image}/etc/s6-rc/kvm/timeout-up | 0
.../{ => image}/etc/s6-rc/kvm/timeout-up.license | 0
host/rootfs/{ => image}/etc/s6-rc/kvm/type | 0
host/rootfs/{ => image}/etc/s6-rc/kvm/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/kvm/up | 0
.../etc/s6-rc/mdevd-coldplug/dependencies | 0
.../{ => image}/etc/s6-rc/mdevd-coldplug/type | 0
.../etc/s6-rc/mdevd-coldplug/type.license | 0
.../rootfs/{ => image}/etc/s6-rc/mdevd-coldplug/up | 0
.../{ => image}/etc/s6-rc/mdevd/notification-fd | 0
.../etc/s6-rc/mdevd/notification-fd.license | 0
host/rootfs/{ => image}/etc/s6-rc/mdevd/run | 0
host/rootfs/{ => image}/etc/s6-rc/mdevd/type | 0
.../{ => image}/etc/s6-rc/mdevd/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/ok-all/contents | 0
host/rootfs/{ => image}/etc/s6-rc/ok-all/type | 0
.../{ => image}/etc/s6-rc/ok-all/type.license | 0
.../rootfs/{ => image}/etc/s6-rc/static-nodes/type | 0
.../etc/s6-rc/static-nodes/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/static-nodes/up | 0
.../{ => image}/etc/s6-rc/sys-vmms/dependencies | 0
host/rootfs/{ => image}/etc/s6-rc/sys-vmms/type | 0
.../{ => image}/etc/s6-rc/sys-vmms/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/sys-vmms/up | 0
host/rootfs/{ => image}/etc/s6-rc/vm-env/contents | 0
host/rootfs/{ => image}/etc/s6-rc/vm-env/type | 0
.../{ => image}/etc/s6-rc/vm-env/type.license | 0
host/rootfs/{ => image}/etc/s6-rc/vmm-env/contents | 0
host/rootfs/{ => image}/etc/s6-rc/vmm-env/type | 0
.../{ => image}/etc/s6-rc/vmm-env/type.license | 0
.../{ => image}/etc/s6-rc/weston/dependencies | 0
.../{ => image}/etc/s6-rc/weston/notification-fd | 0
.../etc/s6-rc/weston/notification-fd.license | 0
host/rootfs/{ => image}/etc/s6-rc/weston/run | 0
host/rootfs/{ => image}/etc/s6-rc/weston/type | 0
.../{ => image}/etc/s6-rc/weston/type.license | 0
host/rootfs/{ => image}/etc/xdg/weston/autolaunch | 0
host/rootfs/{ => image}/etc/xdg/weston/weston.ini | 0
host/rootfs/{ => image}/lib | 0
host/rootfs/{ => image}/sbin | 0
host/rootfs/{ => image}/usr/bin/assign-devices | 0
.../{ => image}/usr/bin/create-vm-dependencies | 0
host/rootfs/{ => image}/usr/bin/run-appimage | 0
host/rootfs/{ => image}/usr/bin/run-vmm | 0
host/rootfs/{ => image}/usr/bin/vm-console | 0
host/rootfs/{ => image}/usr/bin/vm-import | 0
host/rootfs/{ => image}/usr/bin/vm-start | 0
host/rootfs/{ => image}/usr/bin/vm-stop | 0
host/rootfs/{ => image}/usr/bin/xdg-open | 0
.../org.freedesktop.portal.Documents.service | 0
img/app/Makefile | 98 +-------------
img/app/default.nix | 1 +
img/app/file-list.mk | 65 ++++++++++
img/app/{ => image}/bin | 0
img/app/{ => image}/etc/dbus-1/session.conf | 0
img/app/{ => image}/etc/fonts/fonts.conf | 0
img/app/{ => image}/etc/fstab | 0
img/app/{ => image}/etc/init | 0
img/app/{ => image}/etc/mdev.conf | 0
img/app/{ => image}/etc/mdev/iface | 0
img/app/{ => image}/etc/mdev/listen | 0
img/app/{ => image}/etc/mdev/virtiofs | 0
img/app/{ => image}/etc/mdev/wait | 0
img/app/{ => image}/etc/passwd | 0
img/app/{ => image}/etc/passwd.license | 0
img/app/{ => image}/etc/pipewire/pipewire.conf | 0
img/app/{ => image}/etc/resolv.conf | 0
.../etc/s6-linux-init/env/DBUS_SESSION_BUS_ADDRESS | 0
.../env/DBUS_SESSION_BUS_ADDRESS.license | 0
img/app/{ => image}/etc/s6-linux-init/env/DISPLAY | 0
.../etc/s6-linux-init/env/DISPLAY.license | 0
.../etc/s6-linux-init/env/GTK_USE_PORTAL | 0
.../etc/s6-linux-init/env/GTK_USE_PORTAL.license | 0
.../s6-linux-init/env/NIX_XDG_DESKTOP_PORTAL_DIR | 0
.../env/NIX_XDG_DESKTOP_PORTAL_DIR.license | 0
.../etc/s6-linux-init/env/WAYLAND_DISPLAY | 0
.../etc/s6-linux-init/env/WAYLAND_DISPLAY.license | 0
.../etc/s6-linux-init/env/XDG_RUNTIME_DIR | 0
.../etc/s6-linux-init/env/XDG_RUNTIME_DIR.license | 0
.../s6-linux-init/run-image/service/getty-hvc0/run | 0
.../s6-linux-init-shutdownd/notification-fd | 0
.../notification-fd.license | 0
.../run-image/service/s6-linux-init-shutdownd/run | 0
.../{ => image}/etc/s6-linux-init/scripts/rc.init | 2 +-
.../etc/s6-linux-init/scripts/rc.shutdown | 0
.../etc/s6-linux-init/scripts/rc.shutdown.final | 0
.../{ => image}/etc/s6-rc/app/dependencies.d/dbus | 0
.../etc/s6-rc/app/dependencies.d/pipewire | 0
.../s6-rc/app/dependencies.d/wayland-proxy-virtwl | 0
img/app/{ => image}/etc/s6-rc/app/run | 0
img/app/{ => image}/etc/s6-rc/app/type | 0
img/app/{ => image}/etc/s6-rc/app/type.license | 0
img/app/{ => image}/etc/s6-rc/dbus/notification-fd | 0
.../etc/s6-rc/dbus/notification-fd.license | 0
img/app/{ => image}/etc/s6-rc/dbus/run | 0
img/app/{ => image}/etc/s6-rc/dbus/type | 0
img/app/{ => image}/etc/s6-rc/dbus/type.license | 0
.../etc/s6-rc/mdevd-coldplug/dependencies | 0
img/app/{ => image}/etc/s6-rc/mdevd-coldplug/type | 0
.../etc/s6-rc/mdevd-coldplug/type.license | 0
img/app/{ => image}/etc/s6-rc/mdevd-coldplug/up | 0
.../{ => image}/etc/s6-rc/mdevd/notification-fd | 0
.../etc/s6-rc/mdevd/notification-fd.license | 0
img/app/{ => image}/etc/s6-rc/mdevd/run | 0
img/app/{ => image}/etc/s6-rc/mdevd/type | 0
img/app/{ => image}/etc/s6-rc/mdevd/type.license | 0
img/app/{ => image}/etc/s6-rc/ok-all/contents | 0
img/app/{ => image}/etc/s6-rc/ok-all/type | 0
img/app/{ => image}/etc/s6-rc/ok-all/type.license | 0
.../{ => image}/etc/s6-rc/pipewire/notification-fd | 0
.../etc/s6-rc/pipewire/notification-fd.license | 0
img/app/{ => image}/etc/s6-rc/pipewire/run | 0
img/app/{ => image}/etc/s6-rc/pipewire/type | 0
.../{ => image}/etc/s6-rc/pipewire/type.license | 0
.../etc/s6-rc/wayland-proxy-virtwl/notification-fd | 0
.../wayland-proxy-virtwl/notification-fd.license | 0
.../{ => image}/etc/s6-rc/wayland-proxy-virtwl/run | 0
.../etc/s6-rc/wayland-proxy-virtwl/type | 0
.../etc/s6-rc/wayland-proxy-virtwl/type.license | 0
.../etc/s6-rc/wireplumber/dependencies.d/dbus | 0
.../etc/s6-rc/wireplumber/dependencies.d/pipewire | 0
img/app/{ => image}/etc/s6-rc/wireplumber/run | 0
img/app/{ => image}/etc/s6-rc/wireplumber/type | 0
.../{ => image}/etc/s6-rc/wireplumber/type.license | 0
.../{ => image}/etc/ssl/certs/ca-certificates.crt | 0
.../wireplumber.conf.d/99_spectrum.conf | 0
.../etc/xdg/xdg-desktop-portal/portals.conf | 0
img/app/{ => image}/sbin | 0
lib/common.mk | 1 +
lib/erofs.mk | 51 ++++++++
scripts/genfiles.awk | 115 ++++++++++++++++
vm/sys/net/Makefile | 68 +---------
vm/sys/net/default.nix | 1 +
vm/sys/net/file-list.mk | 41 ++++++
vm/sys/net/{ => image}/bin | 0
vm/sys/net/{ => image}/etc/dbus-1/system.conf | 0
vm/sys/net/{ => image}/etc/fstab | 0
vm/sys/net/{ => image}/etc/init | 0
vm/sys/net/{ => image}/etc/mdev.conf | 0
vm/sys/net/{ => image}/etc/mdev/iface | 0
vm/sys/net/{ => image}/etc/nftables.conf | 0
vm/sys/net/{ => image}/etc/passwd | 0
vm/sys/net/{ => image}/etc/passwd.license | 0
.../s6-linux-init/run-image/service/getty-hvc0/run | 0
.../{ => image}/etc/s6-linux-init/scripts/rc.init | 2 +-
.../net/{ => image}/etc/s6-rc/connman/dependencies | 0
vm/sys/net/{ => image}/etc/s6-rc/connman/run | 0
vm/sys/net/{ => image}/etc/s6-rc/connman/type | 0
.../net/{ => image}/etc/s6-rc/connman/type.license | 0
.../net/{ => image}/etc/s6-rc/dbus/notification-fd | 0
.../etc/s6-rc/dbus/notification-fd.license | 0
vm/sys/net/{ => image}/etc/s6-rc/dbus/run | 0
vm/sys/net/{ => image}/etc/s6-rc/dbus/type | 0
vm/sys/net/{ => image}/etc/s6-rc/dbus/type.license | 0
.../etc/s6-rc/mdevd-coldplug/dependencies | 0
.../net/{ => image}/etc/s6-rc/mdevd-coldplug/type | 0
.../etc/s6-rc/mdevd-coldplug/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/mdevd-coldplug/up | 0
.../{ => image}/etc/s6-rc/mdevd/notification-fd | 0
.../etc/s6-rc/mdevd/notification-fd.license | 0
vm/sys/net/{ => image}/etc/s6-rc/mdevd/run | 0
vm/sys/net/{ => image}/etc/s6-rc/mdevd/type | 0
.../net/{ => image}/etc/s6-rc/mdevd/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/nftables/type | 0
.../{ => image}/etc/s6-rc/nftables/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/nftables/up | 0
vm/sys/net/{ => image}/etc/s6-rc/ok-all/contents | 0
vm/sys/net/{ => image}/etc/s6-rc/ok-all/type | 0
.../net/{ => image}/etc/s6-rc/ok-all/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/sysctl/type | 0
.../net/{ => image}/etc/s6-rc/sysctl/type.license | 0
vm/sys/net/{ => image}/etc/s6-rc/sysctl/up | 0
vm/sys/net/{ => image}/etc/sysctl.conf | 0
vm/sys/net/{ => image}/lib | 0
vm/sys/net/{ => image}/sbin | 0
vm/sys/net/{ => image}/var/run | 0
251 files changed, 400 insertions(+), 299 deletions(-)
---
base-commit: 0ac65013a1a29e91ea8476f39113e3598eb0e535
change-id: 20250831-genfiles-46787fc10869
--
Sincerely,
Demi Marie Obenour (she/her/hers)
next reply other threads:[~2025-09-04 2:09 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-04 1:56 Demi Marie Obenour [this message]
2025-09-04 1:56 ` [PATCH 1/4] Move all files for the image into a subdirectory Demi Marie Obenour
2025-09-04 1:56 ` [PATCH 2/4] Generate makefile file lists from a script Demi Marie Obenour
2025-09-08 9:59 ` Alyssa Ross
2025-09-08 18:45 ` Demi Marie Obenour
2025-09-09 14:51 ` Alyssa Ross
2025-09-04 1:56 ` [PATCH 3/4] Common make rules for building erofs images Demi Marie Obenour
2025-09-08 10:01 ` Alyssa Ross
2025-09-08 18:53 ` Demi Marie Obenour
2025-09-09 14:56 ` Alyssa Ross
2025-09-04 1:56 ` [PATCH 4/4] Use /etc/s6-rc/compiled for compiled s6-rc directory Demi Marie Obenour
2025-09-10 5:29 ` [PATCH v2 0/3] Generate file lists from a script Demi Marie Obenour
2025-09-10 5:29 ` [PATCH v2 1/3] Move all files for the image into a subdirectory Demi Marie Obenour
2025-09-10 18:58 ` Alyssa Ross
2025-09-11 12:21 ` Demi Marie Obenour
2025-09-10 5:29 ` [PATCH v2 2/3] Generate makefile file lists from a script Demi Marie Obenour
2025-09-10 5:29 ` [PATCH v2 3/3] Common make rules for building erofs images Demi Marie Obenour
2025-09-11 12:47 ` [PATCH v3 0/4] Generate file lists from a script Demi Marie Obenour
2025-09-11 12:47 ` [PATCH v3 1/4] Do not ignore errors from tar Demi Marie Obenour
2025-09-17 11:48 ` Alyssa Ross
2025-09-18 2:45 ` Demi Marie Obenour
2025-09-19 7:46 ` Alyssa Ross
2025-09-30 12:59 ` Alyssa Ross
2025-09-19 7:55 ` Alyssa Ross
2025-09-19 19:03 ` Demi Marie Obenour
2025-09-11 12:47 ` [PATCH v3 2/4] Move all files for the image into a subdirectory Demi Marie Obenour
2025-09-17 12:30 ` Alyssa Ross
2025-09-17 12:39 ` Alyssa Ross
2025-09-17 13:03 ` Alyssa Ross
2025-09-11 12:47 ` [PATCH v3 3/4] Generate makefile file lists from a script Demi Marie Obenour
2025-09-11 12:47 ` [PATCH v3 4/4] Common make rules for building erofs images Demi Marie Obenour
2025-09-21 2:23 ` [PATCH v3] Generate file lists from a script Demi Marie Obenour
2025-09-21 8:47 ` Alyssa Ross
2025-09-21 16:51 ` Demi Marie Obenour
2025-09-21 17:07 ` Alyssa Ross
2025-09-21 17:24 ` [PATCH v4] " Demi Marie Obenour
2025-09-25 11:22 ` Alyssa Ross
2025-09-26 16:31 ` [PATCH v5] " Demi Marie Obenour
2025-09-27 8:19 ` Alyssa Ross
2025-09-27 8:42 ` Demi Marie Obenour
2025-09-27 16:22 ` [PATCH v6] " Demi Marie Obenour
2025-09-29 8:12 ` Alyssa Ross
2025-09-29 17:20 ` Demi Marie Obenour
2025-09-29 17:18 ` [PATCH v7] " Demi Marie Obenour
2025-10-01 9:20 ` Alyssa Ross
2025-10-01 9:24 ` Demi Marie Obenour
2025-10-01 9:35 ` Alyssa Ross
2025-10-01 18:30 ` [PATCH v8] " Demi Marie Obenour
2025-10-02 9:46 ` Alyssa Ross
2025-10-02 17:37 ` [PATCH v9] " Demi Marie Obenour
2025-10-03 9:04 ` Alyssa Ross
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250903-genfiles-v1-0-cc993fcb1e4c@gmail.com \
--to=demiobenour@gmail.com \
--cc=devel@spectrum-os.org \
--cc=hi@alyssa.is \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://spectrum-os.org/git/crosvm
https://spectrum-os.org/git/doc
https://spectrum-os.org/git/mktuntap
https://spectrum-os.org/git/nixpkgs
https://spectrum-os.org/git/spectrum
https://spectrum-os.org/git/ucspi-vsock
https://spectrum-os.org/git/www
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).