patches and low-level development discussion
 help / color / mirror / code / Atom feed
* [PATCH 0/4] Generate file lists from a script
@ 2025-09-04  1:56 Demi Marie Obenour
  2025-09-04  1:56 ` [PATCH 1/4] Move all files for the image into a subdirectory Demi Marie Obenour
                   ` (4 more replies)
  0 siblings, 5 replies; 51+ messages in thread
From: Demi Marie Obenour @ 2025-09-04  1:56 UTC (permalink / raw)
  To: Spectrum OS Development; +Cc: Demi Marie Obenour, Alyssa Ross

Right now, the makefiles in host/rootfs, vm/sys/net, and img/app have
manually-maintained lists of files and symlinks.  These duplicate the
information in the git repository and can easily get out of sync or
cause unnecessary merge conflicts.

Avoid this problem by moving the files used in the erofs images to an
image/ directory and using what Git knows about this directory to
generate the lists via a script.  This makes maintenance easier and
avoids conflating code used in the build (anything not in image/) and
code for the image (anything that is in image/).

This reveals that there is a lot of duplicated make logic around image
generation, so consolidate it in a single makefile that is included by
all three of the makefiles.  This, in turn, makes tree-wide changes
easier, as shown by the final patch in this series.

Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
---
Demi Marie Obenour (4):
      Move all files for the image into a subdirectory
      Generate makefile file lists from a script
      Common make rules for building erofs images
      Use /etc/s6-rc/compiled for compiled s6-rc directory

 Documentation/development/built-in-vms.adoc        |   7 +
 host/rootfs/Makefile                               | 144 +--------------------
 host/rootfs/default.nix                            |   1 +
 host/rootfs/file-list.mk                           | 100 ++++++++++++++
 host/rootfs/{ => image}/bin                        |   0
 host/rootfs/{ => image}/etc/fonts/fonts.conf       |   0
 host/rootfs/{ => image}/etc/fstab                  |   0
 host/rootfs/{ => image}/etc/group                  |   0
 host/rootfs/{ => image}/etc/group.license          |   0
 host/rootfs/{ => image}/etc/init                   |   0
 host/rootfs/{ => image}/etc/login                  |   0
 host/rootfs/{ => image}/etc/mdev.conf              |   0
 host/rootfs/{ => image}/etc/mdev/listen            |   0
 host/rootfs/{ => image}/etc/mdev/net/add           |   0
 host/rootfs/{ => image}/etc/mdev/wait              |   0
 host/rootfs/{ => image}/etc/parse-devname          |   0
 host/rootfs/{ => image}/etc/passwd                 |   0
 host/rootfs/{ => image}/etc/passwd.license         |   0
 .../etc/s6-linux-init/env/WAYLAND_DISPLAY          |   0
 .../etc/s6-linux-init/env/WAYLAND_DISPLAY.license  |   0
 .../etc/s6-linux-init/env/XDG_RUNTIME_DIR          |   0
 .../etc/s6-linux-init/env/XDG_RUNTIME_DIR.license  |   0
 .../etc/s6-linux-init/run-image/opengl-driver      |   0
 .../run-image/service/dbus/notification-fd         |   0
 .../run-image/service/dbus/notification-fd.license |   0
 .../etc/s6-linux-init/run-image/service/dbus/run   |   0
 .../service/dbus/template/notification-fd          |   0
 .../service/dbus/template/notification-fd.license  |   0
 .../run-image/service/dbus/template/run            |   0
 .../s6-linux-init/run-image/service/getty-tty1/run |   0
 .../s6-linux-init/run-image/service/getty-tty2/run |   0
 .../s6-linux-init/run-image/service/getty-tty3/run |   0
 .../s6-linux-init/run-image/service/getty-tty4/run |   0
 .../service/s6-svscan-log/notification-fd          |   0
 .../service/s6-svscan-log/notification-fd.license  |   0
 .../run-image/service/s6-svscan-log/run            |   0
 .../run-image/service/serial-getty-generator/run   |   0
 .../run-image/service/serial-getty/notification-fd |   0
 .../service/serial-getty/notification-fd.license   |   0
 .../run-image/service/serial-getty/run             |   0
 .../run-image/service/serial-getty/template/run    |   0
 .../service/vhost-user-fs/notification-fd          |   0
 .../service/vhost-user-fs/notification-fd.license  |   0
 .../run-image/service/vhost-user-fs/run            |   0
 .../service/vhost-user-fs/template/notification-fd |   0
 .../vhost-user-fs/template/notification-fd.license |   0
 .../run-image/service/vhost-user-fs/template/run   |   0
 .../service/vhost-user-gpu/notification-fd         |   0
 .../service/vhost-user-gpu/notification-fd.license |   0
 .../run-image/service/vhost-user-gpu/run           |   0
 .../service/vhost-user-gpu/template/data/check     |   0
 .../vhost-user-gpu/template/notification-fd        |   0
 .../template/notification-fd.license               |   0
 .../run-image/service/vhost-user-gpu/template/run  |   0
 .../run-image/service/vhost-user-gpu/template/type |   0
 .../service/vhost-user-gpu/template/type.license   |   0
 .../run-image/service/vmm/notification-fd          |   0
 .../run-image/service/vmm/notification-fd.license  |   0
 .../etc/s6-linux-init/run-image/service/vmm/run    |   0
 .../run-image/service/vmm/template/notification-fd |   0
 .../service/vmm/template/notification-fd.license   |   0
 .../run-image/service/vmm/template/run             |   0
 .../notification-fd                                |   0
 .../notification-fd.license                        |   0
 .../service/xdg-desktop-portal-spectrum-host/run   |   0
 .../template/notification-fd                       |   0
 .../template/notification-fd.license               |   0
 .../xdg-desktop-portal-spectrum-host/template/run  |   0
 .../{ => image}/etc/s6-linux-init/scripts/rc.init  |   2 +-
 host/rootfs/{ => image}/etc/s6-rc/card0/type       |   0
 .../{ => image}/etc/s6-rc/card0/type.license       |   0
 host/rootfs/{ => image}/etc/s6-rc/card0/up         |   0
 host/rootfs/{ => image}/etc/s6-rc/core/type        |   0
 .../rootfs/{ => image}/etc/s6-rc/core/type.license |   0
 host/rootfs/{ => image}/etc/s6-rc/core/up          |   0
 host/rootfs/{ => image}/etc/s6-rc/kvm/timeout-up   |   0
 .../{ => image}/etc/s6-rc/kvm/timeout-up.license   |   0
 host/rootfs/{ => image}/etc/s6-rc/kvm/type         |   0
 host/rootfs/{ => image}/etc/s6-rc/kvm/type.license |   0
 host/rootfs/{ => image}/etc/s6-rc/kvm/up           |   0
 .../etc/s6-rc/mdevd-coldplug/dependencies          |   0
 .../{ => image}/etc/s6-rc/mdevd-coldplug/type      |   0
 .../etc/s6-rc/mdevd-coldplug/type.license          |   0
 .../rootfs/{ => image}/etc/s6-rc/mdevd-coldplug/up |   0
 .../{ => image}/etc/s6-rc/mdevd/notification-fd    |   0
 .../etc/s6-rc/mdevd/notification-fd.license        |   0
 host/rootfs/{ => image}/etc/s6-rc/mdevd/run        |   0
 host/rootfs/{ => image}/etc/s6-rc/mdevd/type       |   0
 .../{ => image}/etc/s6-rc/mdevd/type.license       |   0
 host/rootfs/{ => image}/etc/s6-rc/ok-all/contents  |   0
 host/rootfs/{ => image}/etc/s6-rc/ok-all/type      |   0
 .../{ => image}/etc/s6-rc/ok-all/type.license      |   0
 .../rootfs/{ => image}/etc/s6-rc/static-nodes/type |   0
 .../etc/s6-rc/static-nodes/type.license            |   0
 host/rootfs/{ => image}/etc/s6-rc/static-nodes/up  |   0
 .../{ => image}/etc/s6-rc/sys-vmms/dependencies    |   0
 host/rootfs/{ => image}/etc/s6-rc/sys-vmms/type    |   0
 .../{ => image}/etc/s6-rc/sys-vmms/type.license    |   0
 host/rootfs/{ => image}/etc/s6-rc/sys-vmms/up      |   0
 host/rootfs/{ => image}/etc/s6-rc/vm-env/contents  |   0
 host/rootfs/{ => image}/etc/s6-rc/vm-env/type      |   0
 .../{ => image}/etc/s6-rc/vm-env/type.license      |   0
 host/rootfs/{ => image}/etc/s6-rc/vmm-env/contents |   0
 host/rootfs/{ => image}/etc/s6-rc/vmm-env/type     |   0
 .../{ => image}/etc/s6-rc/vmm-env/type.license     |   0
 .../{ => image}/etc/s6-rc/weston/dependencies      |   0
 .../{ => image}/etc/s6-rc/weston/notification-fd   |   0
 .../etc/s6-rc/weston/notification-fd.license       |   0
 host/rootfs/{ => image}/etc/s6-rc/weston/run       |   0
 host/rootfs/{ => image}/etc/s6-rc/weston/type      |   0
 .../{ => image}/etc/s6-rc/weston/type.license      |   0
 host/rootfs/{ => image}/etc/xdg/weston/autolaunch  |   0
 host/rootfs/{ => image}/etc/xdg/weston/weston.ini  |   0
 host/rootfs/{ => image}/lib                        |   0
 host/rootfs/{ => image}/sbin                       |   0
 host/rootfs/{ => image}/usr/bin/assign-devices     |   0
 .../{ => image}/usr/bin/create-vm-dependencies     |   0
 host/rootfs/{ => image}/usr/bin/run-appimage       |   0
 host/rootfs/{ => image}/usr/bin/run-vmm            |   0
 host/rootfs/{ => image}/usr/bin/vm-console         |   0
 host/rootfs/{ => image}/usr/bin/vm-import          |   0
 host/rootfs/{ => image}/usr/bin/vm-start           |   0
 host/rootfs/{ => image}/usr/bin/vm-stop            |   0
 host/rootfs/{ => image}/usr/bin/xdg-open           |   0
 .../org.freedesktop.portal.Documents.service       |   0
 img/app/Makefile                                   |  98 +-------------
 img/app/default.nix                                |   1 +
 img/app/file-list.mk                               |  65 ++++++++++
 img/app/{ => image}/bin                            |   0
 img/app/{ => image}/etc/dbus-1/session.conf        |   0
 img/app/{ => image}/etc/fonts/fonts.conf           |   0
 img/app/{ => image}/etc/fstab                      |   0
 img/app/{ => image}/etc/init                       |   0
 img/app/{ => image}/etc/mdev.conf                  |   0
 img/app/{ => image}/etc/mdev/iface                 |   0
 img/app/{ => image}/etc/mdev/listen                |   0
 img/app/{ => image}/etc/mdev/virtiofs              |   0
 img/app/{ => image}/etc/mdev/wait                  |   0
 img/app/{ => image}/etc/passwd                     |   0
 img/app/{ => image}/etc/passwd.license             |   0
 img/app/{ => image}/etc/pipewire/pipewire.conf     |   0
 img/app/{ => image}/etc/resolv.conf                |   0
 .../etc/s6-linux-init/env/DBUS_SESSION_BUS_ADDRESS |   0
 .../env/DBUS_SESSION_BUS_ADDRESS.license           |   0
 img/app/{ => image}/etc/s6-linux-init/env/DISPLAY  |   0
 .../etc/s6-linux-init/env/DISPLAY.license          |   0
 .../etc/s6-linux-init/env/GTK_USE_PORTAL           |   0
 .../etc/s6-linux-init/env/GTK_USE_PORTAL.license   |   0
 .../s6-linux-init/env/NIX_XDG_DESKTOP_PORTAL_DIR   |   0
 .../env/NIX_XDG_DESKTOP_PORTAL_DIR.license         |   0
 .../etc/s6-linux-init/env/WAYLAND_DISPLAY          |   0
 .../etc/s6-linux-init/env/WAYLAND_DISPLAY.license  |   0
 .../etc/s6-linux-init/env/XDG_RUNTIME_DIR          |   0
 .../etc/s6-linux-init/env/XDG_RUNTIME_DIR.license  |   0
 .../s6-linux-init/run-image/service/getty-hvc0/run |   0
 .../s6-linux-init-shutdownd/notification-fd        |   0
 .../notification-fd.license                        |   0
 .../run-image/service/s6-linux-init-shutdownd/run  |   0
 .../{ => image}/etc/s6-linux-init/scripts/rc.init  |   2 +-
 .../etc/s6-linux-init/scripts/rc.shutdown          |   0
 .../etc/s6-linux-init/scripts/rc.shutdown.final    |   0
 .../{ => image}/etc/s6-rc/app/dependencies.d/dbus  |   0
 .../etc/s6-rc/app/dependencies.d/pipewire          |   0
 .../s6-rc/app/dependencies.d/wayland-proxy-virtwl  |   0
 img/app/{ => image}/etc/s6-rc/app/run              |   0
 img/app/{ => image}/etc/s6-rc/app/type             |   0
 img/app/{ => image}/etc/s6-rc/app/type.license     |   0
 img/app/{ => image}/etc/s6-rc/dbus/notification-fd |   0
 .../etc/s6-rc/dbus/notification-fd.license         |   0
 img/app/{ => image}/etc/s6-rc/dbus/run             |   0
 img/app/{ => image}/etc/s6-rc/dbus/type            |   0
 img/app/{ => image}/etc/s6-rc/dbus/type.license    |   0
 .../etc/s6-rc/mdevd-coldplug/dependencies          |   0
 img/app/{ => image}/etc/s6-rc/mdevd-coldplug/type  |   0
 .../etc/s6-rc/mdevd-coldplug/type.license          |   0
 img/app/{ => image}/etc/s6-rc/mdevd-coldplug/up    |   0
 .../{ => image}/etc/s6-rc/mdevd/notification-fd    |   0
 .../etc/s6-rc/mdevd/notification-fd.license        |   0
 img/app/{ => image}/etc/s6-rc/mdevd/run            |   0
 img/app/{ => image}/etc/s6-rc/mdevd/type           |   0
 img/app/{ => image}/etc/s6-rc/mdevd/type.license   |   0
 img/app/{ => image}/etc/s6-rc/ok-all/contents      |   0
 img/app/{ => image}/etc/s6-rc/ok-all/type          |   0
 img/app/{ => image}/etc/s6-rc/ok-all/type.license  |   0
 .../{ => image}/etc/s6-rc/pipewire/notification-fd |   0
 .../etc/s6-rc/pipewire/notification-fd.license     |   0
 img/app/{ => image}/etc/s6-rc/pipewire/run         |   0
 img/app/{ => image}/etc/s6-rc/pipewire/type        |   0
 .../{ => image}/etc/s6-rc/pipewire/type.license    |   0
 .../etc/s6-rc/wayland-proxy-virtwl/notification-fd |   0
 .../wayland-proxy-virtwl/notification-fd.license   |   0
 .../{ => image}/etc/s6-rc/wayland-proxy-virtwl/run |   0
 .../etc/s6-rc/wayland-proxy-virtwl/type            |   0
 .../etc/s6-rc/wayland-proxy-virtwl/type.license    |   0
 .../etc/s6-rc/wireplumber/dependencies.d/dbus      |   0
 .../etc/s6-rc/wireplumber/dependencies.d/pipewire  |   0
 img/app/{ => image}/etc/s6-rc/wireplumber/run      |   0
 img/app/{ => image}/etc/s6-rc/wireplumber/type     |   0
 .../{ => image}/etc/s6-rc/wireplumber/type.license |   0
 .../{ => image}/etc/ssl/certs/ca-certificates.crt  |   0
 .../wireplumber.conf.d/99_spectrum.conf            |   0
 .../etc/xdg/xdg-desktop-portal/portals.conf        |   0
 img/app/{ => image}/sbin                           |   0
 lib/common.mk                                      |   1 +
 lib/erofs.mk                                       |  51 ++++++++
 scripts/genfiles.awk                               | 115 ++++++++++++++++
 vm/sys/net/Makefile                                |  68 +---------
 vm/sys/net/default.nix                             |   1 +
 vm/sys/net/file-list.mk                            |  41 ++++++
 vm/sys/net/{ => image}/bin                         |   0
 vm/sys/net/{ => image}/etc/dbus-1/system.conf      |   0
 vm/sys/net/{ => image}/etc/fstab                   |   0
 vm/sys/net/{ => image}/etc/init                    |   0
 vm/sys/net/{ => image}/etc/mdev.conf               |   0
 vm/sys/net/{ => image}/etc/mdev/iface              |   0
 vm/sys/net/{ => image}/etc/nftables.conf           |   0
 vm/sys/net/{ => image}/etc/passwd                  |   0
 vm/sys/net/{ => image}/etc/passwd.license          |   0
 .../s6-linux-init/run-image/service/getty-hvc0/run |   0
 .../{ => image}/etc/s6-linux-init/scripts/rc.init  |   2 +-
 .../net/{ => image}/etc/s6-rc/connman/dependencies |   0
 vm/sys/net/{ => image}/etc/s6-rc/connman/run       |   0
 vm/sys/net/{ => image}/etc/s6-rc/connman/type      |   0
 .../net/{ => image}/etc/s6-rc/connman/type.license |   0
 .../net/{ => image}/etc/s6-rc/dbus/notification-fd |   0
 .../etc/s6-rc/dbus/notification-fd.license         |   0
 vm/sys/net/{ => image}/etc/s6-rc/dbus/run          |   0
 vm/sys/net/{ => image}/etc/s6-rc/dbus/type         |   0
 vm/sys/net/{ => image}/etc/s6-rc/dbus/type.license |   0
 .../etc/s6-rc/mdevd-coldplug/dependencies          |   0
 .../net/{ => image}/etc/s6-rc/mdevd-coldplug/type  |   0
 .../etc/s6-rc/mdevd-coldplug/type.license          |   0
 vm/sys/net/{ => image}/etc/s6-rc/mdevd-coldplug/up |   0
 .../{ => image}/etc/s6-rc/mdevd/notification-fd    |   0
 .../etc/s6-rc/mdevd/notification-fd.license        |   0
 vm/sys/net/{ => image}/etc/s6-rc/mdevd/run         |   0
 vm/sys/net/{ => image}/etc/s6-rc/mdevd/type        |   0
 .../net/{ => image}/etc/s6-rc/mdevd/type.license   |   0
 vm/sys/net/{ => image}/etc/s6-rc/nftables/type     |   0
 .../{ => image}/etc/s6-rc/nftables/type.license    |   0
 vm/sys/net/{ => image}/etc/s6-rc/nftables/up       |   0
 vm/sys/net/{ => image}/etc/s6-rc/ok-all/contents   |   0
 vm/sys/net/{ => image}/etc/s6-rc/ok-all/type       |   0
 .../net/{ => image}/etc/s6-rc/ok-all/type.license  |   0
 vm/sys/net/{ => image}/etc/s6-rc/sysctl/type       |   0
 .../net/{ => image}/etc/s6-rc/sysctl/type.license  |   0
 vm/sys/net/{ => image}/etc/s6-rc/sysctl/up         |   0
 vm/sys/net/{ => image}/etc/sysctl.conf             |   0
 vm/sys/net/{ => image}/lib                         |   0
 vm/sys/net/{ => image}/sbin                        |   0
 vm/sys/net/{ => image}/var/run                     |   0
 251 files changed, 400 insertions(+), 299 deletions(-)
---
base-commit: 0ac65013a1a29e91ea8476f39113e3598eb0e535
change-id: 20250831-genfiles-46787fc10869
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)


^ permalink raw reply	[flat|nested] 51+ messages in thread

end of thread, other threads:[~2025-10-03  9:04 UTC | newest]

Thread overview: 51+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-04  1:56 [PATCH 0/4] Generate file lists from a script Demi Marie Obenour
2025-09-04  1:56 ` [PATCH 1/4] Move all files for the image into a subdirectory Demi Marie Obenour
2025-09-04  1:56 ` [PATCH 2/4] Generate makefile file lists from a script Demi Marie Obenour
2025-09-08  9:59   ` Alyssa Ross
2025-09-08 18:45     ` Demi Marie Obenour
2025-09-09 14:51       ` Alyssa Ross
2025-09-04  1:56 ` [PATCH 3/4] Common make rules for building erofs images Demi Marie Obenour
2025-09-08 10:01   ` Alyssa Ross
2025-09-08 18:53     ` Demi Marie Obenour
2025-09-09 14:56       ` Alyssa Ross
2025-09-04  1:56 ` [PATCH 4/4] Use /etc/s6-rc/compiled for compiled s6-rc directory Demi Marie Obenour
2025-09-10  5:29 ` [PATCH v2 0/3] Generate file lists from a script Demi Marie Obenour
2025-09-10  5:29   ` [PATCH v2 1/3] Move all files for the image into a subdirectory Demi Marie Obenour
2025-09-10 18:58     ` Alyssa Ross
2025-09-11 12:21       ` Demi Marie Obenour
2025-09-10  5:29   ` [PATCH v2 2/3] Generate makefile file lists from a script Demi Marie Obenour
2025-09-10  5:29   ` [PATCH v2 3/3] Common make rules for building erofs images Demi Marie Obenour
2025-09-11 12:47   ` [PATCH v3 0/4] Generate file lists from a script Demi Marie Obenour
2025-09-11 12:47     ` [PATCH v3 1/4] Do not ignore errors from tar Demi Marie Obenour
2025-09-17 11:48       ` Alyssa Ross
2025-09-18  2:45         ` Demi Marie Obenour
2025-09-19  7:46           ` Alyssa Ross
2025-09-30 12:59             ` Alyssa Ross
2025-09-19  7:55       ` Alyssa Ross
2025-09-19 19:03         ` Demi Marie Obenour
2025-09-11 12:47     ` [PATCH v3 2/4] Move all files for the image into a subdirectory Demi Marie Obenour
2025-09-17 12:30       ` Alyssa Ross
2025-09-17 12:39       ` Alyssa Ross
2025-09-17 13:03       ` Alyssa Ross
2025-09-11 12:47     ` [PATCH v3 3/4] Generate makefile file lists from a script Demi Marie Obenour
2025-09-11 12:47     ` [PATCH v3 4/4] Common make rules for building erofs images Demi Marie Obenour
2025-09-21  2:23   ` [PATCH v3] Generate file lists from a script Demi Marie Obenour
2025-09-21  8:47     ` Alyssa Ross
2025-09-21 16:51       ` Demi Marie Obenour
2025-09-21 17:07         ` Alyssa Ross
2025-09-21 17:24     ` [PATCH v4] " Demi Marie Obenour
2025-09-25 11:22       ` Alyssa Ross
2025-09-26 16:31       ` [PATCH v5] " Demi Marie Obenour
2025-09-27  8:19         ` Alyssa Ross
2025-09-27  8:42           ` Demi Marie Obenour
2025-09-27 16:22         ` [PATCH v6] " Demi Marie Obenour
2025-09-29  8:12           ` Alyssa Ross
2025-09-29 17:20             ` Demi Marie Obenour
2025-09-29 17:18           ` [PATCH v7] " Demi Marie Obenour
2025-10-01  9:20             ` Alyssa Ross
2025-10-01  9:24               ` Demi Marie Obenour
2025-10-01  9:35             ` Alyssa Ross
2025-10-01 18:30             ` [PATCH v8] " Demi Marie Obenour
2025-10-02  9:46               ` Alyssa Ross
2025-10-02 17:37               ` [PATCH v9] " Demi Marie Obenour
2025-10-03  9:04                 ` Alyssa Ross

Code repositories for project(s) associated with this public inbox

	https://spectrum-os.org/git/crosvm
	https://spectrum-os.org/git/doc
	https://spectrum-os.org/git/mktuntap
	https://spectrum-os.org/git/nixpkgs
	https://spectrum-os.org/git/spectrum
	https://spectrum-os.org/git/ucspi-vsock
	https://spectrum-os.org/git/www

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).