From: Alyssa Ross <hi@alyssa.is>
To: Demi Marie Obenour <demiobenour@gmail.com>
Cc: Spectrum OS Development <devel@spectrum-os.org>
Subject: Re: [PATCH 3/4] Common make rules for building erofs images
Date: Mon, 08 Sep 2025 12:01:38 +0200 [thread overview]
Message-ID: <87wm69i7p9.fsf@alyssa.is> (raw)
In-Reply-To: <20250903-genfiles-v1-3-cc993fcb1e4c@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 12191 bytes --]
Demi Marie Obenour <demiobenour@gmail.com> writes:
> Instead of duplicating the logic in three different places, write common
> code for all three makefiles to use instead. This will make future
> changes much easier.
>
> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
Makes sense, and we could do this now for the s6-rc directory without
having to wait for the generated file list stuff. Does need to be
updated for my review comments from the previous patch though (stick to
POSIX).
> ---
> host/rootfs/Makefile | 51 ++++---------------------------------------------
> host/rootfs/default.nix | 1 +
> img/app/Makefile | 42 ++++------------------------------------
> img/app/default.nix | 1 +
> lib/erofs.mk | 51 +++++++++++++++++++++++++++++++++++++++++++++++++
> vm/sys/net/Makefile | 35 +++------------------------------
> vm/sys/net/default.nix | 1 +
> 7 files changed, 65 insertions(+), 117 deletions(-)
>
> diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile
> index 3560707d21bcf53e4f3ad5e916d21e8be56cc3a1..dce8315fe5be19d2569d6c9a429276e4abd696ad 100644
> --- a/host/rootfs/Makefile
> +++ b/host/rootfs/Makefile
> @@ -4,11 +4,10 @@
> .POSIX:
>
> include ../../lib/common.mk
> +include ../../lib/erofs.mk
> include file-list.mk
>
> -dest = build/rootfs.erofs
> -
> -DIRS = \
> +override DIRS ::= \
> dev \
> etc/s6-linux-init/env \
> etc/s6-linux-init/run-image/configs \
> @@ -42,46 +41,9 @@ DIRS = \
> proc \
> sys
>
> -FIFOS = etc/s6-linux-init/run-image/service/s6-svscan-log/fifo
> -
> -BUILD_FILES = build/etc/s6-rc
> -
> -$(dest): ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(addprefix image/,$(FILES)) $(BUILD_FILES) build/empty build/fifo file-list.mk
> - ( \
> - cat $(PACKAGES_FILE) ;\
> - for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' $$file $$file; done ;\
> - for file in $(BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\
> - printf 'build/empty\n%s\n' $(DIRS) ;\
> - printf 'build/fifo\n%s\n' $(FIFOS) ;\
> - ) | ../../scripts/make-erofs.sh $@
> -
> -build/fifo:
> - mkdir -p build
> - mkfifo -m 0600 $@
> -
> -build/empty:
> - mkdir -p $@
> +override FIFOS ::= etc/s6-linux-init/run-image/service/s6-svscan-log/fifo
>
> -# s6-rc-compile's input is a directory, but that doesn't play nice
> -# with Make, because it won't know to update if some file in the
> -# directory is changed, or a file is created or removed in a
> -# subdirectory. Using the whole source directory could also end up
> -# including files that aren't intended to be part of the input, like
> -# temporary editor files or .license files. So for all these reasons,
> -# only explicitly listed files are made available to s6-rc-compile.
> -build/etc/s6-rc: $(addprefix image/,$(S6_RC_FILES)) file-list.mk
> - mkdir -p $$(dirname $@)
> - rm -rf $@
> -
> - dir=$$(mktemp -d) && \
> - tar -C image -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 2 && \
> - s6-rc-compile $@ $$dir; \
> - exit=$$?; rm -r $$dir; exit $$exit
> -
> -clean:
> - -chmod -Rf +w build
> - rm -rf build
> -.PHONY: clean
> +all: $(dest)
>
> # veritysetup format produces two files, but Make only (portably)
> # supports one output per rule, so we combine the two outputs then
> @@ -111,11 +73,6 @@ debug:
> $(VMLINUX)
> .PHONY: debug
>
> -update-file-list:
> - ../../scripts/genfiles.awk image > file-list.mk
> -
> -.PHONY: update-file-list
> -
> run: build/live.img $(EXT_FS) build/rootfs.verity.roothash
> @set -x && \
> ext="$$(mktemp build/spectrum-rootfs-extfs.XXXXXXXXXX.img)" && \
> diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix
> index 998220d7b6ed322f64ee52c704e71ec9b4643f59..561bf660829dcd5f5e2ee841662727b262d43ed3 100644
> --- a/host/rootfs/default.nix
> +++ b/host/rootfs/default.nix
> @@ -178,6 +178,7 @@ stdenvNoCC.mkDerivation {
> fileset = fileset.intersection src (fileset.unions [
> ./.
> ../../lib/common.mk
> + ../../lib/erofs.mk
> ../../scripts/make-erofs.sh
> ]);
> };
> diff --git a/img/app/Makefile b/img/app/Makefile
> index 5bb1a6a2f9acd13aba95abb0e918a7f21943b230..0c3a8311973f5f82eb2af6ea3ba43f67d36dec2e 100644
> --- a/img/app/Makefile
> +++ b/img/app/Makefile
> @@ -5,6 +5,7 @@
> .POSIX:
>
> include ../../lib/common.mk
> +include ../../lib/erofs.mk
> include file-list.mk
>
> prefix = build/host
> @@ -18,7 +19,6 @@ HOST_BUILD_FILES = \
> $(imgdir)/appvm/vmlinux
>
> all: $(HOST_BUILD_FILES)
> -.PHONY: all
>
> $(imgdir)/appvm/vmlinux: $(KERNEL)
> mkdir -p $$(dirname $@)
> @@ -30,38 +30,13 @@ $(imgdir)/appvm/blk/root.img: ../../scripts/make-gpt.sh ../../scripts/sfdisk-fie
> build/rootfs.erofs:root:5460386f-2203-4911-8694-91400125c604:root
> mv $@.tmp $@
>
> -VM_DIRS = dev run proc sys tmp \
> +DIRS = dev run proc sys tmp \
> etc/s6-linux-init/run-image/service \
> etc/s6-linux-init/run-image/user \
> etc/s6-linux-init/run-image/wait
> -VM_FIFOS = etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/fifo
> +FIFOS = etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/fifo
>
> -VM_BUILD_FILES = build/etc/s6-rc
> -
> -build/fifo:
> - mkdir -p build
> - mkfifo -m 0600 $@
> -
> -build/empty:
> - mkdir -p $@
> -
> -build/rootfs.erofs: ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(addprefix image/,$(FILES)) $(VM_BUILD_FILES) build/empty build/fifo file-list.mk
> - ( \
> - cat $(PACKAGES_FILE) ;\
> - for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' $$file $$file; done ;\
> - for file in $(VM_BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\
> - printf 'build/empty\n%s\n' $(VM_DIRS) ;\
> - printf 'build/fifo\n%s\n' $(VM_FIFOS) ;\
> - ) | ../../scripts/make-erofs.sh $@
> -
> -build/etc/s6-rc: $(addprefix image/,$(S6_RC_FILES)) file-list.mk
> - mkdir -p $$(dirname $@)
> - rm -rf $@
> -
> - dir=$$(mktemp -d) && \
> - tar -C image -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 2 && \
> - s6-rc-compile $@ $$dir; \
> - exit=$$?; rm -r $$dir; exit $$exit
> +BUILD_FILES =
>
> debug:
> $(GDB) -q \
> @@ -138,14 +113,5 @@ run-crosvm: $(imgdir)/appvm/blk/root.img start-vhost-user-gpu start-virtiofsd
> $(KERNEL)
> .PHONY: run-crosvm
>
> -update-file-list:
> - ../../scripts/genfiles.awk image > file-list.mk
> -
> -.PHONY: update-file-list
> -
> run: run-$(VMM)
> .PHONY: run
> -
> -clean:
> - rm -rf build
> -.PHONY: clean
> diff --git a/img/app/default.nix b/img/app/default.nix
> index d3eed1f0accdc8968d1ba5bdec74ab597789082f..cf10e712ab84a9e85cb1373024be5f64deef0370 100644
> --- a/img/app/default.nix
> +++ b/img/app/default.nix
> @@ -107,6 +107,7 @@ stdenvNoCC.mkDerivation {
> fileset = lib.fileset.intersection src (lib.fileset.unions [
> ./.
> ../../lib/common.mk
> + ../../lib/erofs.mk
> ../../scripts/make-erofs.sh
> ../../scripts/make-gpt.sh
> ../../scripts/sfdisk-field.awk
> diff --git a/lib/erofs.mk b/lib/erofs.mk
> new file mode 100644
> index 0000000000000000000000000000000000000000..b3fc112f5e793725977cd8c4b2e71d6ed8d888c4
> --- /dev/null
> +++ b/lib/erofs.mk
> @@ -0,0 +1,51 @@
> +override basedir ::= $(dir $(lastword $(MAKEFILE_LIST)))/..
> +
> +override BUILD_FILES ::= $(BUILD_FILES) build/etc/s6-rc
> +# No override here so that it can be overridden in host/rootfs/default.nix.
> +dest ::= build/rootfs.erofs
> +
> +all:
> +.PHONY: all
> +$(dest): $(basedir)/scripts/make-erofs.sh $(PACKAGES_FILE) $(addprefix image/,$(FILES)) $(BUILD_FILES) build/empty build/fifo file-list.mk
> + set -euo pipefail; ( \
> + cat $(PACKAGES_FILE) ;\
> + for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' "$$file" "$$file"; done ;\
> + for file in $(BUILD_FILES); do printf '%s\n%s\n' "$$file" "$${file#build/}"; done ;\
> + $(and $(DIRS),printf 'build/empty\n%s\n' $(DIRS);)\
> + $(and $(FIFOS),printf 'build/fifo\n%s\n' $(FIFOS);)\
> + ) | $(basedir)/scripts/make-erofs.sh $@
> +
> +clean:
> + -chmod -Rf +w build
> + rm -rf build
> +.PHONY: clean
> +
> +build/fifo:
> + mkdir -p build
> + mkfifo -m 0600 $@
> +
> +build/empty:
> + mkdir -p $@
> +
> +update-file-list:
> + $(basedir)/scripts/genfiles.awk image > file-list.mk
> +
> +.PHONY: update-file-list
> +.PHONY:
> + $(error cannot make .PHONY!)
> +
> +# s6-rc-compile's input is a directory, but that doesn't play nice
> +# with Make, because it won't know to update if some file in the
> +# directory is changed, or a file is created or removed in a
> +# subdirectory. Using the whole source directory could also end up
> +# including files that aren't intended to be part of the input, like
> +# temporary editor files or .license files. So for all these reasons,
> +# only explicitly listed files are made available to s6-rc-compile.
> +build/etc/s6-rc: $(addprefix image/,$(S6_RC_FILES)) file-list.mk
> + mkdir -p build/etc
> + rm -rf $@
> + set -euo pipefail; \
> + dir=$$(mktemp -d); \
> + { tar -C image -c $(S6_RC_FILES) | tar -C "$$dir" -x --strip-components 2;}; \
> + s6-rc-compile $@ $$dir; \
> + exit=$$?; rm -r $$dir; exit $$exit
> diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile
> index e731bcea7fc61f9b2019689e52a600f3741dd55f..3f8493b9bf2364f7b2d01b7f114dba72126965cc 100644
> --- a/vm/sys/net/Makefile
> +++ b/vm/sys/net/Makefile
> @@ -4,6 +4,7 @@
> .POSIX:
>
> include ../../../lib/common.mk
> +include ../../../lib/erofs.mk
> include file-list.mk
>
> prefix = build/host
> @@ -11,13 +12,14 @@ libdir = $(prefix)/lib
> vmdir = $(libdir)/spectrum/vm
>
> VMM = cloud-hypervisor
> +override DIRS ::=
> +override FIFOS ::=
>
> HOST_BUILD_FILES = \
> $(vmdir)/netvm/blk/root.img \
> $(vmdir)/netvm/vmlinux
>
> all: $(HOST_BUILD_FILES)
> -.PHONY: all
>
> $(vmdir)/netvm/vmlinux: $(KERNEL)
> mkdir -p $$(dirname $@)
> @@ -29,28 +31,6 @@ $(vmdir)/netvm/blk/root.img: ../../../scripts/make-gpt.sh ../../../scripts/sfdis
> build/rootfs.erofs:root:ea21da27-0391-48da-9235-9d2ab2ca7844:root
> mv $@.tmp $@
>
> -VM_BUILD_FILES = build/etc/s6-rc
> -
> -build/empty:
> - mkdir -p $@
> -
> -build/rootfs.erofs: ../../../scripts/make-erofs.sh $(PACKAGES_FILE) $(addprefix image/,$(FILES)) $(VM_BUILD_FILES) build/empty file-list.mk
> - ( \
> - cat $(PACKAGES_FILE) ;\
> - for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' $$file $$file; done ;\
> - for file in $(VM_BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\
> - printf 'build/empty\n%s\n' $(VM_DIRS) ;\
> - ) | ../../../scripts/make-erofs.sh $@
> -
> -build/etc/s6-rc: $(addprefix image/,$(S6_RC_FILES)) file-list.mk
> - mkdir -p $$(dirname $@)
> - rm -rf $@
> -
> - dir=$$(mktemp -d) && \
> - tar -C image -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 2 && \
> - s6-rc-compile $@ $$dir; \
> - exit=$$?; rm -r $$dir; exit $$exit
> -
> debug:
> $(GDB) -q \
> -ex 'set substitute-path .. $(LINUX_SRC)' \
> @@ -102,14 +82,5 @@ run-crosvm: $(vmdir)/netvm/blk/root.img
> $(KERNEL)
> .PHONY: run-crosvm
>
> -update-file-list:
> - ../../scripts/genfiles.awk image > file-list.mk
> -
> -.PHONY: update-file-list
> -
> run: run-$(VMM)
> .PHONY: run
> -
> -clean:
> - rm -rf build
> -.PHONY: clean
> diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix
> index b5873ebe1e80dd88c1ba997f7ebd3ee7369bb40f..7b03264bd30fe2e9157730e5338cd2501b255f7e 100644
> --- a/vm/sys/net/default.nix
> +++ b/vm/sys/net/default.nix
> @@ -104,6 +104,7 @@ stdenvNoCC.mkDerivation {
> fileset = lib.fileset.intersection src (lib.fileset.unions [
> ./.
> ../../../lib/common.mk
> + ../../../lib/erofs.mk
> ../../../scripts/make-erofs.sh
> ../../../scripts/make-gpt.sh
> ../../../scripts/sfdisk-field.awk
>
> --
> 2.51.0
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
next prev parent reply other threads:[~2025-09-08 10:01 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-04 1:56 [PATCH 0/4] Generate file lists from a script Demi Marie Obenour
2025-09-04 1:56 ` [PATCH 1/4] Move all files for the image into a subdirectory Demi Marie Obenour
2025-09-04 1:56 ` [PATCH 2/4] Generate makefile file lists from a script Demi Marie Obenour
2025-09-08 9:59 ` Alyssa Ross
2025-09-08 18:45 ` Demi Marie Obenour
2025-09-09 14:51 ` Alyssa Ross
2025-09-04 1:56 ` [PATCH 3/4] Common make rules for building erofs images Demi Marie Obenour
2025-09-08 10:01 ` Alyssa Ross [this message]
2025-09-08 18:53 ` Demi Marie Obenour
2025-09-09 14:56 ` Alyssa Ross
2025-09-04 1:56 ` [PATCH 4/4] Use /etc/s6-rc/compiled for compiled s6-rc directory Demi Marie Obenour
2025-09-10 5:29 ` [PATCH v2 0/3] Generate file lists from a script Demi Marie Obenour
2025-09-10 5:29 ` [PATCH v2 1/3] Move all files for the image into a subdirectory Demi Marie Obenour
2025-09-10 18:58 ` Alyssa Ross
2025-09-11 12:21 ` Demi Marie Obenour
2025-09-10 5:29 ` [PATCH v2 2/3] Generate makefile file lists from a script Demi Marie Obenour
2025-09-10 5:29 ` [PATCH v2 3/3] Common make rules for building erofs images Demi Marie Obenour
2025-09-11 12:47 ` [PATCH v3 0/4] Generate file lists from a script Demi Marie Obenour
2025-09-11 12:47 ` [PATCH v3 1/4] Do not ignore errors from tar Demi Marie Obenour
2025-09-17 11:48 ` Alyssa Ross
2025-09-18 2:45 ` Demi Marie Obenour
2025-09-19 7:46 ` Alyssa Ross
2025-09-30 12:59 ` Alyssa Ross
2025-09-19 7:55 ` Alyssa Ross
2025-09-19 19:03 ` Demi Marie Obenour
2025-09-11 12:47 ` [PATCH v3 2/4] Move all files for the image into a subdirectory Demi Marie Obenour
2025-09-17 12:30 ` Alyssa Ross
2025-09-17 12:39 ` Alyssa Ross
2025-09-17 13:03 ` Alyssa Ross
2025-09-11 12:47 ` [PATCH v3 3/4] Generate makefile file lists from a script Demi Marie Obenour
2025-09-11 12:47 ` [PATCH v3 4/4] Common make rules for building erofs images Demi Marie Obenour
2025-09-21 2:23 ` [PATCH v3] Generate file lists from a script Demi Marie Obenour
2025-09-21 8:47 ` Alyssa Ross
2025-09-21 16:51 ` Demi Marie Obenour
2025-09-21 17:07 ` Alyssa Ross
2025-09-21 17:24 ` [PATCH v4] " Demi Marie Obenour
2025-09-25 11:22 ` Alyssa Ross
2025-09-26 16:31 ` [PATCH v5] " Demi Marie Obenour
2025-09-27 8:19 ` Alyssa Ross
2025-09-27 8:42 ` Demi Marie Obenour
2025-09-27 16:22 ` [PATCH v6] " Demi Marie Obenour
2025-09-29 8:12 ` Alyssa Ross
2025-09-29 17:20 ` Demi Marie Obenour
2025-09-29 17:18 ` [PATCH v7] " Demi Marie Obenour
2025-10-01 9:20 ` Alyssa Ross
2025-10-01 9:24 ` Demi Marie Obenour
2025-10-01 9:35 ` Alyssa Ross
2025-10-01 18:30 ` [PATCH v8] " Demi Marie Obenour
2025-10-02 9:46 ` Alyssa Ross
2025-10-02 17:37 ` [PATCH v9] " Demi Marie Obenour
2025-10-03 9:04 ` Alyssa Ross
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87wm69i7p9.fsf@alyssa.is \
--to=hi@alyssa.is \
--cc=demiobenour@gmail.com \
--cc=devel@spectrum-os.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://spectrum-os.org/git/crosvm
https://spectrum-os.org/git/doc
https://spectrum-os.org/git/mktuntap
https://spectrum-os.org/git/nixpkgs
https://spectrum-os.org/git/spectrum
https://spectrum-os.org/git/ucspi-vsock
https://spectrum-os.org/git/www
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).