From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id A571E1A0B9; Mon, 08 Sep 2025 10:01:47 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id AACD71A131; Mon, 08 Sep 2025 10:01:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_MISSING,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=4.0.1 Received: from fhigh-b8-smtp.messagingengine.com (fhigh-b8-smtp.messagingengine.com [202.12.124.159]) by atuin.qyliss.net (Postfix) with ESMTPS id 247041A12D for ; Mon, 08 Sep 2025 10:01:44 +0000 (UTC) Received: from phl-compute-12.internal (phl-compute-12.internal [10.202.2.52]) by mailfhigh.stl.internal (Postfix) with ESMTP id 13E1E7A0069; Mon, 8 Sep 2025 06:01:43 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-12.internal (MEProxy); Mon, 08 Sep 2025 06:01:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alyssa.is; h=cc :cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1757325702; x=1757412102; bh=fG8qcSLUcO MRhSXw3coTLTHKjiV/jAL3HTppP5kJ+SQ=; b=KXW7nb4TyEoRNwn5uUpTtGUivN d9tiibfOr6Gu7si0AIMaLYwtAsTdDSwr+o308UeJZnS4Ewy+zQg/u7z5yOkYLFNb uTUgxRnLFRAfCHYEWVoWdT9UqPXQYnftwdcqlcl9PPJ5ZyTX+sq+Gekso7BpbXRz fQB+7hsJTGp2NxdLlfJ2NTs9/C3EeHZLnlH+Nu3dI6Rkfa6GS4UOLntdoYBCx36E BmeigUgH9bEdtrnvF7pk7ZH0487zOphHSEje/WZe+eO45HpTq5Ee5JVY8ocHG7EP Io3q6xksPUdIqrRqFJBbwa1tI+PD+HC41TuoTvqyeQ5pyswcodJVbCNxnF5A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1757325702; x=1757412102; bh=fG8qcSLUcOMRhSXw3coTLTHKjiV/jAL3HTp pP5kJ+SQ=; b=WBB9ujZ/dQ5TXatmBM2evdwyGC/hqIZBqU0Ej/QA5VpxYvO9L8l Ma4ax9ofhQUCUC4muqTQVMzc4ugKRNi2pSWZt39eN5xfvWt97PIeG+1jZICRmtdd PDYcW9jHc5g3PuKfYrBOcj5Opa0WavDB5yNl4fI7dFgqdBu/5sOVn3te4gTz4iXh Oao+XG4gwC94g9bGdzSLF4hAjTW9JI/iZ1c2Cxvrd2wX1JEjQyPKG9P+Xyn8dpRB q2wXzFTlsHXZm8WTBmHwcx94PW1fNI+WsYo/ePu06VRQKSRBSW0RrebAJ6k04ii5 nrht6SimgEMGB/KNx3yv2PKZ4hh3YghqX/w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggddujedvgecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufgjfhffkfggtgesghdtreertddttdenucfhrhhomheptehlhihsshgrucft ohhsshcuoehhihesrghlhihsshgrrdhisheqnecuggftrfgrthhtvghrnhepvdetfeffje evffevgffffeekheegteehfedtgeehgeeilefhffevgfdtkeevudehnecuffhomhgrihhn pehsiedqrhgtqdgtohhmphhilhgvrdgsuhhilhgunecuvehluhhsthgvrhfuihiivgeptd enucfrrghrrghmpehmrghilhhfrhhomhephhhisegrlhihshhsrgdrihhspdhnsggprhgt phhtthhopedvpdhmohguvgepshhmthhpohhuthdprhgtphhtthhopeguvghmihhosggvnh houhhrsehgmhgrihhlrdgtohhmpdhrtghpthhtohepuggvvhgvlhesshhpvggtthhruhhm qdhoshdrohhrgh X-ME-Proxy: Feedback-ID: i12284293:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 8 Sep 2025 06:01:42 -0400 (EDT) Received: by mbp.qyliss.net (Postfix, from userid 1000) id 9D58A1675B22; Mon, 08 Sep 2025 12:01:39 +0200 (CEST) From: Alyssa Ross To: Demi Marie Obenour Subject: Re: [PATCH 3/4] Common make rules for building erofs images In-Reply-To: <20250903-genfiles-v1-3-cc993fcb1e4c@gmail.com> References: <20250903-genfiles-v1-0-cc993fcb1e4c@gmail.com> <20250903-genfiles-v1-3-cc993fcb1e4c@gmail.com> Date: Mon, 08 Sep 2025 12:01:38 +0200 Message-ID: <87wm69i7p9.fsf@alyssa.is> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Message-ID-Hash: B27TOOWDLIAE5S7D4KZPYICPTNHVVWJZ X-Message-ID-Hash: B27TOOWDLIAE5S7D4KZPYICPTNHVVWJZ X-MailFrom: hi@alyssa.is X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Spectrum OS Development X-Mailman-Version: 3.3.9 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Demi Marie Obenour writes: > Instead of duplicating the logic in three different places, write common > code for all three makefiles to use instead. This will make future > changes much easier. > > Signed-off-by: Demi Marie Obenour Makes sense, and we could do this now for the s6-rc directory without having to wait for the generated file list stuff. Does need to be updated for my review comments from the previous patch though (stick to POSIX). > --- > host/rootfs/Makefile | 51 ++++---------------------------------------= ------ > host/rootfs/default.nix | 1 + > img/app/Makefile | 42 ++++------------------------------------ > img/app/default.nix | 1 + > lib/erofs.mk | 51 +++++++++++++++++++++++++++++++++++++++++++= ++++++ > vm/sys/net/Makefile | 35 +++------------------------------ > vm/sys/net/default.nix | 1 + > 7 files changed, 65 insertions(+), 117 deletions(-) > > diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile > index 3560707d21bcf53e4f3ad5e916d21e8be56cc3a1..dce8315fe5be19d2569d6c9a4= 29276e4abd696ad 100644 > --- a/host/rootfs/Makefile > +++ b/host/rootfs/Makefile > @@ -4,11 +4,10 @@ > .POSIX: >=20=20 > include ../../lib/common.mk > +include ../../lib/erofs.mk > include file-list.mk >=20=20 > -dest =3D build/rootfs.erofs > - > -DIRS =3D \ > +override DIRS ::=3D \ > dev \ > etc/s6-linux-init/env \ > etc/s6-linux-init/run-image/configs \ > @@ -42,46 +41,9 @@ DIRS =3D \ > proc \ > sys >=20=20 > -FIFOS =3D etc/s6-linux-init/run-image/service/s6-svscan-log/fifo > - > -BUILD_FILES =3D build/etc/s6-rc > - > -$(dest): ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(addprefix image/= ,$(FILES)) $(BUILD_FILES) build/empty build/fifo file-list.mk > - ( \ > - cat $(PACKAGES_FILE) ;\ > - for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' $$file $$= file; done ;\ > - for file in $(BUILD_FILES); do printf '%s\n%s\n' $$file $${file#bui= ld/}; done ;\ > - printf 'build/empty\n%s\n' $(DIRS) ;\ > - printf 'build/fifo\n%s\n' $(FIFOS) ;\ > - ) | ../../scripts/make-erofs.sh $@ > - > -build/fifo: > - mkdir -p build > - mkfifo -m 0600 $@ > - > -build/empty: > - mkdir -p $@ > +override FIFOS ::=3D etc/s6-linux-init/run-image/service/s6-svscan-log/f= ifo >=20=20 > -# s6-rc-compile's input is a directory, but that doesn't play nice > -# with Make, because it won't know to update if some file in the > -# directory is changed, or a file is created or removed in a > -# subdirectory. Using the whole source directory could also end up > -# including files that aren't intended to be part of the input, like > -# temporary editor files or .license files. So for all these reasons, > -# only explicitly listed files are made available to s6-rc-compile. > -build/etc/s6-rc: $(addprefix image/,$(S6_RC_FILES)) file-list.mk > - mkdir -p $$(dirname $@) > - rm -rf $@ > - > - dir=3D$$(mktemp -d) && \ > - tar -C image -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components= 2 && \ > - s6-rc-compile $@ $$dir; \ > - exit=3D$$?; rm -r $$dir; exit $$exit > - > -clean: > - -chmod -Rf +w build > - rm -rf build > -.PHONY: clean > +all: $(dest) >=20=20 > # veritysetup format produces two files, but Make only (portably) > # supports one output per rule, so we combine the two outputs then > @@ -111,11 +73,6 @@ debug: > $(VMLINUX) > .PHONY: debug >=20=20 > -update-file-list: > - ../../scripts/genfiles.awk image > file-list.mk > - > -.PHONY: update-file-list > - > run: build/live.img $(EXT_FS) build/rootfs.verity.roothash > @set -x && \ > ext=3D"$$(mktemp build/spectrum-rootfs-extfs.XXXXXXXXXX.img)" && \ > diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix > index 998220d7b6ed322f64ee52c704e71ec9b4643f59..561bf660829dcd5f5e2ee8416= 62727b262d43ed3 100644 > --- a/host/rootfs/default.nix > +++ b/host/rootfs/default.nix > @@ -178,6 +178,7 @@ stdenvNoCC.mkDerivation { > fileset =3D fileset.intersection src (fileset.unions [ > ./. > ../../lib/common.mk > + ../../lib/erofs.mk > ../../scripts/make-erofs.sh > ]); > }; > diff --git a/img/app/Makefile b/img/app/Makefile > index 5bb1a6a2f9acd13aba95abb0e918a7f21943b230..0c3a8311973f5f82eb2af6ea3= ba43f67d36dec2e 100644 > --- a/img/app/Makefile > +++ b/img/app/Makefile > @@ -5,6 +5,7 @@ > .POSIX: >=20=20 > include ../../lib/common.mk > +include ../../lib/erofs.mk > include file-list.mk >=20=20 > prefix =3D build/host > @@ -18,7 +19,6 @@ HOST_BUILD_FILES =3D \ > $(imgdir)/appvm/vmlinux >=20=20 > all: $(HOST_BUILD_FILES) > -.PHONY: all >=20=20 > $(imgdir)/appvm/vmlinux: $(KERNEL) > mkdir -p $$(dirname $@) > @@ -30,38 +30,13 @@ $(imgdir)/appvm/blk/root.img: ../../scripts/make-gpt.= sh ../../scripts/sfdisk-fie > build/rootfs.erofs:root:5460386f-2203-4911-8694-91400125c604:root > mv $@.tmp $@ >=20=20 > -VM_DIRS =3D dev run proc sys tmp \ > +DIRS =3D dev run proc sys tmp \ > etc/s6-linux-init/run-image/service \ > etc/s6-linux-init/run-image/user \ > etc/s6-linux-init/run-image/wait > -VM_FIFOS =3D etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd= /fifo > +FIFOS =3D etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/fi= fo >=20=20 > -VM_BUILD_FILES =3D build/etc/s6-rc > - > -build/fifo: > - mkdir -p build > - mkfifo -m 0600 $@ > - > -build/empty: > - mkdir -p $@ > - > -build/rootfs.erofs: ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(addpr= efix image/,$(FILES)) $(VM_BUILD_FILES) build/empty build/fifo file-list.mk > - ( \ > - cat $(PACKAGES_FILE) ;\ > - for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' $$file $$= file; done ;\ > - for file in $(VM_BUILD_FILES); do printf '%s\n%s\n' $$file $${file#= build/}; done ;\ > - printf 'build/empty\n%s\n' $(VM_DIRS) ;\ > - printf 'build/fifo\n%s\n' $(VM_FIFOS) ;\ > - ) | ../../scripts/make-erofs.sh $@ > - > -build/etc/s6-rc: $(addprefix image/,$(S6_RC_FILES)) file-list.mk > - mkdir -p $$(dirname $@) > - rm -rf $@ > - > - dir=3D$$(mktemp -d) && \ > - tar -C image -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components= 2 && \ > - s6-rc-compile $@ $$dir; \ > - exit=3D$$?; rm -r $$dir; exit $$exit > +BUILD_FILES =3D >=20=20 > debug: > $(GDB) -q \ > @@ -138,14 +113,5 @@ run-crosvm: $(imgdir)/appvm/blk/root.img start-vhost= -user-gpu start-virtiofsd > $(KERNEL) > .PHONY: run-crosvm >=20=20 > -update-file-list: > - ../../scripts/genfiles.awk image > file-list.mk > - > -.PHONY: update-file-list > - > run: run-$(VMM) > .PHONY: run > - > -clean: > - rm -rf build > -.PHONY: clean > diff --git a/img/app/default.nix b/img/app/default.nix > index d3eed1f0accdc8968d1ba5bdec74ab597789082f..cf10e712ab84a9e85cb137302= 4be5f64deef0370 100644 > --- a/img/app/default.nix > +++ b/img/app/default.nix > @@ -107,6 +107,7 @@ stdenvNoCC.mkDerivation { > fileset =3D lib.fileset.intersection src (lib.fileset.unions [ > ./. > ../../lib/common.mk > + ../../lib/erofs.mk > ../../scripts/make-erofs.sh > ../../scripts/make-gpt.sh > ../../scripts/sfdisk-field.awk > diff --git a/lib/erofs.mk b/lib/erofs.mk > new file mode 100644 > index 0000000000000000000000000000000000000000..b3fc112f5e793725977cd8c4b= 2e71d6ed8d888c4 > --- /dev/null > +++ b/lib/erofs.mk > @@ -0,0 +1,51 @@ > +override basedir ::=3D $(dir $(lastword $(MAKEFILE_LIST)))/.. > + > +override BUILD_FILES ::=3D $(BUILD_FILES) build/etc/s6-rc > +# No override here so that it can be overridden in host/rootfs/default.n= ix. > +dest ::=3D build/rootfs.erofs > + > +all: > +.PHONY: all > +$(dest): $(basedir)/scripts/make-erofs.sh $(PACKAGES_FILE) $(addprefix i= mage/,$(FILES)) $(BUILD_FILES) build/empty build/fifo file-list.mk > + set -euo pipefail; ( \ > + cat $(PACKAGES_FILE) ;\ > + for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' "$$file" = "$$file"; done ;\ > + for file in $(BUILD_FILES); do printf '%s\n%s\n' "$$file" "$${file#= build/}"; done ;\ > + $(and $(DIRS),printf 'build/empty\n%s\n' $(DIRS);)\ > + $(and $(FIFOS),printf 'build/fifo\n%s\n' $(FIFOS);)\ > + ) | $(basedir)/scripts/make-erofs.sh $@ > + > +clean: > + -chmod -Rf +w build > + rm -rf build > +.PHONY: clean > + > +build/fifo: > + mkdir -p build > + mkfifo -m 0600 $@ > + > +build/empty: > + mkdir -p $@ > + > +update-file-list: > + $(basedir)/scripts/genfiles.awk image > file-list.mk > + > +.PHONY: update-file-list > +.PHONY: > + $(error cannot make .PHONY!) > + > +# s6-rc-compile's input is a directory, but that doesn't play nice > +# with Make, because it won't know to update if some file in the > +# directory is changed, or a file is created or removed in a > +# subdirectory. Using the whole source directory could also end up > +# including files that aren't intended to be part of the input, like > +# temporary editor files or .license files. So for all these reasons, > +# only explicitly listed files are made available to s6-rc-compile. > +build/etc/s6-rc: $(addprefix image/,$(S6_RC_FILES)) file-list.mk > + mkdir -p build/etc > + rm -rf $@ > + set -euo pipefail; \ > + dir=3D$$(mktemp -d); \ > + { tar -C image -c $(S6_RC_FILES) | tar -C "$$dir" -x --strip-components= 2;}; \ > + s6-rc-compile $@ $$dir; \ > + exit=3D$$?; rm -r $$dir; exit $$exit > diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile > index e731bcea7fc61f9b2019689e52a600f3741dd55f..3f8493b9bf2364f7b2d01b7f1= 14dba72126965cc 100644 > --- a/vm/sys/net/Makefile > +++ b/vm/sys/net/Makefile > @@ -4,6 +4,7 @@ > .POSIX: >=20=20 > include ../../../lib/common.mk > +include ../../../lib/erofs.mk > include file-list.mk >=20=20 > prefix =3D build/host > @@ -11,13 +12,14 @@ libdir =3D $(prefix)/lib > vmdir =3D $(libdir)/spectrum/vm >=20=20 > VMM =3D cloud-hypervisor > +override DIRS ::=3D > +override FIFOS ::=3D >=20=20 > HOST_BUILD_FILES =3D \ > $(vmdir)/netvm/blk/root.img \ > $(vmdir)/netvm/vmlinux >=20=20 > all: $(HOST_BUILD_FILES) > -.PHONY: all >=20=20 > $(vmdir)/netvm/vmlinux: $(KERNEL) > mkdir -p $$(dirname $@) > @@ -29,28 +31,6 @@ $(vmdir)/netvm/blk/root.img: ../../../scripts/make-gpt= .sh ../../../scripts/sfdis > build/rootfs.erofs:root:ea21da27-0391-48da-9235-9d2ab2ca7844:root > mv $@.tmp $@ >=20=20 > -VM_BUILD_FILES =3D build/etc/s6-rc > - > -build/empty: > - mkdir -p $@ > - > -build/rootfs.erofs: ../../../scripts/make-erofs.sh $(PACKAGES_FILE) $(ad= dprefix image/,$(FILES)) $(VM_BUILD_FILES) build/empty file-list.mk > - ( \ > - cat $(PACKAGES_FILE) ;\ > - for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' $$file $$= file; done ;\ > - for file in $(VM_BUILD_FILES); do printf '%s\n%s\n' $$file $${file#= build/}; done ;\ > - printf 'build/empty\n%s\n' $(VM_DIRS) ;\ > - ) | ../../../scripts/make-erofs.sh $@ > - > -build/etc/s6-rc: $(addprefix image/,$(S6_RC_FILES)) file-list.mk > - mkdir -p $$(dirname $@) > - rm -rf $@ > - > - dir=3D$$(mktemp -d) && \ > - tar -C image -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components= 2 && \ > - s6-rc-compile $@ $$dir; \ > - exit=3D$$?; rm -r $$dir; exit $$exit > - > debug: > $(GDB) -q \ > -ex 'set substitute-path .. $(LINUX_SRC)' \ > @@ -102,14 +82,5 @@ run-crosvm: $(vmdir)/netvm/blk/root.img > $(KERNEL) > .PHONY: run-crosvm >=20=20 > -update-file-list: > - ../../scripts/genfiles.awk image > file-list.mk > - > -.PHONY: update-file-list > - > run: run-$(VMM) > .PHONY: run > - > -clean: > - rm -rf build > -.PHONY: clean > diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix > index b5873ebe1e80dd88c1ba997f7ebd3ee7369bb40f..7b03264bd30fe2e9157730e53= 38cd2501b255f7e 100644 > --- a/vm/sys/net/default.nix > +++ b/vm/sys/net/default.nix > @@ -104,6 +104,7 @@ stdenvNoCC.mkDerivation { > fileset =3D lib.fileset.intersection src (lib.fileset.unions [ > ./. > ../../../lib/common.mk > + ../../../lib/erofs.mk > ../../../scripts/make-erofs.sh > ../../../scripts/make-gpt.sh > ../../../scripts/sfdisk-field.awk > > --=20 > 2.51.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQRV/neXydHjZma5XLJbRZGEIw/wogUCaL6pggAKCRBbRZGEIw/w osQnAP9iTkBUTs3kvuw8yufBfML8jT5xMH7uZY+RqCsliwy22AEA7L+WgH+zHwhH xn5UqeuA/jUs9+mPqYPoYP0GmeuPDA4= =xRSP -----END PGP SIGNATURE----- --=-=-=--