From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id D2A71269FA; Thu, 04 Sep 2025 02:09:29 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 2E278269B8; Thu, 04 Sep 2025 02:09:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_PASS,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=4.0.1 Received: from mail-yb1-xb2c.google.com (mail-yb1-xb2c.google.com [IPv6:2607:f8b0:4864:20::b2c]) by atuin.qyliss.net (Postfix) with ESMTPS id 12360269A5 for ; Thu, 04 Sep 2025 02:09:21 +0000 (UTC) Received: by mail-yb1-xb2c.google.com with SMTP id 3f1490d57ef6-e96fc00ad79so568027276.0 for ; Wed, 03 Sep 2025 19:09:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1756951760; x=1757556560; darn=spectrum-os.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=I2ppzVVeyDU72ktv7Mt+VnQRrja0OJIq6qfkTl/J/AA=; b=ILghMQ5mJ8meR3E1o2CGAjpmwbbioLD7GWwGA0t86V5ZFGEeAnVuumhL+QD4JbCKOd pAi8iZMlOve2E80cu4/xDFNzeucpZHyMkcB5vS3q9lrMDIIRBNBCo3uVtmIZqr+PxFYz oVpsc5N7aY1TbYJGA7n4t5uhAqduuUntfeiwtxbjunB8svGyEgJaANIb1dWKwf3g3tZY 3V94qxfdvHojfO8G5+GYUxAiu7AFJxvM8ed5k1GuhTAb2k+oRS7e3oDj6BTgmx6kVFO3 sy/bqDOIgO7vLezmZcI5ZmT2+I8SJHc5FpanGSHFYV4V1QOSPewC3wkF2WUP5NOeIV/g vKww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1756951760; x=1757556560; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I2ppzVVeyDU72ktv7Mt+VnQRrja0OJIq6qfkTl/J/AA=; b=eoDkuz5OGEGI+r+dFk8k1eexa0GKZ9/j7EoAShXf5CC22Ul0Oi2BTUZf/QoiNnA5vS tYS5PDAfOKdZxh/O5yytSvT9nFUoGhsW4Zfx6QgmFMs7oze6AdW48Vb4nKCM87JLqhBG haxH1/6io6ilBYEei7raZ0KEbQCvFAytB+lgTEp+YEUmfQzfkMOwyBzSZKl6sKj7lMzF GhYZsNFS3AajCp3ZiTNMpIkRK0TeEzEozXfP7OG7QPNPI+8noQ1e7VIXLhY1E0miBmPS /fH1awEhNpqS/299+PVyzB26f8EHfUHJ7M08IScDY4icllPfjxEaxfey+VxrHGo9jJzM dCcQ== X-Gm-Message-State: AOJu0YzrOz23SDg6H6NJEZs3M+jFIX99ceiA/zd/1/7hBYEVXll+P8+P 9eDnuD38gKXVoqG0CoaCUcR7l47pPNTSV8a81ADxF33OR3cYw8IgaMF7NQh2FA== X-Gm-Gg: ASbGncsU271nX8drXthKaxp5PJQng/00YNyakHDbfL9qnWHqrBoNPqparZqwTwFES9L 4MkW16d3p2t4JlPyScVS8rTSDHjc7awwrX7H+RVA2IEDjTX7PwgbmUbZ4SVxWY6mXt/VCoZuLte wChl2E6Hfb0vLPGUlJlcIVKWNKV6zoBHz4sUfMJ9xm9hwosyzxH7yNYYLLPzjQo4J3bMSOmxVqq cw3TnmgwInRSnyw/hMKZxt1MLlgMTpRs1wr8PPwtCi83wzCur5Qdu+I752QZhsU250DjUu8iwkC zH/4HM8vEzmDpzban5yne29xPm+Gic18XtXj/DcxnKznDo+8HY8mz1PYA/Iv49n25eGoqxiyT9t I/2JM9fqWACpTfZtxREExNxARQDK4rbzKqRnaWRIKNCxaEVffs3f5IAavP8hj8CZy5mt7FuB+nj gjkI5nSJ0DpwgKImLAUcSMIoDhuuQE7wNiXZQepsast9E= X-Google-Smtp-Source: AGHT+IEkEqabikuJHXhVEev9ToIorpthIEs+w+2wMclZZPOep/m3ruDnf10/ZX01vSCF7QgYMIA6Gw== X-Received: by 2002:a05:690c:46c4:b0:723:97ed:b2d6 with SMTP id 00721157ae682-72397edb57amr125129347b3.24.1756951759816; Wed, 03 Sep 2025 19:09:19 -0700 (PDT) Received: from localhost.localdomain (h96-60-249-169.cncrtn.broadband.dynamic.tds.net. [96.60.249.169]) by smtp.gmail.com with UTF8SMTPSA id 00721157ae682-723a8501f5fsm18373907b3.48.2025.09.03.19.09.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Sep 2025 19:09:19 -0700 (PDT) From: Demi Marie Obenour Date: Wed, 03 Sep 2025 21:56:54 -0400 Subject: [PATCH 3/4] Common make rules for building erofs images MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20250903-genfiles-v1-3-cc993fcb1e4c@gmail.com> References: <20250903-genfiles-v1-0-cc993fcb1e4c@gmail.com> In-Reply-To: <20250903-genfiles-v1-0-cc993fcb1e4c@gmail.com> To: Spectrum OS Development X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1756951011; l=11235; i=demiobenour@gmail.com; s=20250729; h=from:subject:message-id; bh=Q+xrhUXCvvoUv1oV35ABe+j0A9XMbjmfRb7q5FNUSyo=; b=GZcfwbNUC6ImGyZocZ8pBwI/spdmxxijOIvbvQxyKohqXari//rh4Qps7zVkago15ubntr+oo fPfjHEWrsVwAdYEITlzvo0W2S9P+rZ/ez9hPphFaHmynobqYfBbhxuQ X-Developer-Key: i=demiobenour@gmail.com; a=ed25519; pk=X57Q4/YQDj9t4SBeKaDwvXYKB6quZJVx/DE2Ly2out0= Message-ID-Hash: RKBRVNP5OKO3UTYTFYXMPMY4ZYCNK3QA X-Message-ID-Hash: RKBRVNP5OKO3UTYTFYXMPMY4ZYCNK3QA X-MailFrom: demiobenour@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Demi Marie Obenour , Alyssa Ross X-Mailman-Version: 3.3.9 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Instead of duplicating the logic in three different places, write common code for all three makefiles to use instead. This will make future changes much easier. Signed-off-by: Demi Marie Obenour --- host/rootfs/Makefile | 51 ++++--------------------------------------------- host/rootfs/default.nix | 1 + img/app/Makefile | 42 ++++------------------------------------ img/app/default.nix | 1 + lib/erofs.mk | 51 +++++++++++++++++++++++++++++++++++++++++++++++++ vm/sys/net/Makefile | 35 +++------------------------------ vm/sys/net/default.nix | 1 + 7 files changed, 65 insertions(+), 117 deletions(-) diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index 3560707d21bcf53e4f3ad5e916d21e8be56cc3a1..dce8315fe5be19d2569d6c9a429276e4abd696ad 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -4,11 +4,10 @@ .POSIX: include ../../lib/common.mk +include ../../lib/erofs.mk include file-list.mk -dest = build/rootfs.erofs - -DIRS = \ +override DIRS ::= \ dev \ etc/s6-linux-init/env \ etc/s6-linux-init/run-image/configs \ @@ -42,46 +41,9 @@ DIRS = \ proc \ sys -FIFOS = etc/s6-linux-init/run-image/service/s6-svscan-log/fifo - -BUILD_FILES = build/etc/s6-rc - -$(dest): ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(addprefix image/,$(FILES)) $(BUILD_FILES) build/empty build/fifo file-list.mk - ( \ - cat $(PACKAGES_FILE) ;\ - for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' $$file $$file; done ;\ - for file in $(BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ - printf 'build/empty\n%s\n' $(DIRS) ;\ - printf 'build/fifo\n%s\n' $(FIFOS) ;\ - ) | ../../scripts/make-erofs.sh $@ - -build/fifo: - mkdir -p build - mkfifo -m 0600 $@ - -build/empty: - mkdir -p $@ +override FIFOS ::= etc/s6-linux-init/run-image/service/s6-svscan-log/fifo -# s6-rc-compile's input is a directory, but that doesn't play nice -# with Make, because it won't know to update if some file in the -# directory is changed, or a file is created or removed in a -# subdirectory. Using the whole source directory could also end up -# including files that aren't intended to be part of the input, like -# temporary editor files or .license files. So for all these reasons, -# only explicitly listed files are made available to s6-rc-compile. -build/etc/s6-rc: $(addprefix image/,$(S6_RC_FILES)) file-list.mk - mkdir -p $$(dirname $@) - rm -rf $@ - - dir=$$(mktemp -d) && \ - tar -C image -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 2 && \ - s6-rc-compile $@ $$dir; \ - exit=$$?; rm -r $$dir; exit $$exit - -clean: - -chmod -Rf +w build - rm -rf build -.PHONY: clean +all: $(dest) # veritysetup format produces two files, but Make only (portably) # supports one output per rule, so we combine the two outputs then @@ -111,11 +73,6 @@ debug: $(VMLINUX) .PHONY: debug -update-file-list: - ../../scripts/genfiles.awk image > file-list.mk - -.PHONY: update-file-list - run: build/live.img $(EXT_FS) build/rootfs.verity.roothash @set -x && \ ext="$$(mktemp build/spectrum-rootfs-extfs.XXXXXXXXXX.img)" && \ diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index 998220d7b6ed322f64ee52c704e71ec9b4643f59..561bf660829dcd5f5e2ee841662727b262d43ed3 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -178,6 +178,7 @@ stdenvNoCC.mkDerivation { fileset = fileset.intersection src (fileset.unions [ ./. ../../lib/common.mk + ../../lib/erofs.mk ../../scripts/make-erofs.sh ]); }; diff --git a/img/app/Makefile b/img/app/Makefile index 5bb1a6a2f9acd13aba95abb0e918a7f21943b230..0c3a8311973f5f82eb2af6ea3ba43f67d36dec2e 100644 --- a/img/app/Makefile +++ b/img/app/Makefile @@ -5,6 +5,7 @@ .POSIX: include ../../lib/common.mk +include ../../lib/erofs.mk include file-list.mk prefix = build/host @@ -18,7 +19,6 @@ HOST_BUILD_FILES = \ $(imgdir)/appvm/vmlinux all: $(HOST_BUILD_FILES) -.PHONY: all $(imgdir)/appvm/vmlinux: $(KERNEL) mkdir -p $$(dirname $@) @@ -30,38 +30,13 @@ $(imgdir)/appvm/blk/root.img: ../../scripts/make-gpt.sh ../../scripts/sfdisk-fie build/rootfs.erofs:root:5460386f-2203-4911-8694-91400125c604:root mv $@.tmp $@ -VM_DIRS = dev run proc sys tmp \ +DIRS = dev run proc sys tmp \ etc/s6-linux-init/run-image/service \ etc/s6-linux-init/run-image/user \ etc/s6-linux-init/run-image/wait -VM_FIFOS = etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/fifo +FIFOS = etc/s6-linux-init/run-image/service/s6-linux-init-shutdownd/fifo -VM_BUILD_FILES = build/etc/s6-rc - -build/fifo: - mkdir -p build - mkfifo -m 0600 $@ - -build/empty: - mkdir -p $@ - -build/rootfs.erofs: ../../scripts/make-erofs.sh $(PACKAGES_FILE) $(addprefix image/,$(FILES)) $(VM_BUILD_FILES) build/empty build/fifo file-list.mk - ( \ - cat $(PACKAGES_FILE) ;\ - for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' $$file $$file; done ;\ - for file in $(VM_BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ - printf 'build/empty\n%s\n' $(VM_DIRS) ;\ - printf 'build/fifo\n%s\n' $(VM_FIFOS) ;\ - ) | ../../scripts/make-erofs.sh $@ - -build/etc/s6-rc: $(addprefix image/,$(S6_RC_FILES)) file-list.mk - mkdir -p $$(dirname $@) - rm -rf $@ - - dir=$$(mktemp -d) && \ - tar -C image -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 2 && \ - s6-rc-compile $@ $$dir; \ - exit=$$?; rm -r $$dir; exit $$exit +BUILD_FILES = debug: $(GDB) -q \ @@ -138,14 +113,5 @@ run-crosvm: $(imgdir)/appvm/blk/root.img start-vhost-user-gpu start-virtiofsd $(KERNEL) .PHONY: run-crosvm -update-file-list: - ../../scripts/genfiles.awk image > file-list.mk - -.PHONY: update-file-list - run: run-$(VMM) .PHONY: run - -clean: - rm -rf build -.PHONY: clean diff --git a/img/app/default.nix b/img/app/default.nix index d3eed1f0accdc8968d1ba5bdec74ab597789082f..cf10e712ab84a9e85cb1373024be5f64deef0370 100644 --- a/img/app/default.nix +++ b/img/app/default.nix @@ -107,6 +107,7 @@ stdenvNoCC.mkDerivation { fileset = lib.fileset.intersection src (lib.fileset.unions [ ./. ../../lib/common.mk + ../../lib/erofs.mk ../../scripts/make-erofs.sh ../../scripts/make-gpt.sh ../../scripts/sfdisk-field.awk diff --git a/lib/erofs.mk b/lib/erofs.mk new file mode 100644 index 0000000000000000000000000000000000000000..b3fc112f5e793725977cd8c4b2e71d6ed8d888c4 --- /dev/null +++ b/lib/erofs.mk @@ -0,0 +1,51 @@ +override basedir ::= $(dir $(lastword $(MAKEFILE_LIST)))/.. + +override BUILD_FILES ::= $(BUILD_FILES) build/etc/s6-rc +# No override here so that it can be overridden in host/rootfs/default.nix. +dest ::= build/rootfs.erofs + +all: +.PHONY: all +$(dest): $(basedir)/scripts/make-erofs.sh $(PACKAGES_FILE) $(addprefix image/,$(FILES)) $(BUILD_FILES) build/empty build/fifo file-list.mk + set -euo pipefail; ( \ + cat $(PACKAGES_FILE) ;\ + for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' "$$file" "$$file"; done ;\ + for file in $(BUILD_FILES); do printf '%s\n%s\n' "$$file" "$${file#build/}"; done ;\ + $(and $(DIRS),printf 'build/empty\n%s\n' $(DIRS);)\ + $(and $(FIFOS),printf 'build/fifo\n%s\n' $(FIFOS);)\ + ) | $(basedir)/scripts/make-erofs.sh $@ + +clean: + -chmod -Rf +w build + rm -rf build +.PHONY: clean + +build/fifo: + mkdir -p build + mkfifo -m 0600 $@ + +build/empty: + mkdir -p $@ + +update-file-list: + $(basedir)/scripts/genfiles.awk image > file-list.mk + +.PHONY: update-file-list +.PHONY: + $(error cannot make .PHONY!) + +# s6-rc-compile's input is a directory, but that doesn't play nice +# with Make, because it won't know to update if some file in the +# directory is changed, or a file is created or removed in a +# subdirectory. Using the whole source directory could also end up +# including files that aren't intended to be part of the input, like +# temporary editor files or .license files. So for all these reasons, +# only explicitly listed files are made available to s6-rc-compile. +build/etc/s6-rc: $(addprefix image/,$(S6_RC_FILES)) file-list.mk + mkdir -p build/etc + rm -rf $@ + set -euo pipefail; \ + dir=$$(mktemp -d); \ + { tar -C image -c $(S6_RC_FILES) | tar -C "$$dir" -x --strip-components 2;}; \ + s6-rc-compile $@ $$dir; \ + exit=$$?; rm -r $$dir; exit $$exit diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile index e731bcea7fc61f9b2019689e52a600f3741dd55f..3f8493b9bf2364f7b2d01b7f114dba72126965cc 100644 --- a/vm/sys/net/Makefile +++ b/vm/sys/net/Makefile @@ -4,6 +4,7 @@ .POSIX: include ../../../lib/common.mk +include ../../../lib/erofs.mk include file-list.mk prefix = build/host @@ -11,13 +12,14 @@ libdir = $(prefix)/lib vmdir = $(libdir)/spectrum/vm VMM = cloud-hypervisor +override DIRS ::= +override FIFOS ::= HOST_BUILD_FILES = \ $(vmdir)/netvm/blk/root.img \ $(vmdir)/netvm/vmlinux all: $(HOST_BUILD_FILES) -.PHONY: all $(vmdir)/netvm/vmlinux: $(KERNEL) mkdir -p $$(dirname $@) @@ -29,28 +31,6 @@ $(vmdir)/netvm/blk/root.img: ../../../scripts/make-gpt.sh ../../../scripts/sfdis build/rootfs.erofs:root:ea21da27-0391-48da-9235-9d2ab2ca7844:root mv $@.tmp $@ -VM_BUILD_FILES = build/etc/s6-rc - -build/empty: - mkdir -p $@ - -build/rootfs.erofs: ../../../scripts/make-erofs.sh $(PACKAGES_FILE) $(addprefix image/,$(FILES)) $(VM_BUILD_FILES) build/empty file-list.mk - ( \ - cat $(PACKAGES_FILE) ;\ - for file in $(FILES) $(LINKS); do printf 'image/%s\n%s\n' $$file $$file; done ;\ - for file in $(VM_BUILD_FILES); do printf '%s\n%s\n' $$file $${file#build/}; done ;\ - printf 'build/empty\n%s\n' $(VM_DIRS) ;\ - ) | ../../../scripts/make-erofs.sh $@ - -build/etc/s6-rc: $(addprefix image/,$(S6_RC_FILES)) file-list.mk - mkdir -p $$(dirname $@) - rm -rf $@ - - dir=$$(mktemp -d) && \ - tar -C image -c $(S6_RC_FILES) | tar -C $$dir -x --strip-components 2 && \ - s6-rc-compile $@ $$dir; \ - exit=$$?; rm -r $$dir; exit $$exit - debug: $(GDB) -q \ -ex 'set substitute-path .. $(LINUX_SRC)' \ @@ -102,14 +82,5 @@ run-crosvm: $(vmdir)/netvm/blk/root.img $(KERNEL) .PHONY: run-crosvm -update-file-list: - ../../scripts/genfiles.awk image > file-list.mk - -.PHONY: update-file-list - run: run-$(VMM) .PHONY: run - -clean: - rm -rf build -.PHONY: clean diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix index b5873ebe1e80dd88c1ba997f7ebd3ee7369bb40f..7b03264bd30fe2e9157730e5338cd2501b255f7e 100644 --- a/vm/sys/net/default.nix +++ b/vm/sys/net/default.nix @@ -104,6 +104,7 @@ stdenvNoCC.mkDerivation { fileset = lib.fileset.intersection src (lib.fileset.unions [ ./. ../../../lib/common.mk + ../../../lib/erofs.mk ../../../scripts/make-erofs.sh ../../../scripts/make-gpt.sh ../../../scripts/sfdisk-field.awk -- 2.51.0