From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id 33CC66807; Sun, 21 Sep 2025 17:07:13 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 269826780; Sun, 21 Sep 2025 17:07:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_MISSING,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=4.0.1 Received: from fout-b1-smtp.messagingengine.com (fout-b1-smtp.messagingengine.com [202.12.124.144]) by atuin.qyliss.net (Postfix) with ESMTPS id 3798C66FF for ; Sun, 21 Sep 2025 17:07:09 +0000 (UTC) Received: from phl-compute-02.internal (phl-compute-02.internal [10.202.2.42]) by mailfout.stl.internal (Postfix) with ESMTP id EC3D41D0016B; Sun, 21 Sep 2025 13:07:07 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-02.internal (MEProxy); Sun, 21 Sep 2025 13:07:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alyssa.is; h=cc :cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1758474427; x=1758560827; bh=EZ8RQolD17 Adg0O4G9JBqlz6agOu0enl+JUXrTO7R00=; b=clOi2WQ1R6Ry9FCJ+dROdxEDFM xEOja/YF5B2HgtYJHnrUQwfqiYtBbqVe2qoI7fx+03wHYtfNJjpwfNSGShzHUe6m w2EZF+7cq0tEpKSL5u1mlQVt063BaXJcYBeqQop5xM3kSXwG1ZG9mR6VcdktHhJM kqjvKcmTHdNRPg+CCDico7iImIai7wH4DoXWMQwjzMCQSRPNubqxFkfCrpywFdJW 5+MrAZOOzygutwOOvViN2WsoMUg6JofgmMM3JsMPjEBThSlWP3iHxvVMKsfoStUU Coz5cLWh4McdO877Q/+m6QkoBwnppAhSV3JZxe3P08sG/QHEcaGSUsOtZ/9Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1758474427; x=1758560827; bh=EZ8RQolD17Adg0O4G9JBqlz6agOu0enl+JU XrTO7R00=; b=IaIg0QNx20H4ggK7A3yLtSTueu9L2CMdpPChSDr0Pdp8/45UqUm hpVg5r+Xs9UrDLc/4vdew+xNt47dXrKF7h5D85hpBNgi3GiXTz6/YFDGwMGBQRqd hH5DE28/mNJIFYD72sloziAAqW6O4C4j5qJ8f2UWK+atPKbnCX3M4QPReBdFrR7T NVNOfemUard4KsNN0d0oLvYW9uSrZGuscsfQRwkd4XXFnhVlh8G29AvJCQbF9bLE nlOseRyRkefDLb1foCPwXgeXnQtFaCwzkaCv3ChSvV01Afdvk1fhdDFVsha5TXkG NehlWdXz4e3YHT+A3fvm+rg2d1FhjqBqMHA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdeggdehheehfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufgjfhffkfggtgesghdtreertddtjeenucfhrhhomheptehlhihsshgrucft ohhsshcuoehhihesrghlhihsshgrrdhisheqnecuggftrfgrthhtvghrnhepteehvedugf ejgfehhfeijeduleekleejgedvkeeuuefhhfegvdevfeetveegteeinecuvehluhhsthgv rhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhephhhisegrlhihshhsrgdrih hspdhnsggprhgtphhtthhopedvpdhmohguvgepshhmthhpohhuthdprhgtphhtthhopegu vghmihhosggvnhhouhhrsehgmhgrihhlrdgtohhmpdhrtghpthhtohepuggvvhgvlhessh hpvggtthhruhhmqdhoshdrohhrgh X-ME-Proxy: Feedback-ID: i12284293:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 21 Sep 2025 13:07:07 -0400 (EDT) Received: by mbp.qyliss.net (Postfix, from userid 1000) id 30DFC24BF2E2; Sun, 21 Sep 2025 19:07:06 +0200 (CEST) From: Alyssa Ross To: Demi Marie Obenour Subject: Re: [PATCH v3] Generate file lists from a script In-Reply-To: References: <20250910-genfiles-v2-0-37ebe07a3cdc@gmail.com> <20250920-genfiles-v3-1-d6c2b6767b42@gmail.com> <87bjn4b39h.fsf@alyssa.is> Date: Sun, 21 Sep 2025 19:07:04 +0200 Message-ID: <87h5wv20qv.fsf@alyssa.is> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Message-ID-Hash: 3OXIBKDJHVB6JSOND3LOLVVR7N635J43 X-Message-ID-Hash: 3OXIBKDJHVB6JSOND3LOLVVR7N635J43 X-MailFrom: hi@alyssa.is X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Spectrum OS Development X-Mailman-Version: 3.3.9 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Demi Marie Obenour writes: >>> diff --git a/scripts/genfiles.awk b/scripts/genfiles.awk >>> new file mode 100644 >>> index 0000000000000000000000000000000000000000..6fe327fd0a314d226dbce23= 854aa8f119e9c8f34 >>> --- /dev/null >>> +++ b/scripts/genfiles.awk >>> @@ -0,0 +1,120 @@ >>> +#!/usr/bin/env -S LC_ALL=3DC LANGUAGE=3DC awk -E >>> +# SPDX-License-Identifier: EUPL-1.2+ >>> +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour >>> +BEGIN { >>> + RS =3D "\n"; >>> + FS =3D "\t"; >>> + file_count =3D 0; >>> + symlink_count =3D 0; >>> + rc_count =3D 0; >>> + is_rc =3D 0; >>> + exit_code =3D 0; >>> + done =3D 0; >>=20 >> awk variables are implicitly initialized to 0 when you try to do >> arithmetic on an undefined variable, so no need for these. > > GNU Awk can lint against that. I used its lint mode because it also > warns against non-portable constructs. Also, an undefined awk > variable used as an array subscript is treated as the empty string, > not 0, which could lead to confusion. Okay, happy to leave them if you want. >>> + >>> +filename ~ /^image\/etc\/s6-rc\// { >>> + if (mode !=3D "regular") { >>> + fail("s6-rc-compile input '" filename "' isn't a regular file"); >>> + } >>> + rc_count +=3D 1; >>> + rc_files[rc_count] =3D filename; >>=20 >> rc_files[rc_count++] >>=20 >> (will make it 0-indexed though so update the loops too) > > I think this might break without explicit variable initialization. It does not. >>> + "# Generated by scripts/genfile.sh. Any changes will be overw= ritten.\n" \ >>> + "FILES ::=3D") > out_file; >>=20 >> I note the change to ::=3D. Do you think we should do that across the >> board in our Makefiles? > > POSIX specifies ::=3D and it has better semantics in most cases, but I do= n't > know if the BSD makes implement it. ::=3D causes the RHS to be expanded = immediately, > so subsequent changes in variables referenced by it do not affect the LHS. Happy to change, but would prefer we did it all at once. >>> + for (array_index =3D 1; array_index <=3D file_count; array_index +=3D= 1) { >>> + printf " \\\n\t%s", files[array_index] > out_file; >>> + } >>> + printf ("\n\n" \ >>> +"# These are separate because they need to be included, but putting\n"= \ >>> +"# them as make dependencies would confuse make.\n" \ >>> +"LINKS ::=3D") > out_file; >>> + for (array_index =3D 1; array_index <=3D symlink_count; array_index += =3D 1) { >>> + printf " \\\n\t%s", symlinks[array_index] > out_file; >>> + } >>> + printf "\n\nS6_RC_FILES ::=3D" > out_file; >>> + for (array_index =3D 1; array_index <=3D rc_count; array_index +=3D 1= ) { >>> + printf " \\\n\t%s", rc_files[array_index] > out_file; >>> + } >>> + printf "\n" > out_file; >>> + if (close(out_file)) { >>> + print ("Cannot close output file: " ERRNO "\n") > "/dev/stderr"; >>> + exit 1; >>> + } >>> +} >>> diff --git a/scripts/genfiles.sh b/scripts/genfiles.sh >>> new file mode 100755 >>> index 0000000000000000000000000000000000000000..77a8d95e88b6851be944769= 8556efe4f1eab174b >>> --- /dev/null >>> +++ b/scripts/genfiles.sh >>> @@ -0,0 +1,29 @@ >>> +#!/usr/bin/env -S LC_ALL=3DC LANGUAGE=3DC bash -- >>=20 >> env -S is not portable, and I don't think anything here needs bash >> specifically. > > $'\t' doesn't work with all shells, though I believe it is either > part of the current POSIX standard or will be added. I'll use > /usr/bin/env bash, which breaks if the script is renamed to something > starting with '-'. It's in 2024. I'd prefer this was /bin/sh like our other scripts =E2=80=94= we already have shellcheck checking for non-portable constructs. >>> +case $0 in >>> +(/*) cd "${0%/*}/..";; >>> +(*/*) cd "./${0%/*}/..";; >>> +(*) cd ..;; >>> +esac >>=20 >> Perhaps we could use git rev-parse --show-toplevel? > > git ls-files doesn't have that option. I mean we could cd "$(git rev-parse --show-toplevel)", and then be in a consistent starting place. >>> +for i in host/rootfs img/app vm/sys/net; do >>> + output_file=3D$i/file-list.mk >>> + { >>> + git -C "$i" -c core.quotePath=3Dtrue ls-files $'--format=3D%(objectmo= de)\t%(path)' -- image | >>> + sort -t $'\t' -k 2 >>=20 >> TIL sort -t and -k! =F0=9F=A4=AF >>=20 >>> + echo DONE >>=20 >> Why do we need this? > > To avoid producing any output file if the input is truncated. > >>> + } | >>> + gawk -v "out_file=3D$output_file.tmp" -E scripts/genfiles.awk >>=20 >> Why not stdout? > > The output file is created by awk so that it is only created if > nothing went wrong. For both of these, we already have exit status to communicate if something goes wrong. When would output get truncated without that being an unsuccessful exit? >>> + if [ -f "$output_file" ]; then >>> + # Avoid changing output file if it is up to date, as that >>> + # would cause unnecessary rebuilds. >>> + if cmp -s -- "$output_file.tmp" "$output_file"; then >>> + rm -- "$output_file.tmp" >>> + continue >>> + else >>> + astatus=3D$? >>> + if [ "$astatus" !=3D 1 ]; then exit "$astatus"; fi >>=20 >> Could avoid the need for the variable and multiple ifs. Up to you >> whether you prefer it: >>=20 >> set +e >> cmp -s -- "$output_file.tmp" "$output_file" >> set -e >> case $? in >> 0) >> rm -- "$output_file.tmp" >> continue >> ;; >> 1) >> ;; >> *) >> exit $? >> ;; >> esac > > This might set $? to the return value of 'set -e' (0). Whether or > not it actually does is at least not obvious from reading the code. Oh good point. Let's stick with your way then. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQRV/neXydHjZma5XLJbRZGEIw/wogUCaNAwuQAKCRBbRZGEIw/w oqqgAQDSTS8l8OvFAFC7S+nWDJoysmKiMfMuvwRRpMMV4TW80QD/czozj5AhZW4P 2xpSy4xXEGNKawP4oTGPf3m8TxD4agA= =hWxW -----END PGP SIGNATURE----- --=-=-=--