From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id D85074078; Sat, 06 Dec 2025 14:53:10 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 8A7EE4099; Sat, 06 Dec 2025 14:53:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_MISSING,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=4.0.1 Received: from fhigh-a5-smtp.messagingengine.com (fhigh-a5-smtp.messagingengine.com [103.168.172.156]) by atuin.qyliss.net (Postfix) with ESMTPS id 6859E4097 for ; Sat, 06 Dec 2025 14:53:07 +0000 (UTC) Received: from phl-compute-01.internal (phl-compute-01.internal [10.202.2.41]) by mailfhigh.phl.internal (Postfix) with ESMTP id 83025140003F; Sat, 6 Dec 2025 09:53:04 -0500 (EST) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-01.internal (MEProxy); Sat, 06 Dec 2025 09:53:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alyssa.is; h=cc :cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1765032784; x=1765119184; bh=ibHWFYD8/n z6EYaDocMoPeWau6+PWgliSyTreYmicFw=; b=fkLV1B5n6T7DDd3K4Wt4pn/blg YTL+qLrZUX+EoOXc3koVv76gfSJYHUbC6hFjzj4VpSRm8DPmILtgK2XLEjOIVu4x eF3ixrBM3YfS9e2LVETqYW9z0BBsu4FxbUlJDDRSmjywSrMYynkzK+ODkgnMoYeZ bSQDBUD/RugH0ODX1rKO6ZWfWRf56BEPfyYu3BpvXVumWsOApjx1b2XNJNEIu7fU xQLzqoXoZIEh+csu6hGWVNjvb8xtOSfst8uIs1cgKj9W8XfM58CF4R9EsKU0eBVf SzgE+tHUovPdZdUI9JGg1AAp0DgBxFgIZCsYrA3eZGVE6uudjoN/aSx08rHg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1765032784; x=1765119184; bh=ibHWFYD8/nz6EYaDocMoPeWau6+PWgliSyT reYmicFw=; b=L/c8eZbzYCWo5Lc0E0npmPS+4QTPSeFy0eeXNDq2zAZA9o5QBvc U9nkZs3z8B9sDViRMC7dwZUgKSeDnIc6o3t02EitMpC6sYw6AD7E4WOCnrbtLwvA pgH8lJomPRQSmBL0LykCpgX90nKRP+qasY97ORofZwJJ0GyMRKTbBecoAsZ9LIEF pPRAsDXPSXOETpw11noE+yT0GtvnD/ujj5/bf0hjdbCbUE/us00Dv74Q/MCrzQ5x tbDQybHHOSfFI7KGD8GKTD7nO9ajrVe7NnQy+BJId0x4GI20i7dVWcBfyMtOi1ZP LEe5uGc3AasrmG8Fd//VUIh5bzog4cM7fDw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefgedrtddtgdduuddvvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufgjfhffkfggtgesghdtreertddtjeenucfhrhhomheptehlhihsshgrucft ohhsshcuoehhihesrghlhihsshgrrdhisheqnecuggftrfgrthhtvghrnhepveeuieeive etieekueduffeijeehvdelueekfeevleekhfettefhleegkeeiteevnecuffhomhgrihhn pehpohhsthhmrghrkhgvthhoshdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurf grrhgrmhepmhgrihhlfhhrohhmpehhihesrghlhihsshgrrdhishdpnhgspghrtghpthht ohepfedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepuggvmhhiohgsvghnohhurh esghhmrghilhdrtghomhdprhgtphhtthhopehquhgsvghsqdguvghvvghlsehgohhoghhl vghgrhhouhhpshdrtghomhdprhgtphhtthhopeguvghvvghlsehsphgvtghtrhhumhdqoh hsrdhorhhg X-ME-Proxy: Feedback-ID: i12284293:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 6 Dec 2025 09:53:03 -0500 (EST) Received: by fw12.qyliss.net (Postfix, from userid 1000) id 6380748FBD3D; Sat, 06 Dec 2025 15:53:01 +0100 (CET) From: Alyssa Ross To: Demi Marie Obenour Subject: Re: [qubes-devel] openQA bisectability In-Reply-To: <5017a410-0cee-4f15-8eed-dfd836cd42a0@gmail.com> References: <87bjkblrt0.fsf@alyssa.is> <21226c74-35a0-4bb3-b8f3-1163d4f5ad10@gmail.com> <878qfflrjm.fsf@alyssa.is> <875xajlojr.fsf@alyssa.is> <9e46e5d0-b2a7-4753-afcd-61ea74d1ddf9@gmail.com> <87345nlmf2.fsf@alyssa.is> <5017a410-0cee-4f15-8eed-dfd836cd42a0@gmail.com> Date: Sat, 06 Dec 2025 15:52:59 +0100 Message-ID: <87zf7vk6gk.fsf@alyssa.is> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Message-ID-Hash: BZSBXJ36Z7CL7BUCIKO42RREGZR64IFM X-Message-ID-Hash: BZSBXJ36Z7CL7BUCIKO42RREGZR64IFM X-MailFrom: hi@alyssa.is X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Spectrum OS Development , Qubes Developer Mailing List X-Mailman-Version: 3.3.9 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Demi Marie Obenour writes: > On 12/6/25 09:22, Alyssa Ross wrote: >> Demi Marie Obenour writes: >>=20 >>> On 12/6/25 08:36, Alyssa Ross wrote: >>>> Demi Marie Obenour writes: >>>> >>>>> On 12/6/25 07:32, Alyssa Ross wrote: >>>>>> Demi Marie Obenour writes: >>>>>> >>>>>>> On 12/6/25 07:26, Alyssa Ross wrote: >>>>>>>> Demi Marie Obenour writes: >>>>>>>> >>>>>>>>> While trying to sandbox the file chooser portal, I broke it. >>>>>>>>> This caused files not to be saved, resulting in silent data loss. >>>>>>>>> Unfortunately, the integration test still passed. >>>>>>>>> >>>>>>>>> Is this a bug in the test? Is there a better alternative to manu= al >>>>>>>>> testing? >>>>>>>> >>>>>>>> Not presently, but we can work on improving the test. The current >>>>>>>> portal test was written as a regression test for a specific issue = we >>>>>>>> had. It's quite hard to test completely end to end but we could d= o a >>>>>>>> lot better. >>>>>>>> >>>>>>>> I would quite like to spend some time in February or so working on= our >>>>>>>> tests. >>>>>>> >>>>>>> Would it make sense to use openQA for this? Qubes OS uses openQA >>>>>>> and it works very well. openQA is written in Perl, but it=E2=80=99= s the >>>>>>> best tool I know of for this. >>>>>> >>>>>> First blocker there would be packaging openQA in Nixpkgs. I do not >>>>>> personally relish the idea of doing that. >>>>> >>>>> Would it be possible to instead use a Fedora container? openQA is >>>>> packaged in Fedora. Qubes OS uses dedicated CI machines for openQA, >>>>> so I'm not worried about whether this would be permitted on your dev >>>>> box or the binary cache builders. >>>>> >>>>> I use Fedora for everything that isn't Spectrum-related dev work, >>>>> so I know how to maintain a Fedora system. That said, a container >>>>> shouldn't need much (if any) ongoing maintenance. >>>> >>>> I think the hermicity and bisectability of our build and tests are >>>> important properties worth preserving. We lose that if we start relyi= ng >>>> on an opaque container image. If an openQA update breaks something, >>>> it's not possible to easily figure out why. >>> Fedora container images contain an RPM database that can be used >>> to determine which packages changed. There will likely be many >>> packages that changed between images, but the same is true of Nixpkgs. >>> I totally agree that using a mutable Fedora system that is upgraded >>> in-place would be a mistake. >>=20 >> This is not sufficient for bisectability, because I have no access to >> intermediate steps between the two images. > > How is Nixpkgs better in this regard? Is it because Nixpkgs only > changes one package at a time and has a linear history? Yes, exactly. It can be surprising to people used to traditional packaging systems how much of a productivity win this is. See also: https://gitlab.postmarketos.org/postmarketOS/postmarketos/-/issues/94 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQQGoGac7QfI+H5ZtFCZddwkt31pFQUCaTRDSwAKCRCZddwkt31p FXaeAQCAqcfhCz9gAgPINi2YklTDDZUlL0u4MtxDPv9peXfnmwEA2FRwY0SYb2eJ ljPZ3WofNdDlwvt5NbO8k6ysYJfEDAw= =sMu+ -----END PGP SIGNATURE----- --=-=-=--