From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id C41463D25; Sat, 06 Dec 2025 14:23:06 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 070403D1F; Sat, 06 Dec 2025 14:23:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_MISSING,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=4.0.1 Received: from fout-a4-smtp.messagingengine.com (fout-a4-smtp.messagingengine.com [103.168.172.147]) by atuin.qyliss.net (Postfix) with ESMTPS id 011C23DC2 for ; Sat, 06 Dec 2025 14:23:03 +0000 (UTC) Received: from phl-compute-01.internal (phl-compute-01.internal [10.202.2.41]) by mailfout.phl.internal (Postfix) with ESMTP id 5161DEC03CA; Sat, 6 Dec 2025 09:23:01 -0500 (EST) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-01.internal (MEProxy); Sat, 06 Dec 2025 09:23:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alyssa.is; h=cc :cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1765030981; x=1765117381; bh=pC1CAveSTM +DQNAUAgeZjIvEiZJN4X4JNJpYZQQ+6bY=; b=MuTOM2xtwkV24U8uQy4FUrIhLC LmdEIwuXspL2kedKCNxsx/7FvRmy/DCk5OnYEFua02c284mAk532uL4dvFRLYAmO CzwqjjEL2Z5+E3spgR5MqrhRxbPc+uPJQtYHqY9ruUsDcWtQXo7jpeSnF2N4FQsb 8xnawBTkyc7QGIdXLLo90zjjNgaQx/7P1dAhQZoP4J2yW4o3ANxDEE+pYFYIoLyE ZudeDZRFxNRj6ROfUeSfE0cGNy2H3ObulV3No0nxgLK1+k2HY61Rzr6VahukBKjd 6Ap3rusDbpI1L5EHMmBHiSMCIXLVLLaKvbiXRzApGnA2bCGCANUIS2/Du7vg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1765030981; x=1765117381; bh=pC1CAveSTM+DQNAUAgeZjIvEiZJN4X4JNJp YZQQ+6bY=; b=bAkQiw8oDedVy97fhU6eIn3CKWIP15rESbx1kzaNzdIbiBVhSyU L35YRcd6VrFpoUPTgSp2YMZm2KhkS7PP/Qe1fijVWNuvwEjgrjj0D7SodseKgl5s pXnbL/0WIuhbkYXgxprn0maKIPt+gNpSKr5zuII5fH/MJX/mZ4Abgwc62U3Rz/zp eTmI+EtJ9Yjab3sT0fydbVqGwIwwJj1aI96hyrQBPQL30aH8OI0DyxUW+m+5CALA dBiTgbKQPtxFGOINZvdE686Iaca2niJaufk9l9wgKFg+gQlplbAHKOotGdly7iCe cUBcVG3holT3yLMR4Ibg7ClTwdn0PKPbHDA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefgedrtddtgdduudduiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufgjfhffkfggtgesghdtreertddtjeenucfhrhhomheptehlhihsshgrucft ohhsshcuoehhihesrghlhihsshgrrdhisheqnecuggftrfgrthhtvghrnhepteehvedugf ejgfehhfeijeduleekleejgedvkeeuuefhhfegvdevfeetveegteeinecuvehluhhsthgv rhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhephhhisegrlhihshhsrgdrih hspdhnsggprhgtphhtthhopedvpdhmohguvgepshhmthhpohhuthdprhgtphhtthhopegu vghmihhosggvnhhouhhrsehgmhgrihhlrdgtohhmpdhrtghpthhtohepuggvvhgvlhessh hpvggtthhruhhmqdhoshdrohhrgh X-ME-Proxy: Feedback-ID: i12284293:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 6 Dec 2025 09:23:00 -0500 (EST) Received: by fw12.qyliss.net (Postfix, from userid 1000) id 7C79C48FB82A; Sat, 06 Dec 2025 15:22:58 +0100 (CET) From: Alyssa Ross To: Demi Marie Obenour Subject: Re: Test for portals passes even when file portal is broken In-Reply-To: <9e46e5d0-b2a7-4753-afcd-61ea74d1ddf9@gmail.com> References: <87bjkblrt0.fsf@alyssa.is> <21226c74-35a0-4bb3-b8f3-1163d4f5ad10@gmail.com> <878qfflrjm.fsf@alyssa.is> <875xajlojr.fsf@alyssa.is> <9e46e5d0-b2a7-4753-afcd-61ea74d1ddf9@gmail.com> Date: Sat, 06 Dec 2025 15:22:57 +0100 Message-ID: <87345nlmf2.fsf@alyssa.is> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Message-ID-Hash: QU7QWERLDJ726R27HLKQSMYNELGQM74W X-Message-ID-Hash: QU7QWERLDJ726R27HLKQSMYNELGQM74W X-MailFrom: hi@alyssa.is X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Spectrum OS Development X-Mailman-Version: 3.3.9 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Demi Marie Obenour writes: > On 12/6/25 08:36, Alyssa Ross wrote: >> Demi Marie Obenour writes: >>=20 >>> On 12/6/25 07:32, Alyssa Ross wrote: >>>> Demi Marie Obenour writes: >>>> >>>>> On 12/6/25 07:26, Alyssa Ross wrote: >>>>>> Demi Marie Obenour writes: >>>>>> >>>>>>> While trying to sandbox the file chooser portal, I broke it. >>>>>>> This caused files not to be saved, resulting in silent data loss. >>>>>>> Unfortunately, the integration test still passed. >>>>>>> >>>>>>> Is this a bug in the test? Is there a better alternative to manual >>>>>>> testing? >>>>>> >>>>>> Not presently, but we can work on improving the test. The current >>>>>> portal test was written as a regression test for a specific issue we >>>>>> had. It's quite hard to test completely end to end but we could do a >>>>>> lot better. >>>>>> >>>>>> I would quite like to spend some time in February or so working on o= ur >>>>>> tests. >>>>> >>>>> Would it make sense to use openQA for this? Qubes OS uses openQA >>>>> and it works very well. openQA is written in Perl, but it=E2=80=99s = the >>>>> best tool I know of for this. >>>> >>>> First blocker there would be packaging openQA in Nixpkgs. I do not >>>> personally relish the idea of doing that. >>> >>> Would it be possible to instead use a Fedora container? openQA is >>> packaged in Fedora. Qubes OS uses dedicated CI machines for openQA, >>> so I'm not worried about whether this would be permitted on your dev >>> box or the binary cache builders. >>> >>> I use Fedora for everything that isn't Spectrum-related dev work, >>> so I know how to maintain a Fedora system. That said, a container >>> shouldn't need much (if any) ongoing maintenance. >>=20 >> I think the hermicity and bisectability of our build and tests are >> important properties worth preserving. We lose that if we start relying >> on an opaque container image. If an openQA update breaks something, >> it's not possible to easily figure out why. > Fedora container images contain an RPM database that can be used > to determine which packages changed. There will likely be many > packages that changed between images, but the same is true of Nixpkgs. > I totally agree that using a mutable Fedora system that is upgraded > in-place would be a mistake. This is not sufficient for bisectability, because I have no access to intermediate steps between the two images. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQQGoGac7QfI+H5ZtFCZddwkt31pFQUCaTQ8QQAKCRCZddwkt31p FWMjAP4zg9ygQAVvpKNc0sO6ogESUqxNomZg1kK8kLDh/NS62wEAhDDGa4m+prA6 p9Gctl7uTy8nqa+eAWFtui4aWxq7mAk= =rVgT -----END PGP SIGNATURE----- --=-=-=--