From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id DD75B3938; Sat, 06 Dec 2025 13:37:06 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 993) id 2D3E6399C; Sat, 06 Dec 2025 13:37:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DMARC_MISSING,RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=4.0.1 Received: from fout-a3-smtp.messagingengine.com (fout-a3-smtp.messagingengine.com [103.168.172.146]) by atuin.qyliss.net (Postfix) with ESMTPS id 393ED390D for ; Sat, 06 Dec 2025 13:37:03 +0000 (UTC) Received: from phl-compute-04.internal (phl-compute-04.internal [10.202.2.44]) by mailfout.phl.internal (Postfix) with ESMTP id BE086EC03F6; Sat, 6 Dec 2025 08:37:00 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-04.internal (MEProxy); Sat, 06 Dec 2025 08:37:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alyssa.is; h=cc :cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1765028220; x=1765114620; bh=T80ch2mfLT Qy/8SwmeJgtGTKJVWaeA3W+0NEFyOTanw=; b=ax9jkLQUcPsH32LBggzVBBaMJ8 NkbGQx91gQy//2HQj+d1Nk1EmdkRlUh3AgK0GybFrvBb4dRRot5NSdaIPmVfL1+S 66U2BtqL/zUTHJFJP/J0P/kQC6B14fVe4NMCKYr31c5XPXOoeCSFnMeHUjDy/KKF 0nPTdzDQaJb1i3BuP+vdX4apjCaK4JPeJhiTvZIX30H96uN4OKrdyAKHUE3Norxl XVXxsHpGTshkbRrUKrilnPuyXdN3NkbVHzRt8Vff8Hjda08Rm66VCBy+U/KC6CrQ lbe3X0ko2cRSAjc6QahhIKgQYJl04TRKAfxmLKUTqXfFnsr00P3nOYplzgEA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1765028220; x=1765114620; bh=T80ch2mfLTQy/8SwmeJgtGTKJVWaeA3W+0N EFyOTanw=; b=aswkp5imP8l6Fi55RCEswZmG3Rz83s0QB3Mn6iUCQJdMhnmbilA AvEpy2fJACYVk6Q0feZW7FLlCMEZd2frWlxXKhRb0jPbVFueGXCA0L4w6R7aYlRl ydQK1wKpXgHBPjwkdNB13R9YI8QUmb3/opYlaDJzlgsGSklvWlgKie/erHwHQV64 iiCH0FJUR1mj6igc9X8odLJQcDSCIVVio7dfl4xIhwc0B2LTJidKwLB/YGRfqFcy gKMxo4J6NfhLCM9rU3L4PWk7cjZNkqKyjQ2ZU1JVed/m5uCOsDK6TX6rA5Y58zYf 30z6RRI1nV/7YO7YkCUpP3nzg5w2M0iXOHA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefgedrtddtgdduuddtjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjug hrpefhvfevufgjfhffkfggtgesghdtreertddtjeenucfhrhhomheptehlhihsshgrucft ohhsshcuoehhihesrghlhihsshgrrdhisheqnecuggftrfgrthhtvghrnhepteehvedugf ejgfehhfeijeduleekleejgedvkeeuuefhhfegvdevfeetveegteeinecuvehluhhsthgv rhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhephhhisegrlhihshhsrgdrih hspdhnsggprhgtphhtthhopedvpdhmohguvgepshhmthhpohhuthdprhgtphhtthhopegu vghmihhosggvnhhouhhrsehgmhgrihhlrdgtohhmpdhrtghpthhtohepuggvvhgvlhessh hpvggtthhruhhmqdhoshdrohhrgh X-ME-Proxy: Feedback-ID: i12284293:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 6 Dec 2025 08:36:59 -0500 (EST) Received: by fw12.qyliss.net (Postfix, from userid 1000) id 099D7485228E; Sat, 06 Dec 2025 14:36:58 +0100 (CET) From: Alyssa Ross To: Demi Marie Obenour Subject: Re: Test for portals passes even when file portal is broken In-Reply-To: References: <87bjkblrt0.fsf@alyssa.is> <21226c74-35a0-4bb3-b8f3-1163d4f5ad10@gmail.com> <878qfflrjm.fsf@alyssa.is> Date: Sat, 06 Dec 2025 14:36:56 +0100 Message-ID: <875xajlojr.fsf@alyssa.is> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Message-ID-Hash: QJCGR3562VIQ5F4QUXWWUZEFTXU7BB2J X-Message-ID-Hash: QJCGR3562VIQ5F4QUXWWUZEFTXU7BB2J X-MailFrom: hi@alyssa.is X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-devel.spectrum-os.org-0; header-match-devel.spectrum-os.org-1; header-match-devel.spectrum-os.org-2; header-match-devel.spectrum-os.org-3; header-match-devel.spectrum-os.org-4; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Spectrum OS Development X-Mailman-Version: 3.3.9 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Demi Marie Obenour writes: > On 12/6/25 07:32, Alyssa Ross wrote: >> Demi Marie Obenour writes: >>=20 >>> On 12/6/25 07:26, Alyssa Ross wrote: >>>> Demi Marie Obenour writes: >>>> >>>>> While trying to sandbox the file chooser portal, I broke it. >>>>> This caused files not to be saved, resulting in silent data loss. >>>>> Unfortunately, the integration test still passed. >>>>> >>>>> Is this a bug in the test? Is there a better alternative to manual >>>>> testing? >>>> >>>> Not presently, but we can work on improving the test. The current >>>> portal test was written as a regression test for a specific issue we >>>> had. It's quite hard to test completely end to end but we could do a >>>> lot better. >>>> >>>> I would quite like to spend some time in February or so working on our >>>> tests. >>> >>> Would it make sense to use openQA for this? Qubes OS uses openQA >>> and it works very well. openQA is written in Perl, but it=E2=80=99s the >>> best tool I know of for this. >>=20 >> First blocker there would be packaging openQA in Nixpkgs. I do not >> personally relish the idea of doing that. > > Would it be possible to instead use a Fedora container? openQA is > packaged in Fedora. Qubes OS uses dedicated CI machines for openQA, > so I'm not worried about whether this would be permitted on your dev > box or the binary cache builders. > > I use Fedora for everything that isn't Spectrum-related dev work, > so I know how to maintain a Fedora system. That said, a container > shouldn't need much (if any) ongoing maintenance. I think the hermicity and bisectability of our build and tests are important properties worth preserving. We lose that if we start relying on an opaque container image. If an openQA update breaks something, it's not possible to easily figure out why. I'm also not sure whether we'd be able to keep our tests as lightweight and fast as they currently are with openQA. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYKAB0WIQQGoGac7QfI+H5ZtFCZddwkt31pFQUCaTQxeAAKCRCZddwkt31p FVIgAP0Zm7YwuK/Q6/loScXjQ39+4GERmr/ZnKmlpurVJU49cgD/Tc0THn3Q4tMt Wa7ab2+fDFaApl4EyzPXkXJKvQ+LcwM= =XPh6 -----END PGP SIGNATURE----- --=-=-=--