Demi Marie Obenour writes: > On 12/6/25 08:36, Alyssa Ross wrote: >> Demi Marie Obenour writes: >> >>> On 12/6/25 07:32, Alyssa Ross wrote: >>>> Demi Marie Obenour writes: >>>> >>>>> On 12/6/25 07:26, Alyssa Ross wrote: >>>>>> Demi Marie Obenour writes: >>>>>> >>>>>>> While trying to sandbox the file chooser portal, I broke it. >>>>>>> This caused files not to be saved, resulting in silent data loss. >>>>>>> Unfortunately, the integration test still passed. >>>>>>> >>>>>>> Is this a bug in the test? Is there a better alternative to manual >>>>>>> testing? >>>>>> >>>>>> Not presently, but we can work on improving the test. The current >>>>>> portal test was written as a regression test for a specific issue we >>>>>> had. It's quite hard to test completely end to end but we could do a >>>>>> lot better. >>>>>> >>>>>> I would quite like to spend some time in February or so working on our >>>>>> tests. >>>>> >>>>> Would it make sense to use openQA for this? Qubes OS uses openQA >>>>> and it works very well. openQA is written in Perl, but it’s the >>>>> best tool I know of for this. >>>> >>>> First blocker there would be packaging openQA in Nixpkgs. I do not >>>> personally relish the idea of doing that. >>> >>> Would it be possible to instead use a Fedora container? openQA is >>> packaged in Fedora. Qubes OS uses dedicated CI machines for openQA, >>> so I'm not worried about whether this would be permitted on your dev >>> box or the binary cache builders. >>> >>> I use Fedora for everything that isn't Spectrum-related dev work, >>> so I know how to maintain a Fedora system. That said, a container >>> shouldn't need much (if any) ongoing maintenance. >> >> I think the hermicity and bisectability of our build and tests are >> important properties worth preserving. We lose that if we start relying >> on an opaque container image. If an openQA update breaks something, >> it's not possible to easily figure out why. > Fedora container images contain an RPM database that can be used > to determine which packages changed. There will likely be many > packages that changed between images, but the same is true of Nixpkgs. > I totally agree that using a mutable Fedora system that is upgraded > in-place would be a mistake. This is not sufficient for bisectability, because I have no access to intermediate steps between the two images.