Cannot build release.nix without KVM

Cannot build release.nix without KVM

[Demi Marie Obenour]

[-- Attachment #1.1.1: Type: text/plain, Size: 1179 bytes --]

On a Qubes OS VM (which doesn't have nested virtualization), building
release.nix fails because start-vmm-test (rightly) errors out.  Having
KVM be a *build*-time dependency of Spectrum OS seems rather strict,
especially because I can always copy the generated files over to another
machine and run them there.  In particular, I can copy the resulting image
and run it in QEMU.

Ideally I could use Nix's remote build support for this.  However, that
is designed for the case where I am offloading work to a powerful and
trusted server, whereas in my case the remote machine is for testing
purposes only.  Therefore, to save money, it is cheap, and it is also
generally treated as untrusted because I don't hold it to the same
standards I hold my highly trusted main laptop to.  For instance, it
doesn't have credentials to send emails.

Is it possible to copy what I need for the tests (using 'nix copy'
or 'nix-copy-closure' perhaps?) and skip them locally?  Can I run
the tests that do not need KVM while still running the rest?  This
is probably simple for anyone familiar with Nix, but that is not
me.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Cannot build release.nix without KVM

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 2351 bytes --]

Demi Marie Obenour <demiobenour@gmail.com> writes:

> On a Qubes OS VM (which doesn't have nested virtualization), building
> release.nix fails because start-vmm-test (rightly) errors out.  Having
> KVM be a *build*-time dependency of Spectrum OS seems rather strict,
> especially because I can always copy the generated files over to another
> machine and run them there.  In particular, I can copy the resulting image
> and run it in QEMU.

Ah, I've made a mistake here I think. KVM shouldn't be a build
dependency, because the Scaleway aarch64 builders used to populate the
binary cache don't have it.  It's only recently that release.nix
includes start-vmm-test, so I haven't run those builders since, and
hadn't noticed it.

One option would be to introduce a build configuration[1] option that
just disables any tests that need KVM.  Another would be to run those
tests inside QEMU TCG if KVM is available, which would have the benefit
of the tests still running (and not introducing more config options).  I
wonder what the most lightweight way to optionally run some code in a
TCG VM would be…

[1]: https://spectrum-os.org/doc/development/build-configuration.html

> Ideally I could use Nix's remote build support for this.  However, that
> is designed for the case where I am offloading work to a powerful and
> trusted server, whereas in my case the remote machine is for testing
> purposes only.  Therefore, to save money, it is cheap, and it is also
> generally treated as untrusted because I don't hold it to the same
> standards I hold my highly trusted main laptop to.  For instance, it
> doesn't have credentials to send emails.

Using a TCG VM as a remote builder would also be an option, but it would
probably be nicer to have the build take care of that.

> Is it possible to copy what I need for the tests (using 'nix copy'
> or 'nix-copy-closure' perhaps?) and skip them locally?  Can I run
> the tests that do not need KVM while still running the rest?  This
> is probably simple for anyone familiar with Nix, but that is not
> me.

You can pass --keep-going to your build command, and then any failures
won't get in the way of the rest of the build.  nix-output-monitor[2]
gives nice output that makes it obvious what failed.

[2]: https://github.com/maralorn/nix-output-monitor

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Cannot build release.nix without KVM

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 1412 bytes --]

Alyssa Ross <hi@alyssa.is> writes:

> Demi Marie Obenour <demiobenour@gmail.com> writes:
>
>> On a Qubes OS VM (which doesn't have nested virtualization), building
>> release.nix fails because start-vmm-test (rightly) errors out.  Having
>> KVM be a *build*-time dependency of Spectrum OS seems rather strict,
>> especially because I can always copy the generated files over to another
>> machine and run them there.  In particular, I can copy the resulting image
>> and run it in QEMU.
>
> Ah, I've made a mistake here I think. KVM shouldn't be a build
> dependency, because the Scaleway aarch64 builders used to populate the
> binary cache don't have it.  It's only recently that release.nix
> includes start-vmm-test, so I haven't run those builders since, and
> hadn't noticed it.
>
> One option would be to introduce a build configuration[1] option that
> just disables any tests that need KVM.  Another would be to run those
> tests inside QEMU TCG if KVM is available, which would have the benefit
> of the tests still running (and not introducing more config options).  I
> wonder what the most lightweight way to optionally run some code in a
> TCG VM would be…
>
> [1]: https://spectrum-os.org/doc/development/build-configuration.html

Sadly the obvious solution of running it under QEMU user emulation
doesn't work…

https://gitlab.com/qemu-project/qemu/-/issues/871

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Cannot build release.nix without KVM

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 1958 bytes --]

Alyssa Ross <hi@alyssa.is> writes:

> Alyssa Ross <hi@alyssa.is> writes:
>
>> Demi Marie Obenour <demiobenour@gmail.com> writes:
>>
>>> On a Qubes OS VM (which doesn't have nested virtualization), building
>>> release.nix fails because start-vmm-test (rightly) errors out.  Having
>>> KVM be a *build*-time dependency of Spectrum OS seems rather strict,
>>> especially because I can always copy the generated files over to another
>>> machine and run them there.  In particular, I can copy the resulting image
>>> and run it in QEMU.
>>
>> Ah, I've made a mistake here I think. KVM shouldn't be a build
>> dependency, because the Scaleway aarch64 builders used to populate the
>> binary cache don't have it.  It's only recently that release.nix
>> includes start-vmm-test, so I haven't run those builders since, and
>> hadn't noticed it.
>>
>> One option would be to introduce a build configuration[1] option that
>> just disables any tests that need KVM.  Another would be to run those
>> tests inside QEMU TCG if KVM is available, which would have the benefit
>> of the tests still running (and not introducing more config options).  I
>> wonder what the most lightweight way to optionally run some code in a
>> TCG VM would be…
>>
>> [1]: https://spectrum-os.org/doc/development/build-configuration.html
>
> Sadly the obvious solution of running it under QEMU user emulation
> doesn't work…
>
> https://gitlab.com/qemu-project/qemu/-/issues/871

… and even if it did, QEMU user emulation doesn't implement the KVM API.

Anyway, I've come to the conclusion that since start-vmm-test isn't
testing anything that isn't covered by the integration tests we recently
gained (which run a full Spectrum system in QEMU, and can therefore use
TCG), it can just be removed, and I've now done so.  I think that means
that everything can now be built without KVM, so the problem is solved
for now.  (Let me know if not.)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]