Qubes-lite With KVM and Wayland

Alyssa Ross
2021-03-09T16:59:50Z

[-- Attachment #1: Type: text/plain, Size: 3845 bytes --]

On Sun, Mar 07, 2021 at 12:52:36PM +0000, Thomas Leonard wrote:
> On Wed, 27 Jan 2021 at 17:31, Thomas Leonard <talex5@gmail.com> wrote:
> [...]
> > If any of this sounds useful for spectrum let me know. I can try and
> > tidy it up; it's all a huge mess at the moment!
>
> I got a bit further (fixed my sommelier problems), but have run out of
> time for now :-(
>
> I've written up where I got to here:
>
> https://roscidus.com/blog/blog/2021/03/07/qubes-lite-with-kvm-and-wayland/

I saw this online the other day and started reading it without realising
it was you, and then I saw you were using Nix and thought "wow, that's
close to what I'm (not) doing", and then I saw the Spectrum section, and
then realised who the author was. :)

I'll quote a little from it and reply to bits:

> When I wanted a newer package (socat with vsock support, only just
> released) I just told Nix to install it from the latest Git checkout of
> nixpkgs.

I'm excited to learn that socat has vsock support now! That's going to
be very useful. I have a half-done patch somewhere that adds vsock
support to strace that I should finish up as well.

> True, my squashfs image is getting a bit big. Maybe I should instead
> make a minimal squashfs boot image, plus a shared directory of hard
> links to the required files. That would allow sharing the data with the
> host. I could also just share the whole /nix/store directory, if I
> wanted to make all host software available to guests.

I think the solution I will end up going with for this will be a custom
virtiofsd implementation that can implement some access controls. The
even simpler solution would be to seperately expose every store path we
want to share as a virtio-fs device, but that's a lot of virtio devices!
(I vaguely remember the maximum might be as low as 16, too).

> I didn’t have time to write and debug C++ code for every missing
> Wayland protocol, so I took a short-cut: I wrote my own Wayland library,
> ocaml-wayland, and then used that to write my own version of sommelier.
> With that, adding support for copying text was fairly easy.

Well this is interesting! I definitely want to learn more about this.

> * One problem with virtwl is that, while we can receive shared
> memory FDs from the host, we can’t export guest memory to the
> host. This is unfortunate, because in Wayland the shared memory for
> window contents is allocated by the application from guest memory,
> and the proxy therefore has to copy each frame. If the host
> provided the memory to the guest, this wouldn’t be needed. There
> is a wl_drm protocol for allocating video memory, which might help
> here, but I don’t know how that works and, like many Wayland
> specifications, it seems to be in the process of being replaced by
> something else.

Yeah, this comes up on the virtio mailing list from time to time. It's
a very difficult problem to solve, but there might be a solution some
day. I think I've written about my own explorations in this area on
this list before.

> I’m not sure how guest-to-guest communication works with KVM.

It... doesn't really, at least not the way it does with Xen.
virtio-vhost-user[1] is promising, but very early stages. I've talked
in quite a lot of detail about how that works on this list before as
well. guest-to-guest communication was my main area of work for most of
the second half of last year (and what ended up causing me to burn out).

[1]: https://wiki.qemu.org/Features/VirtioVhostUser

> I hope the SpectrumOS project will resume at some point

Me too! Maybe it's resuming right now! (Although I'm not committing --
just because I'm feeling ready to get back into it today doesn't mean
that's going to be sustainable again yet.)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]