On Sun, Mar 07, 2021 at 12:52:36PM +0000, Thomas Leonard wrote: > On Wed, 27 Jan 2021 at 17:31, Thomas Leonard wrote: > [...] > > If any of this sounds useful for spectrum let me know. I can try and > > tidy it up; it's all a huge mess at the moment! > > I got a bit further (fixed my sommelier problems), but have run out of > time for now :-( > > I've written up where I got to here: > > https://roscidus.com/blog/blog/2021/03/07/qubes-lite-with-kvm-and-wayland/ I saw this online the other day and started reading it without realising it was you, and then I saw you were using Nix and thought "wow, that's close to what I'm (not) doing", and then I saw the Spectrum section, and then realised who the author was. :) I'll quote a little from it and reply to bits: > When I wanted a newer package (socat with vsock support, only just > released) I just told Nix to install it from the latest Git checkout of > nixpkgs. I'm excited to learn that socat has vsock support now! That's going to be very useful. I have a half-done patch somewhere that adds vsock support to strace that I should finish up as well. > True, my squashfs image is getting a bit big. Maybe I should instead > make a minimal squashfs boot image, plus a shared directory of hard > links to the required files. That would allow sharing the data with the > host. I could also just share the whole /nix/store directory, if I > wanted to make all host software available to guests. I think the solution I will end up going with for this will be a custom virtiofsd implementation that can implement some access controls. The even simpler solution would be to seperately expose every store path we want to share as a virtio-fs device, but that's a lot of virtio devices! (I vaguely remember the maximum might be as low as 16, too). > I didn’t have time to write and debug C++ code for every missing > Wayland protocol, so I took a short-cut: I wrote my own Wayland library, > ocaml-wayland, and then used that to write my own version of sommelier. > With that, adding support for copying text was fairly easy. Well this is interesting! I definitely want to learn more about this. > * One problem with virtwl is that, while we can receive shared > memory FDs from the host, we can’t export guest memory to the > host. This is unfortunate, because in Wayland the shared memory for > window contents is allocated by the application from guest memory, > and the proxy therefore has to copy each frame. If the host > provided the memory to the guest, this wouldn’t be needed. There > is a wl_drm protocol for allocating video memory, which might help > here, but I don’t know how that works and, like many Wayland > specifications, it seems to be in the process of being replaced by > something else. Yeah, this comes up on the virtio mailing list from time to time. It's a very difficult problem to solve, but there might be a solution some day. I think I've written about my own explorations in this area on this list before. > I’m not sure how guest-to-guest communication works with KVM. It... doesn't really, at least not the way it does with Xen. virtio-vhost-user[1] is promising, but very early stages. I've talked in quite a lot of detail about how that works on this list before as well. guest-to-guest communication was my main area of work for most of the second half of last year (and what ended up causing me to burn out). [1]: https://wiki.qemu.org/Features/VirtioVhostUser > I hope the SpectrumOS project will resume at some point Me too! Maybe it's resuming right now! (Although I'm not committing -- just because I'm feeling ready to get back into it today doesn't mean that's going to be sustainable again yet.)