[RFC PATCH nixpkgs 0/9] virtio-gpu with crosvm and cloud-hypervisor

Alyssa Ross
2022-09-28T17:01:43Z

Introduction
------------

This series contains all the changes needed to make it possible to run
Wayland over cross-domain virtio-gpu using cloud-hypervisor. It only
contains changes needed to packages in Nixpkgs, so there's no Spectrum
integration here. That'll be coming soon.

There's also still some work to do here, hence being marked RFC — in
the cloud-hypervisor patch you'll spot a few TODOs and FIXMEs — but
it's ready for other people to have a look at and test.

With these changes, it's possible to run a cloud-hypervisor VM with a
GPU attached as follows:

crosvm device gpu --socket vhost-user-gpu.sock ...
cloud-hypervisor --gpu socket=vhost-user-gpu.sock ...

Then in the guest, just run a Wayland application under
wayland-proxy-virtwl:

wayland-proxy-virtwl --virtio-gpu hello-wayland

For Wayland over cross-domain virtio-gpu, which we're particularly
interested in, the full crosvm command line would be something like:

crosvm device gpu --socket vhost-user-gpu.sock \
--wayland-sock "$XDG_RUNTIME_DIR/$WAYLAND_DISPLAY" \
--params '{"context-types": "cross-domain"}'

Overview
--------

We start by upgrading crosvm to a beta version (since very recent
changes changed their vhost-user implementation in a way that makes it
significantly easier to interoperate with from cloud-hypervisor), then
applying a few small fixes for cases where crosvm had made assumptions
that were violated by cloud-hypervisor.

Next, we apply patches to cloud-hypervisor and some of its
dependencies, to implement the frontend of the GPU device.

Finally, we upgrade wayland-proxy-virtwl (the program that speaks
virtio-gpu and acts as the Wayland compositor inside the guest) to a
version that supports virtio-gpu. I'm using wayland-proxy-virtwl over
Sommelier here primary because Sommelier appears to have broken with
Linux 5.19, but another nice benefit is that wayland-proxy-virtwl is
maintained by somebody that we know and talk to sometimes on
discuss@spectrum-os.org.

Upstreamability
---------------

A question anybody looking at the volume of changes here should be
asking is: "To what extent is this upstreamable?". It's a bit of a
mixed bag — most notably, cloud-hypervisor is not interested in
virtio-gpu[1], so we'll have to maintain the cloud-hypervisor patches
ourselves until we can come up with a more upstream-friendly solution
(e.g. a vfio-user implementation of virtio-gpu). We also can't
upstream the changes to the vhost crate, as the protocol we are using
is crosvm specific, and so some standardisation work would need to
happen there.

But we can upstream the changes to crosvm, and the virtio-bindings
crate. I've already started on the latter. As for the patches to
Nixpkgs itself here, we can upstream the crosvm update (although
should probably target 105 if I get round to it soon, since 106 is
still beta), the rustPlatform change, and the updates to
wayland-proxy-virtwl and the wayland library it depends on.

[1]: https://github.com/cloud-hypervisor/cloud-hypervisor/discussions/3960#discussioncomment-2542915

Alyssa Ross (9):
crosvm: switch back to old git repo URL
crosvm.updateScript: update release branch format
crosvm: 104.0 -> 106.2
crosvm.updateScript: don't vendor Cargo.lock
crosvm: add fixes for cloud-hypervisor virtio-gpu
rustPlatform: forward unpack hooks to cargo fetch
cloud-hypervisor: add virtio-gpu support
ocamlPackages.wayland: 1.0 -> unstable-2022-05-07
wayland-proxy-virtwl: unstable-2021-12-05 -> unstable-2022-08-06

.../0001-build-use-local-vhost.patch | 39 +
...dings-regenerate-with-bindgen-0.60.1.patch | 2589 ++++++++++++
...0002-build-use-local-virtio-bindings.patch | 39 +
...gs-remove-workaround-for-old-bindgen.patch | 28 +
...-bindings-regenerate-with-Glibc-2.36.patch | 247 ++
...-devices-add-a-vhost-user-gpu-device.patch | 1253 ++++++
...-bindings-regenerate-with-Linux-5.19.patch | 1067 +++++
...tio-bindings-add-virtio-gpu-bindings.patch | 3587 +++++++++++++++++
.../cloud-hypervisor/default.nix | 64 +-
...ser-add-shared-memory-region-support.patch | 724 ++++
.../virtualization/crosvm/Cargo.lock | 2214 ----------
.../crosvm/default-seccomp-policy-dir.diff | 38 +-
.../virtualization/crosvm/default.nix | 15 +-
...-consider-shm-buuffers-when-setting-.patch | 34 +
...t_user-loosen-expected-message-order.patch | 71 +
...ces-vhost_user-remove-spurious-check.patch | 42 +
.../virtualization/crosvm/update.py | 15 +-
.../rust/build-rust-package/default.nix | 4 +-
.../ocaml-modules/wayland/default.nix | 12 +-
.../wayland/wayland-proxy-virtwl/default.nix | 14 +-
20 files changed, 9834 insertions(+), 2262 deletions(-)
create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0001-build-use-local-vhost.patch
create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0001-virtio-bindings-regenerate-with-bindgen-0.60.1.patch
create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0002-build-use-local-virtio-bindings.patch
create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0002-virtio-bindings-remove-workaround-for-old-bindgen.patch
create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0003-virtio-bindings-regenerate-with-Glibc-2.36.patch
create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0003-virtio-devices-add-a-vhost-user-gpu-device.patch
create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0004-virtio-bindings-regenerate-with-Linux-5.19.patch
create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0005-virtio-bindings-add-virtio-gpu-bindings.patch
create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/vhost-vhost_user-add-shared-memory-region-support.patch
delete mode 100644 pkgs/applications/virtualization/crosvm/Cargo.lock
create mode 100644 pkgs/applications/virtualization/crosvm/devices-properly-consider-shm-buuffers-when-setting-.patch
create mode 100644 pkgs/applications/virtualization/crosvm/devices-vhost_user-loosen-expected-message-order.patch
create mode 100644 pkgs/applications/virtualization/crosvm/devices-vhost_user-remove-spurious-check.patch

base-commit: 768879638cfd6a54664b762ec98a7f8f45620c44
--
2.37.1