From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on atuin.qyliss.net X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE autolearn=unavailable autolearn_force=no version=3.4.6 Received: from atuin.qyliss.net (localhost [IPv6:::1]) by atuin.qyliss.net (Postfix) with ESMTP id C86DC7CC5F; Wed, 28 Sep 2022 17:01:43 +0000 (UTC) Received: by atuin.qyliss.net (Postfix, from userid 496) id 4C38D7CC29; Wed, 28 Sep 2022 17:01:40 +0000 (UTC) Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by atuin.qyliss.net (Postfix) with ESMTPS id 449837CBB3 for ; Wed, 28 Sep 2022 17:01:38 +0000 (UTC) Received: by mail-ej1-x62e.google.com with SMTP id r18so28419644eja.11 for ; Wed, 28 Sep 2022 10:01:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=unikie.com; s=google; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date; bh=iRGsJsc3GrENUm2XpFLFxi2MISAkRHpNjnkLDZceGLQ=; b=DY+3URRniQGOxEigOTu9sHY16C+Z/xVd4TiF1Ikcx/RNyCiPvNHccaHC8GnPhAqiyi sc9dPwb1KWC8agLGDSlm4vY62KC9lcHXmi2oAZWC6BAUcSR2H/cwZ4/kWj7vtUklzviZ lsu2SvUP3bV8ynSJImBt1BF8aI3KgAcwtnnIe7kjR4OcK/6ZQs7NeUrYKszOOwjO5QrR uy9iiItV2mSHBxeUdkPSpVDJBr+UHhyLJVbCXCXJJjyZaHjHSdkhcMBdalgmjY6yAsF/ s8/XjACmIJcn8LvFfek4m7tI6qnfL9Ii+frc/r2DEwtraRzlwpjW43b9dDeSO3vtU4xr gG4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date; bh=iRGsJsc3GrENUm2XpFLFxi2MISAkRHpNjnkLDZceGLQ=; b=LZF4al6Z1bQcuDI9z0f/N/hxco7qLfsdHPU73vvaqR5GDeHAn5mJgp+kBzaPQYAIMR /mPaSSk1p3yfasoHTAPbkQfLcUCcWDKSTFKZ9GV6hh80orHAzDpKUi1NP96/kePWJzGk utWobQ0MG6tmdTTo4onFEZGD0CplPpccAv0XcHQuD/qXRjR/jz4Kyvb9VRg6q1+UEE/X //WWFBj8BbyQdo7wehHsA/fbkRNNVlhceUsmTtJPCrqQ9h7SnA5+xVkUsmLZQzgDZYPo wRRxs78+tezmfITKx2vveUAS1siKFpb5agC0za6vZ342yVJcV5z6rIWl7oIEwF7Mon+C pdRQ== X-Gm-Message-State: ACrzQf3I3SBaTh4z3jif4p2HwTDOyJUG/PvShutUoWFznwITouY6sA8w CEnxzVKc9LNzofupI+pnZIX8mL0hvkTwXCuS X-Google-Smtp-Source: AMsMyM7ptUdGqHcJ0/TlqUGXB2pj00lmoppxCMaCzoJO3kmJ6CMWskSTkS7hLkKObfjvKxjKMH8QvQ== X-Received: by 2002:a17:907:3fa9:b0:782:ed33:df8d with SMTP id hr41-20020a1709073fa900b00782ed33df8dmr20243357ejc.745.1664384497749; Wed, 28 Sep 2022 10:01:37 -0700 (PDT) Received: from x220.qyliss.net (p4feb786f.dip0.t-ipconnect.de. [79.235.120.111]) by smtp.gmail.com with ESMTPSA id e3-20020a170906844300b007415f8ffcbbsm2639984ejy.98.2022.09.28.10.01.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 28 Sep 2022 10:01:37 -0700 (PDT) Received: by x220.qyliss.net (Postfix, from userid 1000) id 19A6E2D3; Wed, 28 Sep 2022 17:01:37 +0000 (UTC) From: Alyssa Ross To: devel@spectrum-os.org Subject: [RFC PATCH nixpkgs 0/9] virtio-gpu with crosvm and cloud-hypervisor Date: Wed, 28 Sep 2022 17:01:19 +0000 Message-Id: <20220928170128.1583791-1-alyssa.ross@unikie.com> X-Mailer: git-send-email 2.37.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Message-ID-Hash: XTAJM2DDIAJUWPIWCO43W73ZTGP5SEQH X-Message-ID-Hash: XTAJM2DDIAJUWPIWCO43W73ZTGP5SEQH X-MailFrom: alyssa.ross@unikie.com X-Mailman-Rule-Hits: header-match-devel.spectrum-os.org-0 X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-config-1 CC: Ville Ilvonen X-Mailman-Version: 3.3.5 Precedence: list List-Id: Patches and low-level development discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Introduction ------------ This series contains all the changes needed to make it possible to run Wayland over cross-domain virtio-gpu using cloud-hypervisor. It only contains changes needed to packages in Nixpkgs, so there's no Spectrum integration here. That'll be coming soon. There's also still some work to do here, hence being marked RFC — in the cloud-hypervisor patch you'll spot a few TODOs and FIXMEs — but it's ready for other people to have a look at and test. With these changes, it's possible to run a cloud-hypervisor VM with a GPU attached as follows: crosvm device gpu --socket vhost-user-gpu.sock ... cloud-hypervisor --gpu socket=vhost-user-gpu.sock ... Then in the guest, just run a Wayland application under wayland-proxy-virtwl: wayland-proxy-virtwl --virtio-gpu hello-wayland For Wayland over cross-domain virtio-gpu, which we're particularly interested in, the full crosvm command line would be something like: crosvm device gpu --socket vhost-user-gpu.sock \ --wayland-sock "$XDG_RUNTIME_DIR/$WAYLAND_DISPLAY" \ --params '{"context-types": "cross-domain"}' Overview -------- We start by upgrading crosvm to a beta version (since very recent changes changed their vhost-user implementation in a way that makes it significantly easier to interoperate with from cloud-hypervisor), then applying a few small fixes for cases where crosvm had made assumptions that were violated by cloud-hypervisor. Next, we apply patches to cloud-hypervisor and some of its dependencies, to implement the frontend of the GPU device. Finally, we upgrade wayland-proxy-virtwl (the program that speaks virtio-gpu and acts as the Wayland compositor inside the guest) to a version that supports virtio-gpu. I'm using wayland-proxy-virtwl over Sommelier here primary because Sommelier appears to have broken with Linux 5.19, but another nice benefit is that wayland-proxy-virtwl is maintained by somebody that we know and talk to sometimes on discuss@spectrum-os.org. Upstreamability --------------- A question anybody looking at the volume of changes here should be asking is: "To what extent is this upstreamable?". It's a bit of a mixed bag — most notably, cloud-hypervisor is not interested in virtio-gpu[1], so we'll have to maintain the cloud-hypervisor patches ourselves until we can come up with a more upstream-friendly solution (e.g. a vfio-user implementation of virtio-gpu). We also can't upstream the changes to the vhost crate, as the protocol we are using is crosvm specific, and so some standardisation work would need to happen there. But we can upstream the changes to crosvm, and the virtio-bindings crate. I've already started on the latter. As for the patches to Nixpkgs itself here, we can upstream the crosvm update (although should probably target 105 if I get round to it soon, since 106 is still beta), the rustPlatform change, and the updates to wayland-proxy-virtwl and the wayland library it depends on. [1]: https://github.com/cloud-hypervisor/cloud-hypervisor/discussions/3960#discussioncomment-2542915 Alyssa Ross (9): crosvm: switch back to old git repo URL crosvm.updateScript: update release branch format crosvm: 104.0 -> 106.2 crosvm.updateScript: don't vendor Cargo.lock crosvm: add fixes for cloud-hypervisor virtio-gpu rustPlatform: forward unpack hooks to cargo fetch cloud-hypervisor: add virtio-gpu support ocamlPackages.wayland: 1.0 -> unstable-2022-05-07 wayland-proxy-virtwl: unstable-2021-12-05 -> unstable-2022-08-06 .../0001-build-use-local-vhost.patch | 39 + ...dings-regenerate-with-bindgen-0.60.1.patch | 2589 ++++++++++++ ...0002-build-use-local-virtio-bindings.patch | 39 + ...gs-remove-workaround-for-old-bindgen.patch | 28 + ...-bindings-regenerate-with-Glibc-2.36.patch | 247 ++ ...-devices-add-a-vhost-user-gpu-device.patch | 1253 ++++++ ...-bindings-regenerate-with-Linux-5.19.patch | 1067 +++++ ...tio-bindings-add-virtio-gpu-bindings.patch | 3587 +++++++++++++++++ .../cloud-hypervisor/default.nix | 64 +- ...ser-add-shared-memory-region-support.patch | 724 ++++ .../virtualization/crosvm/Cargo.lock | 2214 ---------- .../crosvm/default-seccomp-policy-dir.diff | 38 +- .../virtualization/crosvm/default.nix | 15 +- ...-consider-shm-buuffers-when-setting-.patch | 34 + ...t_user-loosen-expected-message-order.patch | 71 + ...ces-vhost_user-remove-spurious-check.patch | 42 + .../virtualization/crosvm/update.py | 15 +- .../rust/build-rust-package/default.nix | 4 +- .../ocaml-modules/wayland/default.nix | 12 +- .../wayland/wayland-proxy-virtwl/default.nix | 14 +- 20 files changed, 9834 insertions(+), 2262 deletions(-) create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0001-build-use-local-vhost.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0001-virtio-bindings-regenerate-with-bindgen-0.60.1.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0002-build-use-local-virtio-bindings.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0002-virtio-bindings-remove-workaround-for-old-bindgen.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0003-virtio-bindings-regenerate-with-Glibc-2.36.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0003-virtio-devices-add-a-vhost-user-gpu-device.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0004-virtio-bindings-regenerate-with-Linux-5.19.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0005-virtio-bindings-add-virtio-gpu-bindings.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/vhost-vhost_user-add-shared-memory-region-support.patch delete mode 100644 pkgs/applications/virtualization/crosvm/Cargo.lock create mode 100644 pkgs/applications/virtualization/crosvm/devices-properly-consider-shm-buuffers-when-setting-.patch create mode 100644 pkgs/applications/virtualization/crosvm/devices-vhost_user-loosen-expected-message-order.patch create mode 100644 pkgs/applications/virtualization/crosvm/devices-vhost_user-remove-spurious-check.patch base-commit: 768879638cfd6a54664b762ec98a7f8f45620c44 -- 2.37.1