Using virtio-gpu instead of virtwl

Thomas Leonard
2022-03-16T16:19:38Z

On Tue, 15 Mar 2022 at 20:23, Alyssa Ross <hi@alyssa.is> wrote:
>
> On Tue, Mar 15, 2022 at 02:06:04PM +0000, Alyssa Ross wrote:
> > On Sun, Mar 13, 2022 at 03:08:04PM +0000, Thomas Leonard wrote:
[ virtio-gpu ]
> > > I'm happy to test it now with Linux 5.16, but I think I need a newer
> > > crosvm too. Looks like support got added in
> > > https://github.com/google/crosvm/commit/231a54f36fbfb90ebac152d66f937af3644f6676,
> > > so it should be in 94.14150.
> > >
> > > Is there a new-enough package for crosvm available somewhere? NixOS
> > > only has 81.12871.0.0-rc1, and spectrum seems to be on
> > > 91.13904.0.0-rc2.
> >
> > I don't know of one. I haven't been keeping it up to date since we've
> > mostly switched over to cloud-hypervisor for Spectrum, but it doesn't
> > have any way of doing virtio-gpu (yet, porting crosvm's vhost-user-gpu
> > implementation is on my list).
> >
> > I'll have a go at updating it in Nixpkgs and report back.
>
> Okay, here's a package for crosvm 99.14468.0.0-rc1:
> https://github.com/NixOS/nixpkgs/pull/164306

Thanks - that's very useful!

Testing it from NixOS stable, it crashed with

[ERROR:src/linux.rs:3264] child pcivirtio-block (pid 47933) died:
signo 17, status 11, code 2

In the journal, I saw:

[src/panic_hook.rs:90] thread 'virtio_blk' panicked at 'Failed to
create an executor: Uring(CreatingContext(Setup(12)))',
devices/src/virtio/block/asynchronous.rs:767:50

From a brief look at the code, it looks like crosvm sees that the host
is running Linux >= 5.10 and creates a test uring, which works. Then
it tries to create a real one, and that fails. I fixed it by
increasing the limit on locked memory in configuration.nix:

security.pam.loginLimits = [
{
domain = "*";
type = "-";
item = "memlock";
value = "64000";
}
];

Then my VM booted and worked as before.

> I've tested some basic stuff, but haven't tested virtio-gpu. I don't
> see any reason it wouldn't work, though.

I tried running with `crosvm --gpu`, but after `modprobe virtio-gpu`,
crosvm crashed with:

[ 31.326763] Invalid ELF header magic: != ELF
[ 31.331450] [drm] pci: virtio-gpu-pci detected at 0000:00:07.0
[ 31.333020] [drm] Host memory window: 0x200000000 +0x200000000
[ 31.333983] [drm] features: +virgl -edid +resource_blob +host_visible
[ 31.333984] [drm] features: +context_init
[ 31.337289] [drm] number of scanouts: 1
[ 31.337938] [drm] number of cap sets: 1
[ERROR:src/linux.rs:3264] child pcivirtio-gpu (pid 63446) died: signo
17, status 11, code 2
[ERROR:devices/src/proxy.rs:212] failed write to child device process
pcivirtio-gpu: failed to send packet: Broken pipe (os error 32)

Then I tried with `--gpu=backend=2d` and that didn't crash, but
instead opened a window showing some bootloader stuff. I now have
/dev/dri/{card0, renderD128} devices, so I guess the next step is
figuring out what they do!

> Good luck! Let me know if I can help with anything else.

If you happen to know of any documentation, that would be great. But I
guess I probably just need to spend a load of time reading crosvm,
Linux, and Sommelier source code.

--
talex5 (GitHub/Twitter)