Vhost-guest (was virtio vhost-user) vs virtio-msg

[Demi Marie Obenour <demiobenour@gmail.com>]

[-- Attachment #1.1.1: Type: text/plain, Size: 3195 bytes --]

On 5/28/26 01:47, Manos Pitsidianakis wrote:
> On Thu, May 28, 2026 at 8:22 AM Parav Pandit <parav@nvidia.com> wrote:
>>
>>
>>> From: Demi Marie Obenour <demiobenour@gmail.com>
>>> Sent: 28 May 2026 05:23 AM
>>> To: virtio-comment@lists.linux.dev
>>> Subject: MSI-X vector limits and reserving a virtio device ID
>>>
>>> I'd like to reserve a virtio device ID for virtio vhost-guest,
>>> formally virtio vhost-user.  Would this be possible?
>>>
>> Vhost user is an implementation of the device.
>> I believe it stays as implementation and not a new device type.
> 
> This exactly.
> 
> Furthermore, we already have a mechanism for "providing" an arbitrary
> virtio device; it's called a transport.
> 
> Demi, I suggest you look into virtio-msg transport, which would allow
> you to do what you want.

I'm aware of virtio-msg, and in fact considered using it.  However, I
found that virtio-msg doesn't meet my requirements.

1. Virtio-msg needs to run over a transport of its own.  None of the
   proposed transports support KVM guests on x86.  FF-A is the only
   one that would make sense with KVM, but FF-A is specific to Arm.

2. Virtio-msg isn't compatible with existing frontend VMMs.
   Vhost-guest can be used with any frontend VMM that implements
   vhost-user.

3. Virtio-msg isn't compatible with existing frontend drivers.  While I
   expect that drivers for Linux will eventually be upstreamed,
   I doubt that drivers for Windows or *BSD will ever be written.
   I don't even know if the Windows Driver Framework provides enough
   to write one.  I don't need this myself, but I suspect that this
   is enough to make virtio-msg unsuitable in many environments.
   
4. Virtio-msg requires invasive changes to existing userspace device
   implementations.  The project I work on doesn't use QEMU, and the
   existing frontends are targeted at server use-cases.  Using the
   vhost-user protocol lets me reuse these with little effort.

5. To the best of my knowledge, virtio-msg doesn't support live
   migration of frontend VMs.  Vhost-guest uses the vhost-user
   protocol, which has supported this for a very long time.  I don't
   need live migration myself, but for many server use-cases, not
   having live migration is a dealbreaker.
   
6. I don't know if virtio-msg can achieve comparable performance.
   It appears to be optimized for reliability and isolation,
   not processing tens of gigabits of network traffic per second.
   Vhost-guest is designed with performance in mind.

Vhost-guest is designed for storage and networking appliances on
servers, whereas virtio-msg is designed for safety-critical embedded
systems.  These domains have very different requirements, and as a
result they arrived at very different solutions.

I work on Spectrum (https://spectrum-os.org), which uses Cloud
Hypervisor.  As the name implies, Cloud Hypervisor is primarily
intended for cloud workloads, though it can also be used on clients.
I don't think that an implementation of virtio-msg would be accepted,
as it benefits none of Cloud Hypervisor's other users.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]