Re: [PATCH v9 0/1] virtio-media: Add device specification

Re: [PATCH v9 0/1] virtio-media: Add device specification

[Demi Marie Obenour]

[-- Attachment #1.1.1: Type: text/plain, Size: 4243 bytes --]

On 7/9/25 05:34, Albert Esteve wrote:
> New attempt of including virtio-media
> device specification.
> 
> v8->v9:
> - Rebased to virtio-1.4
> 
> v7->v8:
> - Merged DQBUF and DQEVENT into the same
>   item for unsupported ioctls
> - Rewrote/clarified a couple paragraphs
>   based on previous version reviews.
> 
> Virtio-media came from a discussion on virtio-dev
> mailing list, which lead to Alex Courbot presenting
> virtio-v4l2[1] specification as an alternative to
> virtio-video.
> 
> Later, virtio-v4l2 was renamed to virtio-media[2]
> and published at:
> 
> https://github.com/chromeos/virtio-media
> 
> The repository above includes a virtio-media driver able
> to pass v4l2-compliance when proxying the vivid/vicodec
> virtual devices or an actual UVC camera using the
> V4L2 vhost device (available in the repository).
> It also includes a FFmpeg-based video encoder
> device. Steps to reproduce are also detailed[3].
> 
> Recently, virtio-media has landed in AOSP[4].
> Also the driver patch has been sent to the
> kernel, currently in its v2 [5].
> 
> Furthermore, virtio-media got a proposal to reserve
> device ID 48, which was finally approved for
> inclusion in v1.4.
> 
> There is some overlap with virtio-video in regards
> to which devices it can handle. However, they take
> different approaches, potentially making them
> the preferable choice for different scenarios.
> Furthermore, virtio-media has matured since it
> was first presented, and there are public efforts
> to have upstream driver and devices.
> 
> But mainly, given that virtio-media will be the virtualization
> solution for media devices for ChromeOS, Android, and
> possibly others, I think is important that it gets
> standardized and included in the specification,
> despite the aforementioned overlap.
> 
> Full PDF: https://drive.google.com/file/d/1sDZXWVlvPbHKKguhekCXMzkpNZZ0g3hV/view?usp=sharing
> PDF with the media section only: https://drive.google.com/file/d/1JjOIzecnkFFlpFBRjPQEUyAOD3BDHAdw/view?usp=sharing
> 
> [1] https://mail.google.com/mail/u/0?ui=2&ik=73ebd65ebd&attid=0.1&permmsgid=msg-f:1767388565327924962&th=1887068940754ee2&view=att&disp=inline&realattid=f_libalimc0
> [2] https://www.mail-archive.com/virtio-dev@lists.oasis-open.org/msg12665.html
> [3] https://github.com/chromeos/virtio-media/blob/main/TRY_IT_OUT.md
> [4] https://cs.android.com/android/platform/superproject/main/+/main:external/virtio-media/
> [5] https://lore.kernel.org/lkml/20250201-virtio-media-v2-1-ac840681452d@gmail.com/T/
> 
> Albert Esteve (1):
>   virtio-media: Add virtio media device specification
> 
>  conformance.tex                           |   6 +
>  content.tex                               |   1 +
>  device-types/media/description.tex        | 639 ++++++++++++++++++++++
>  device-types/media/device-conformance.tex |  15 +
>  device-types/media/driver-conformance.tex |  11 +
>  5 files changed, 672 insertions(+)
>  create mode 100644 device-types/media/description.tex
>  create mode 100644 device-types/media/device-conformance.tex
>  create mode 100644 device-types/media/driver-conformance.tex

It's probably too late, but I have some serious concerns about this
device.  Specifically:

1. I don't see a reasonable way to support untrusted virtio-media
   devices.  v4l2 has too many ioctls and I can't realistically
   see a way to enforce that the return values from all of them
   are consistent with each other.  It is possible to only allow
   a fixed format (such as uncompressed ARGB) and only allow the
   video source and/or sink to provide resolutions and frame rates,
   but this means losing a lot of performance and abandoning any
   attempt at zero-copy.

   The use-case for untrusted virtio-media devices is not
   confidential computing, but rather disaggregated systems where
   video sources (like webcams) may be attached to untrusted virtual
   machines.

2. v4l2 is a chatty protocol and using it implies a lot of
   guest <=> host round-trips.  This is bad for performance.
   Is this overhead actually significant, and if so, are there
   plans to reduce it?
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v9 0/1] virtio-media: Add device specification

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 2464 bytes --]

Hi Demi,

Le mercredi 09 juillet 2025 à 13:26 -0400, Demi Marie Obenour a écrit :
> On 7/9/25 05:34, Albert Esteve wrote:
> > 
[...]

> 
> It's probably too late, but I have some serious concerns about this
> device.  Specifically:
> 
> 1. I don't see a reasonable way to support untrusted virtio-media
>    devices.  v4l2 has too many ioctls and I can't realistically
>    see a way to enforce that the return values from all of them
>    are consistent with each other.  It is possible to only allow
>    a fixed format (such as uncompressed ARGB) and only allow the
>    video source and/or sink to provide resolutions and frame rates,
>    but this means losing a lot of performance and abandoning any
>    attempt at zero-copy.
> 
>    The use-case for untrusted virtio-media devices is not
>    confidential computing, but rather disaggregated systems where
>    video sources (like webcams) may be attached to untrusted virtual
>    machines.

Please consider that you are posting broadly and understanding of what you mean
by untrusted virtio-media can be difficult for non-virtio devs like me.

With the enclosed information, I believe implementer are not forced to map to a
host V4L2 driver. You can implement a userspace layer in between, and possibly
even implement reclaiming by simulation some hot-unplug of the devices, or by
not scheduling any work for M2M.

I don't really see the relation between "trust" and the pixel format selection.
Nor why preventing zero-copy would help handling untrusted virtual machines. I
do believe you should not attempt this if you don't have the required knowledge
though, as its unlikely the end-result will be safe.

For me the main issue for untrusted VM is the lack of control measure, such as
cgroup for resource usage. This is being worked on though, Maxime Ripard have
given an update at the Linux Media Summit in Nice.

> 
> 2. v4l2 is a chatty protocol and using it implies a lot of
>    guest <=> host round-trips.  This is bad for performance.
>    Is this overhead actually significant, and if so, are there
>    plans to reduce it?

You should try and demonstrate this. Virtio Video tried to make their own and
endup with something kind of similar with about the same level of back and
forth. We already had few concerns with ioctl() overhead, and some measure have
been implemented, notably s_ctrl_ext.

Nicolas

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH v9 0/1] virtio-media: Add device specification

[Demi Marie Obenour]

[-- Attachment #1.1.1: Type: text/plain, Size: 3183 bytes --]

On 7/9/25 13:58, Nicolas Dufresne wrote:
> Hi Demi,
> 
> Le mercredi 09 juillet 2025 à 13:26 -0400, Demi Marie Obenour a écrit :
>> On 7/9/25 05:34, Albert Esteve wrote:
>>>
> [...]
> 
>>
>> It's probably too late, but I have some serious concerns about this
>> device.  Specifically:
>>
>> 1. I don't see a reasonable way to support untrusted virtio-media
>>    devices.  v4l2 has too many ioctls and I can't realistically
>>    see a way to enforce that the return values from all of them
>>    are consistent with each other.  It is possible to only allow
>>    a fixed format (such as uncompressed ARGB) and only allow the
>>    video source and/or sink to provide resolutions and frame rates,
>>    but this means losing a lot of performance and abandoning any
>>    attempt at zero-copy.
>>
>>    The use-case for untrusted virtio-media devices is not
>>    confidential computing, but rather disaggregated systems where
>>    video sources (like webcams) may be attached to untrusted virtual
>>    machines.
> 
> Please consider that you are posting broadly and understanding of what you mean
> by untrusted virtio-media can be difficult for non-virtio devs like me.

My use-case is this:

1. There is a VM that has all USB devices.
2. Some of those USB devices, like untrusted webcams, may be
   passed through to another VM via usbip.
3. The VM with the usbip device provides a video device to yet another VM.

The VM with the webcam and the VM receiving the video stream do not trust
each other.

> With the enclosed information, I believe implementer are not forced to map to a
> host V4L2 driver. You can implement a userspace layer in between, and possibly
> even implement reclaiming by simulation some hot-unplug of the devices, or by
> not scheduling any work for M2M.

I expect that any implementation that cares about security is going to do this.

> I don't really see the relation between "trust" and the pixel format selection.
> Nor why preventing zero-copy would help handling untrusted virtual machines. I
> do believe you should not attempt this if you don't have the required knowledge
> though, as its unlikely the end-result will be safe.

How can I validate the various parameters provided by a backend?

> For me the main issue for untrusted VM is the lack of control measure, such as
> cgroup for resource usage. This is being worked on though, Maxime Ripard have
> given an update at the Linux Media Summit in Nice.

Denial of service is not a concern in my use-cases.

>> 2. v4l2 is a chatty protocol and using it implies a lot of
>>    guest <=> host round-trips.  This is bad for performance.
>>    Is this overhead actually significant, and if so, are there
>>    plans to reduce it?
> 
> You should try and demonstrate this. Virtio Video tried to make their own and
> endup with something kind of similar with about the same level of back and
> forth. We already had few concerns with ioctl() overhead, and some measure have
> been implemented, notably s_ctrl_ext.
I'll leave this for later.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]