Investigate Video Frame Buffer Type

[mikhail-dcl]

LiveKit natively supports different video buffer types:

  public enum VideoFrameBufferType {
    [pbr::OriginalName("NATIVE")] Native = 0,
    [pbr::OriginalName("I420")] I420 = 1,
    [pbr::OriginalName("I420A")] I420A = 2,
    [pbr::OriginalName("I422")] I422 = 3,
    [pbr::OriginalName("I444")] I444 = 4,
    [pbr::OriginalName("I010")] I010 = 5,
    [pbr::OriginalName("NV12")] Nv12 = 6,
  }

Those formats differ by compression/quality and color depth.

However, in the Unity plugin all video up- and down-streams convert video frames into I420. You can track them by new ToI420Request();

• It produces an additional overhead
• We don't have control over downstream as it's possible to stream from any client that supports LiveKit

We need to investigate the problem of not bringing support for each format and potential ways of its resolution