Show thumbnails of videos in media section

[skttl]

This isn't very helpful to an editor:

Can we show the video without controls, autoplay etc. instead?

If you put a video tag, with just a src pointing to the video, most browsers will download the first frame and show that.

[iOvergaard]

That's a good idea! One important architectural thing to fix beforehand is that collection views (like the media library grid) do not know which content types they are displaying.

They don't know what a "Video" is, or what an "Image" is either, for that matter.

That is why we ask the server for each media item, "Hey server, here is the GUID, what kind of thumbnail can you resolve for that, please?", and for image types, it will reply with an image generated through ImageSharp.

You can intercept these calls and add more providers to the list. For example, if you had a way to generate an image from a PDF file type, you could add that to the mix.

One way to resolve into <video /> elements for "Video" types is that we would need to add more metadata to those responses with a kind of "type": "Video" | "Image" or something similar, and for videos, we would then provide the source URL instead of the generated ImageSharp parameters.

Another way is to make the Backoffice aware of its media types in collection views, and more importantly, what they "mean," and then have a way to add extensions to the preview of grid items. We actually sort of do this already with the media preview you see, when you go into the details view of a media item, but that is based on the source file mimetype, for instance, "forMimeTypes": "image/*".

[skttl]

For example, if you had a way to generate an image from a PDF file type, you could add that to the mix.
https://github.com/skttl/ImageSharpCommunity.Formats.Pdf ;)

Not saying Umbraco should ship with that, but if we could make it extendable somehow, that would be very nice!

[skttl]

Is there any docs, or code examples on that?

[iOvergaard]

Your package already adds support to ImageSharp. So what's left is to add .pdf as a SupportedImageFileType. There is a number of ways to do that through configuration or providing an IImageUrlGenerator.

See how the factory behind the endpoint fetches the thumbnails here:

Umbraco-CMS/src/Umbraco.Cms.Api.Management/Factories/ReziseImageUrlFactory.cs

Lines 47 to 69 in 8dfeea1

	if (_imageUrlGenerator.SupportedImageFileTypes.InvariantContains(extension) is false)
	{
	// It's okay to return just the image URL for SVGs, as they are always scalable.
	if (extension == "svg")
	{
	yield return new MediaUrlInfo
	{
	Culture = null,
	Url = _absoluteUrlBuilder.ToAbsoluteUrl(url).ToString(),
	};
	}
	continue;
	}
	var options = new ImageUrlGenerationOptions(url)
	{
	Height = height,
	Width = width,
	ImageCropMode = mode,
	};
	var relativeUrl = _imageUrlGenerator.GetImageUrl(options);

Arguably, we have dug a small tech debt trench here by locking it a bit firmly unto ImageSharp, if you consider that other extension types could provide images as it were as well. You may also notice we have a special handling of .svg files in there. One could argue this calls for a different architecture. But the structure is mostly there to provide thumbnails from ImageSharp providers, just not from other sources such as videos.

[skttl]

Arh, that actually explains some things I've noticed.

When my PDF provider is installed, Umbraco actually thinks a pdf can be either an image or an "article", so when dragging and dropping into media, the editor gets to choose.

The backoffice actually tries to show the thumbnail too, but because the image url is missing the format parameter, ImageSharp will try to keep the existing format. And my provider doesn't encode as PDF, so that just throws an error.

[iOvergaard]

Good point about 'format'. We did not consider that before. Would it make sense to supply format: 'webp' by default for all thumbnails? If you presume all browsers that support web components would also support webp, we could make the requests quite a few bytes lighter by that and be able to support PDFs as well.

[skttl]

At least for me it would :)