Introduction

Since 2020, I got into HDR10 video production. I got an HDR10 monitor and started playing games and even recording them in HDR10. It's nice. But I've been wondering, if I can record videos in HDR10, why not take HDR10 images?

Well, turns out, images in true HDR10 hasn't been looked into a lot until recently. Formats like JPG and PNG do support higher bit depths than 8bpc (bits per channel). The idea is there for at least storing the colour information. But what about the HDR10 metadata? How will the monitor know how to calibrate the content to the monitor? Maybe HEIC can store the metadata just like HEVC in a video container? Let's look into it a little.

Let's clear up some confusion about HDR...

What really is HDR?

First off, HDR stands for High Dynamic Range. There's two completely different meanings for it and they are easily confused. So let's "Clara"-fy it.

1. HDR Imaging - This is where a photographist will take the same picture at different exposures and then merge all of them into a single image, causing a greater dynamic range in the resulting photo. The image taken is stored like a normal image (JPG, PNG, RAW, etc). In the case of a smartphone, it will likely still be 8bpc just like every other image on the Internet. It's also what the "HDR" option in your smartphone camera app does. As they are normal images, these images can be viewed on any monitor. The final image is still SDR (Standard Dynamic Range), and only supports 100 nits of brightness.
2. HDR Video - A method to improve dynamic range by increasing colour depth (usually from 8bpc to 10bpc. 10bpc is mandatory for HDR10), and including metadata to adjust the monitor brightness and colours to push the picture detail further than the limits of a normal SDR monitor. Because of the increased colour depth and brightness detail, HDR can achieve 1000 or even 10000 nits of brightness. Images look much more vibrant and colourful. Brights can be brighter and darks can be darker. This form of HDR requires an HDR monitor to view properly. They will either display greyed out or too bright on an SDR monitor if not properly converted.

Confusing, isn't it? Half-Life 2: Lost Coast came out in 2005 with the goal being to show off HDR rendering. But yet the HDR10 standard was announced on 2015, a decade later. The buzzword is thrown everywhere. Here is one of several Reddit threads where people are understandably confused over it.

Colour depth

In terms of colour depth, an 8bpc image has 8 bits for each channel: red, green, blue. This means that each of them can range 0-255. This allows for 16,777,216 colours. A 10bpc image has 10 bits for each channel, which means they can range 0-1023. I think a table is necessary here.

This table introduces "bpp", which is bits per pixel. Obviously the more the bits, the larger the filesize of the image taken.

Having more bits of colour is mandatory for HDR video. But more bits of colour doesn't automatically make an image HDR. It helps to prevent colour banding, as the image has much more fidelity. You can see an extreme example of banding here from 8bpc to 4bpc and back:

Since that's cleared up, this blog post will be involving HDR Video detail being put into an image. A 10bpc image displaying with the capabilities of HDR10? Let's experiment.

A word of warning

This blog post is using a lot of experimental stuff, at least for 2021. If you are on a PC or laptop, you should be fine with displaying the images even on an SDR display. The browser should be smart enough to figure out and tonemap accordingly. However, I wish I could say the same about mobile devices.

Correct SDR Estimation	Incorrectly rendered

Ironically, my phone supports HDR10+ but can't display the images in this post correctly. This is shown above. So as a word of warning, if the images appear wrong down below, that tells you that they are not being displayed correctly. They should appear natural, brightly and vibrantly coloured, with no harsh colour crushing or banding. If they do not, assume something is wrong with your browser's rendering. Again, it's experimental. So I can't control that.

HDR10 via AVIF

Upon some googling, I found that HDR10 is possible on web browsers via AVIF. This is an image format that compresses images with AV1 and currently competes with HEIC. It's exactly what I need, and it's probably what you were looking for when you clicked to read this post. You can see what web browsers support AVIF here. Firefox, Chrome, and Opera support it at the time of me writing this blog post.

Step 1: Extract a 16bpc image from an HDR10 video

Let's take a video of some gameplay of Call of Duty: Vanguard in HDR10. There's two ways to capture HDR10 footage from software: GeForce Experience, and Dxtory (with some extra processing). Whatever you choose, we can extract a screenshot from the HDR10 footage as a 16bpc image bgr48le via FFmpeg:

ffmpeg -i VIDEO.mp4 -ss SECOND_TO_GO_TO -vframes 1 out.png

With that, we get a very dull image:

Screenshot from Call of Duty Vanguard 2021.11.28 - 17.14.20.01.mkv.

Do not be fooled by this. Remember what I said earlier: "But more bits of colour doesn't automatically make an image HDR." The dullness of an HDR10 image simply means it is lacking HDR10 metadata. This is the actual video feed your monitor receives along with HDR10 metadata to make it all colourful.

Step 2: Encode via avifenc

Next, we can use avifenc (AVIF Encoder) to compress the image into an AVIF file, which is what we want here. The command I'll use is:

avifenc -c aom --speed 0 --jobs 12 --depth 10 --ignore-icc --lossless --cicp 9/16/9 in.png out.avif

The key is in --ccip 9/16/9. As mentioned here:

Now, let's look at the results!

AVICEnc'd result

This isn't some kind of fancy colour correction. This is exactly how the game looked as I was recording the HDR10 video. We are just able to tell the browser to interpret the image as such and display it correctly.

Revisiting an older image

What about the screenshot from A software "solution" to recording HDR10 via Dxtory & FFmpeg? This one:

Original demo screenshot from aforementioned blog post

The only thing about this is, while it's grey'd out like an HDR10 image, it's only recorded in 8bpc bgr24le, which results in banding when it's converted to HDR10. Regardless, it looks good when converted over. Behold:

AVICEnc'd result

How about Note 10+ Video?

As an owner of a Samsung Galaxy Note 10+, that phone comes with the capability to shoot HDR10+ video. I wouldn't recommend shooting photos on a smartphone, but let's take advantage of this.

The key word though, is "video". That is the only way that I am able to access the 10-bit colour depth and automatic HDR10 grading. When shooting in raw, the images are only 8-bit depth. Anyways, we can do the same method as shown above to extract frames from HDR10+ videos caught with this camera. Honestly, it's not an efficient or practical way to take photos on a phone at all. It does make for a really fun experiment though.

Samples

Below are sample images taken with the Samsung Galaxy Note 10+. They are single frames taken from a video shot with "HDR10+ videos" setting enabled. All of them are 3840×1620. Obviously, being from a video compressed on-the-fly, they aren't going to look as good as my usual pictures on this blog taken with a Canon. Don't be fooled if these images appear too bright for your screen. They do not calibrate to a specific HDR display. So if your display's max nits are less than whatever these pictures are "mastered" at (and I have no idea what that even is), it's going to crush the bright colours. Hey, this stuff is experimental anyways.

Anyways, left and right are the same exact image. Just the left isn't with the browser being told the image is HDR10. The right is AVIF, telling the browser it's HDR10. If your browser doesn't support it, well, it won't work.

Non-HDR	HDR10 via AVIF
My front door(s) with a lot of sunlight outside.
Phở at Bida Saigon. Notice the light striking down on the table from the outside.
The HyperX QuadCast S microphone shining in a dimly lit room.
Ditto, except lighting behind the camera is off, making the room darker.
One of my cats on a dining table. She shouldn't be there...

So what now?

I'll keep this post brief. I just wanted to bring up a curious topic of mine and see how it actually works on a web page with my own pictures. It just made a lot of sense to have image support if video support existed. I hope it improves from here. I won't put actual HDR10 content on my blog other than this post unless more appealing reasons come around. I have a hobby of photography. So there may be some points in expanding my creativity a little.