HEIF and HEVC: the future of multimedia files explained

Jump to:

• What are HEIF and HEVC?
• Where can we find these formats?
• Why use HEIF or HEVC?

What are HEIF and HEVC?

First of all, we need to distinguish the two terms:

• HEIF: is a container image file format that incorporates other important data.
• HEVC: This is a high efficiency compression standard usually used for video and also called H.265.

The first term refers to a digital container in which photos and other data with different extensions can be stored, such as JPG, but not only. The second term refers to a video encoding system whose result can be stored in different digital containers such as, for example, MOV or even HEIF, as mentioned above. Confused? Let us make it clear.

HEIF

Acronym of High Efficiency Image File Format, HEIF is a format used to contain single or multiple images. Files using this format can host at the same time high quality images compressed with the HEVC codec (which we will talk about later) and further information related to these images also of a different type.

HEIF can efficiently contain some image transformations such as for example:

• Rotation of the image by 90, 180, or 270 degrees.
• Crop the image given a certain rectangle.
• Overlapping any number of incoming images in the order and positions indicated on the source image's canvas window.

These operations are saved inside the container as instructions for the imager to perform. This means that the original image is not changed (it preserves the shot, it is not overwritten with changes) or compressed again at the end of the operation, thus avoiding loss of quality.

• The best Android apps for photo editing

The HEIF introduces also a framework for other non-destructive editing operations that can be clarified by external specifications. Once these operations are applied, the result is known as a derived image.

The following data types can be found in this particular HEIF container:

• Images: Individual with properties and previews.
• Derivative images: This type of non-destructive editing allows the image viewing software to apply a set of instructions "on the fly" without destroying the source image as described above.
• Image Sequences: multiple images that share certain properties such as close shooting time, their properties, and their thumbnails. This format is used mainly for serial exposures or cinemagraph animations.
• Auxiliary image objects: This data is not directly visible when viewing the file and can represent alpha planes or depth maps.
• Metadata: Additional information such as EXIF, XMP or similar data.

Some possible examples of the use of HEIF containers:

• A high quality image file and a preview of the same in JPG format of lower quality.
• Multiple image files per burst of photos so that they are all stored in the same package.
• Multiple image files with different focal lengths.
• Multiple image files with different exposures.
• An image file and a video captured at the same time (read Motion Photos by Google Pixel 2 and Live Photos by iOS).

HEVC

Also known as H.265, it is the successor of the now universally used AVC/H.264. It is a high efficiency compression standard that can support resolutions up to 8192x4320 and 8K UHD. HEVC has been designed with the aim of reducing the bitrate in video while maintaining the same quality at the cost of greater computational complexity. This codec is also used to compress images inside the HEIF containers described above.

As already happens in the H.264 codec, HEVC controls the frames of the image stream in search of areas where they are redundant both within the same image and within consecutive frames. These redundant areas are then replaced by a description of the area itself instead of the individual pixels in order to occupy less space in the memory.

Key changes for HEVC include expanding the areas of model comparison and encoding differences from 16×16 pixels to 64×64 pixels, improving variable size segmentation, better intra-forecast within the same image, better motion vector prediction and motion region fusion, better motion compensation filtering, and a further filtration step called adaptive sample offset filtering. 

For a more in-depth analysis of the topic and the method of compression and encoding, please refer to this link.

The first taste of the HEVC codec comes from 2012 when Qualcomm, during the MWC, showed a tablet equipped with SoC Snapdragon S4 dual core able to play the same video encoded in H.264 and H.265 side by side. The video in HEVC already showed 50% bit rate reduction at the same quality.

2014 saw the first introduction of the codec on consumer mobile devices thanks to Apple that used the standard for FaceTime video calls over data network via iPhone 6 and iPhone 6 Plus.

The first mobile SoC with specific hardware dedicated to HEVC decoding arrived in 2015 from Nvidia who used a decoder on its Tegra X1 similar to the one inserted a few months earlier in the GeForce 900 series video cards.

Where do we find these formats?

HEIF and HEVC files can be found on some of the most modern devices. For example, Apple has introduced support for this type of file on iOS 11, so you can capture photos and record videos in high efficiency formats on iPhone 7 and later (playback is supported up to iPhone 5S but with limitations). Of course, some iPads with iOS 11 and Macs updated to MacOS High Sierra can also play these files.

Apple uses the .heic (.heics for image sequences) extension for images, but supports playback of .heifs (.heifs for image sequences) files produced by third parties. Recorded videos are saved by Apple devices in .mov format like normal videos but are still encoded using the HEVC/H.265 codec.

In the Android world, official support for these formats has arrived at different times: HEVC is supported since Android Lollipop while HEIF was introduced only recently with Android P (available in a first Developer Preview) even if the Google Pixel Camera application of first or second generation still does not have the option related to their use.

Samsung allows you to record HEVC encoded video files on the new Galaxy S9 and S9+ even if it does not have the HEIF format for saving photos. It seems that the Xiaomi Mi MIX 2S will also have HEVC codecs for videos.

HEIF containers can also be used on web pages according to HTML 5.2 standard in order to store different versions of the same photo and leave to the browser the task of deciding which to upload and use according to different situations.

Why use HEIF and HEVC?

The main reason we should encourage ourselves to use these new image and video file formats is because they are more efficient. As we know, on smartphones the available space is never enough and thanks to these formats it is possible to store the same number of files at the same quality but in less space or the same number of files in higher quality but occupying the same amount of memory.

According to the HEIF standard, it is possible to save twice as much information in these files as in a JPG file of the same size and even higher quality. To say that the result is impressive is understatement.

In addition, HEIF/HEIC files (and all derivatives) may contain information about source image transformations without altering the original. This allows you to edit your photos on a device, save the result and can undo or make further changes later and on a different device without affecting the quality of the original image.

For example, on a JPG file it is not possible to reverse the editing process later and the image quality deteriorates with every saving due to the continuous switching through the JPG compression algorithm.

The same efficiency is true for videos encoded with HEVC (H.265): this video codec is able to offer twice the compression at the same quality level as an AVC encoded file (H.264) or a substantial increase in quality with the same bit rate. The practical use of the improvements introduced requires more power during encoding and compression but much less computing power during decompression and playback.

You'll understand that every megabyte is vital on smartphones that have 32GB of internal memory or less and these new formats promise to take many more photos and record many more videos before running out of space.

Do you already use HEVC or HEIF? What do you think about these formats?