Computational RAW on smartphones: The best of both worlds

What is RAW anyway?

The English word "raw" means "raw." And that's exactly what RAW is: the format contains raw data from the camera sensor, which has usually just passed through the analog-to-digital converter. A lot of work is required before a JPEG photo can be created from this raw data - corresponding to the often-used comparison: the RAW image corresponds to the analog negative, while the JPEG image is the fully developed photo.

The first step in turning a RAW into a JPEG in the digital photo lab is demosaicing. The image spreading algorithm makes RGB pixels out of only red, green, blue, and possibly white pixels and casts them into a color space. The software then optimizes the image with noise reduction, re-sharpening, lens correction, white balance, and other measures, saving the result as a JPEG file.

This conversion process from RAW to JPEG is not only destructive but also irreversible, i.e. information is irrevocably lost. Whether the image result really corresponds to the photographer's visual impression or the desired result is a matter of luck, and it also depends on the quality of the image-processing algorithms.

With RAW format, the photographer retains control.

Why does RAW not play a role on smartphones?

While the vast majority of photographers shoot RAW or at least RAW+JPEG on dedicated cameras, hardly any smartphone users use the RAW option that is typically buried underneath layers within the menu in Pro mode. In reality, why did they do that?

Besides making it more tedious to access, the biggest reason might be this: It simply doesn't make sense in many cases. While current smartphones often condense a dozen photos into a single image in the AI-boosted JPEG mode, the raw data of a single shot usually ends up in the memory in RAW mode. With the help of intelligent algorithms, smartphones nowadays are able to obtain far more quality out of five, ten or fifteen photos welded together than a photographer can get out of a single RAW image.

So for a long time, messing with smartphone RAW files was a simple case of diminishing returns where plenty of effort results in very little reward. Now all of that could change.

RAW makes a comeback in 2022 as Computational RAW?

Slowly but surely, "Computational RAW" is catching on. And at least in theory, the potential is huge, because Computational RAW welds together multiple photos into a single RAW shot. So the advantages of both worlds are combined in one format.

The idea is not new. The Google Pixel 3 from 2018 was already capable of Computational RAW and packs data of 15 individual pictures into a single RAW file. As an order of magnitude, in 15 frames, the 1/2.55-inch image sensor from the Google Pixel 3 (about 23.5 mm square) collects as much light as an APS-C format sensor (about 350 mm square) that is found in most mid-range DSLMs.

Apple received more attention in 2020 for the ProRAW format in the iPhone 12 Pro (Max), which also condenses an entire image stack into a single shot. Samsung followed suit in November with Expert RAW for the Galaxy S21 Ultra. Oppo announced the MariSilicon X image processor in December, which apparently also supports Computational RAW and will probably be found in the Find X4 Pro. The OnePlus 10 Pro, which was recently released in China, also features a corresponding mode in the form of RAW Plus.

There are now many smartphone apps for image editing in RAW format, both from third-party providers and from the manufacturers themselves. And the hunger for storage of RAW photos should no longer be a huge problem in 2022.

So what remains missing?

One would be the courage on the part of the manufacturer to place the feature prominently in the camera app and to offer a simple workflow for editing. Once that's in place, users will follow suit once they've experienced the flexibility of the RAW format for themselves.

What's next for computational RAW?

The next big challenge for computational photography, and therefore, computational RAW, is combining photos from different cameras. That's because sensors in smartphones can't be enlarged much further without making the bodies significantly thicker and/or hampering image processing algorithms with optical effects such as selective depth of field.

The only way to increase the light-sensitive area is to divide it among several sensors. The Nokia 9 PureView and the Light L16 have already tried this before but failed due to a lack of computing power in the handsets. The problem will be solved in the coming months and years, so beware, trypophobes!

And you like this?

What about the NextPit community? Do you already shoot in RAW format and use a "real" camera or even your smartphone? And if not: Will you give the RAW format on your smartphone a chance soon?