How Samsung uses AI to improve photos taken on the Galaxy S21
Read in other languages:
In our reviews, we made clear that the Galaxy S21 Ultra and the S21+ are capable of capturing excellent photos. While the camera sensors do play a part in this, modern photography is now also a lot about AI and image processing and that is exactly what Samsung reveals in a blog post that details how the AI-algorithm works on the new Samsung flagships.
- Samsung reveals specific details about the AI post-processing of its photos
- It involves the camera detecting faces and separating them from their surroundings before editing them
- Blurred backgrounds in portraits are also more accurate and feature a sharpening gradient
It's always exciting to see what technology smartphone manufacturers use to improve the images on their latest smartphones. Samsung, in its blog post, details the magic it uses to improve portraits captured on its phones in fair detail.
First of all, it's all about making faces look natural and a little bit out of the subject at the same time. To do this, the software detects the two faces of the subjects and separates them from the rest of the image for post-processing. Samsung then uses "natural enhancements" to tweak things like the eyes, hair, face shape, and white balance to create natural faces. Here, Samsung basically takes the same approach as it does with manual digital photo editing, which also uses masks to tweak only certain areas of the image.
Precise portrait mode with sharpness gradient
Masks are also about blurring backgrounds. It's been a trend in camera smartphones for a few years now to mimic the depth of field of portrait lenses with a small open aperture, and in fact it worked very well in my review of the Galaxy S21+. For portraits, Samsung takes advantage of the depth information gained from the Time-of-Flight sensor on the S21 Ultra and the additional and slightly differently positioned cameras on the S21+.
As with the faces in the first example, Samsung's camera software creates a mask and then optimizes the transition between subject and background. By using a tri-mask consisting of background, subject and the area of transition, Samsung can also identify hair with greater accuracy and omit it when blurring.
At the same time, the software segments subjects if they are at different distances from the camera. This explains the quite natural-looking sharpness gradient you can see in my test photo of the Galaxy S21.
Source: Samsung
