Nvidia's AI turns doodles into photorealistic images
Nvidia has introduced a new AI that can create photorealistic landscapes from simple lines and surfaces. This is packaged in a simple program that has the charm of Microsoft Paint. You can't get anything out of your child's latest scribble? Transform this beautiful memory into something worth looking at with the help of a heartless machine!
But first of all: Nvidia's AI cannot turn every abstract scribble into a beautiful landscape. Artificial intelligence uses a certain method here. First the "artist" gets a white canvas on the computer, on which he can draw with strokes and filling tools, MS Paint style. The AI understands the lines and individual segments as instructions for the composition and transforms the strokes and areas into a photorealistic image depending on the selected colors.
The neuronal GAN network (GauGAN) developed by Nvidia was fed with millions of landscape shots from the Flickr photo platform. However, the AI does not put together individual picture sections here, but creates its own composition and thus new pictures from scratch.
In order to achieve this, the AI has extracted typical correlations from the training data. Thus, on a quiet lake, the reflection of the mountains behind it and on a snow-covered landscape, the sky appears rather grayish and the trees bear no leaves.
Fast computer an advantage
Even though the AI app is reminiscent of an even more rudimentary version of Microsoft Paint, it still needs some computing power. Nvidia uses a tensor computing unit and an RTX titanium graphics card. Although it can also be used on weaker computers, it takes much longer to calculate.
Of course, Nvidia doesn't intend to create an AI painter. In the future, GauGAN should be a relief in the development of virtual worlds and create first basic models for game worlds from designs by architects and other drawings or create training environments for self-driving cars. A publicly available version will be available soon.
Source: Nvidia
My immediate thought was to take the output of this program and feed it back into one of those filters that turns photographs into watercolor or oil paint images.