For a novice painter, an
attempt to create a sunset landscape with snow-capped mountains on the horizon
may end up looking like a multicoloured inkblot. But not if you are armed with
AI and deep learning.
In
fact, a deep learning model of Nvidia Research can do just the opposite: Turn
rough doodles into photorealistic masterpieces. The tool uses generative
adversarial networks, or GANs, to convert segmentation maps into lifelike
images. The interactive app using the model, in a lighthearted nod to a
painter, is named GauGAN.
Smart
paintbrush
GauGAN
can be a great tool for creating virtual worlds for everyone from architects
and urban planners to landscape designers and game developers. With an AI that
understands how the real world looks, professionals could better prototype
ideas and make rapid changes to a virtual scene.
The
tech behind GauGAN can be linked to a ‘smart paintbrush’ that can fill in the
details inside rough segmentation maps, the high-level outlines that show the
location of objects in a scene.
Synthesising
new images
Trained
on a million images, the deep learning model then fills in the landscape with
surprising results: Draw in a pond, and nearby elements like trees and rocks
will appear as reflections in the water. Swap a segment label from ‘grass’ to
‘snow’ and the entire image changes to a winter scene, with a formerly leafy
tree turning barren.
The
tool also allows users to add a style filter, changing an image to adapt the
style of a painter or change a daytime scene to sunset. While it focuses on
natural elements like land, sea and sky, it is capable of filling in other
landscape features, including buildings, roads and people.
