Google’s latest upscaling technology makes CSI TV shows a reality

Tips & Techniques

Google has introduced a new upscaling technology that turns low-resolution images into detailed high-resolution ones It can start with a portrait as tiny as 64×64 upscale it to 1024×1024 while preserving all the detail.

Researchers Jonathan Ho and Chitwan Saharia shared the details of the technology in a post on Google AI Blog. The new upsampling approach relies on diffusion models, which Google first proposed back in 2015. These models learn by gradually corrupting the image until it becomes “pure noise.” Then, the algorithm learns to gradually denoise it until it produces a clean image again.

For the latest approach, the researchers combined Super-Resolution via Repeated Refinements (SR3) and a model for class-conditioned synthesis, called Cascaded Diffusion Models (CDM).

The Image Super Resolution works more or less like the diffusion model described above. After the algorithm is trained, takes as input a low-resolution image, and builds a corresponding high resolution image from pure noise.

CDM is a class-conditional diffusion model trained on ImageNet data to generate high-resolution natural images, and Google built it “as a cascade of multiple diffusion models.”

“This cascade approach involves chaining together multiple generative models over several spatial resolutions: one diffusion model that generates data at a low resolution, followed by a sequence of SR3 super-resolution diffusion models that gradually increase the resolution of the generated image to the highest resolution.”

With the latest upsampling technique, Google also introduced a new data augmentation technique called conditioning augmentation, which should improve the sample quality results of CDM even further. You can read the full blog post on Google AI Blog and learn a bit more about the whole process and its results.

A few uses of this technique come to mind. One of them is upsampling your own photos downloaded from Facebook, Instagram, or WhatsApp that compress them. It can sometimes be faster than searching for originals. On the other hand, it could be tricky because someone can steal a small resolution of your photo, upsample it and use it as they please. Like with many technologies, this one’s not good or bad per se, it all depends on who uses it.

Ultimately, it could be of help to the authorities when trying to identify criminals from low-res security camera shots. Maybe those cringeworthy “Enhance!” moments from the CSI TV series will finally become a reality.

[via DPReview]

Products You May Like

Articles You May Like

Photography News: Imminent Nikon Announcements, Sony Trade-in Deals
The New Nikon Z30 is a Vlogger-Focused 20MP APS-C Camera
This music video took a year to create with video animation and rotoscoping
The Xiaomi 12S Ultra Smartphone Will Use a Full Sony 1-Inch Sensor
‘I Went for a Dive in the Gulf of Maine and I Saw One Fish’

Leave a Reply

Your email address will not be published.