Duke Productions: Code Change HD AI One-Second Restore Mosaic

Not enough pixels, late fixes to piece together? In the knowledge of searching for low-pixel repair map, the results of the help to the post to brush more than finished, and from PS skills, plug-in artifacts to various types of map map app tutorials to dazzling, the focus is on the effect do not know how to do. However, a recent Duke University team developed an AI-stuy-black technology, PULSE, that can solve all the low-pixel worries.

It is said to be able to enlarge the original resolution of the image by 64 times, any slag quality can be seconds into high-definition, realistic images, and even mosaic images of the face, pores, wrinkles, hair can be clearly restored.

Mosaic Seconds to HD Portrait

PULSE is a new super-resolution algorithm that scales up to 16×16 pixels of low resolution (Low ResolutionLR) to the high resolution of 1024×1024 pixels (High Resolution, or HR) in seconds, up to a maximum of 8 times in a few seconds, compared with the traditional method of exploring the sampling of photos.

Let’s start with a set of examples, the most difficult LR head shots in the mapping world to handle, after PULSE can also be seconds into high-definition, delicate images.

Duke Productions: Code Change HD AI One-Second Restore Mosaic

What’s more, PULSE can locate key features of the face to produce a similar set of details at higher resolution. Despite the mosaic, PULSE can also “imagine” facial details such as eyebrows, eyelashes, hair, face shape, etc., to form a high-definition, realistic portrait.

However, the over-imagining of the human image is just a virtual new face, in fact it does not exist. That’s why this technology can’t be used for identification. For example, the surveillance camera shot out of focus, unrecognizable pictures, can not be restored through PULSE to the real human image.

Duke Productions: Code Change HD AI One-Second Restore Mosaic

“Never before have such an ultra-high resolution image been produced, it can produce new faces that don’t exist, and it looks real,” said Cynthia Rudin, a computer scientist with a Duke University team.

At the same time, she added, the techniques used in the study could be widely used in medicine, microscopy, astronomy, and satellite imagery. In addition, the research team has published the paper to the pre-printed paper library arVix, and was included in the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2020).

“Reduced losses” goes beyond the conventional fix

For an LR image, the traditional way of matching HR resolution to the LR image and obtaining ultra-high resolution (SR) often results in poor sensitivity, unevenness, and distorted picture.

In this study, the Duke University team developed a new approach to the super-resolution algorithm PULSE, which does not traverse the LR image to slowly add details, but finds the RRs corresponding to the HR to obtain The SR image by “reducing the loss”.

Duke Productions: Code Change HD AI One-Second Restore Mosaic

Original LR (first line), PULSE output HR (middle line), HR corresponding to LR (last line)

PULSE uses the Build Adversarial Network (GAN), a training model that, as the name suggests, performs goal training through an adversarial game. Its main structures include a generator and an authenticator (Discriminator), one responsible for training the received image and output in the same set of photo training, one for receiving the output and verifying that it is realistic enough.

The following results were compared to the original figure:

Duke Productions: Code Change HD AI One-Second Restore Mosaic

In the figure, the first behavior figure, the second behavior through the “reduce the loss” to get the HR corresponding LR, and the third line through PULSE obtained HR, can be seen that although there are subtle differences with the original figure, but the reduction degree is already very high.

In order to test the advantages of PULSE in SR, the Duke University team compared four different image scaling methods with them. The study used 1,440 images from the CelebA HQ dataset to test LR facial images, especially details such as eyes, lips, and hair, at the x8, x64 scale factor.

Duke Productions: Code Change HD AI One-Second Restore Mosaic

PULSE offers a clear advantage, especially at X64 resolution, where fuzzy avatars are completely restored, especially in details such as eye lips, which are almost impossible.

In addition, for the test results, the researchers used the perceptive ultra-resolution common MOS test method, invited five scorers to score the image results 1-5, the results showed that the HR source HD image resolution score of 3.74, and PULSE reached 3.60, only 0.14 poor, can be said to have reached almost the level of real high-quality images.

Duke Productions: Code Change HD AI One-Second Restore Mosaic

However, the researchers acknowledge that PULSE is not perfect. The high-resolution images it produces are somewhat different from those of professional original images. But with the improvement of technology and tools, the technology will be a little bit perfected.

Now the team has released PULSE to the Github open source platform and harvested 569 stars. Friends who have trouble with the map can install and experience it (Github Address: https://github.com/adamian98/pulse)