In the process of advancing the full screen, punching pre-cameras are already a more sophisticated commercial solution, but will soon be replaced by underscreen front cameras. The underscreen front camera means that the camera is located under the transparent OLED panel, which can still function as a normal display when inactive, allowing light to pass through the front sensor when shooting.
There are many advantages to on-screen front-up, but the existing technology is not enough to commercialize it, one of the unresolved problems is that the diffraction of the screen pixel structure blurs the image, reduces contrast, reduces the available light, and even completely obscures some of the image content.
Fortunately, this effect occurs in a predictable manner, and because of the pixel structure, it usually occurs only horizontally. To this end, Microsoft’s research department after unremitting research efforts, finally found to compensate for the T-OLED screen shooting image blurry effects.
The researchers used a U-Net convolutional neural network structure that improved the signal-to-noise ratio while eliminating blurring of images. The team was able to achieve a recovery image that was almost indistinguishable from a directly shot image. The following is a comparison of the processing effect before and after.
The effect of shooting directly with the camera.
Effect shot under the T-OLED screen.
Use artificial intelligence algorithms to correct the effect.
Involve artificial intelligence in image capture and use other interesting technologies, such as blurring or replacing backgrounds, and other video manipulation techniques for better and more natural video calls. Microsoft seems to be developing this work primarily for use on the big screen in video conferencing settings, but I’m sure it will apply equally to your next flagship smartphone.