Microsoft may be developing a “depth and multispectral” camera that could be equipped with future Surface devices such as Surface Neo/Duo, according to newly disclosed technology patents. According to patent descriptions, Microsoft’s multispectral cameras can surpass Apple’s TrueDepth camera over long distances and several other ways.
The patent, titled “Object Recognition with Depth and Multispectral Cameras” (OBJECT RECOGNITION UNDER AGE AND MULTI-CAMERA), was approved and included in the second half of 2019. So this included node coincides with the surface Neo/Duo release time.
While it’s not entirely certain that the technology will be on surface Duo/Neo,media speculate that Microsoft doesn’t want to be based on a traditional 2D facial recognition camera when it launches high-end dual-screen devices. The patent describes a camera capable of outputing a deep multispectral image, and each pixel contains at least one depth and spectral value. The structure of an object is understood to produce output by means of the object recognition of the depth and multispectral image.
In the patent description, Microsoft said it would use convolutional neural networks to identify 2D visual features including face shape, contours and facial features such as nose, eyes and lips, and 3D visual features of the face, including the forehead and chin. Some layers of convolution are designed to understand and detect spectral features of the face, such as freckles or eye color.