Facebook’s plans for future AR smart glasses are still in the works, but some of them may soon be announced,media CNET reported. Space audio and augmented audio reality are two areas of active interest to Facebook, and the company’s latest Facebook Reality Lab research update details how the company plans to build devices that can bring up the real world and make virtual objects sound like they’re right next to users, even if they don’t really exist.
Two of Facebook’s stand-alone technologies, Audio Presence and Enhanced Hearing, are part of the audio puzzle the company is trying to crack. At the same time, the larger augmented reality space remains an open question, with companies including Apple, Google, Microsoft and Magic Leap aiming to drive innovation over the next decade.
Facebook had planned to present it at its upcoming Facebook Connect conference later this month. But because of the social distance problems caused by the new crown pandemic, reporters were given a virtual introduction in a chat with Michael Abrash, head of research at Facebook’s Reality Labs, and an audio research team.
Smart Audio AR is new: Bose tried it for years before its AR audio division was apparently shut down earlier this year. Apple’s AirPods Pro features active noise reduction and pass-through capabilities and will add 3D space audio support this fall. Space audio is already a key feature of real-world VR, but Facebook’s next wave of plans is more focused on dialogue, attention and audio fidelity, including placing invisible speakers in 3D in some places to make them feel like they’re in the same room. “The great thing about me is that everything (about Facebook’s new audio technology) is absolutely feasible in a reasonable amount of time,” Abrash said. “
According to Abbash, the audio technology could eventually appear on VR helmets such as Oculus Quest and even on other Facebook devices.
Facebook’s ongoing experiments with using spatial audio involve using a silence chamber and a robotic arm with an array of speakers to surround a person and calculate specific spatial audio geometry for their ears, which takes 30 minutes, so it’s hard to carry. The next step would be to narrow the process down to take a picture of the ear to get the measurement (Facebook calls it head-related transfer, or HRTF). Facebook has long used to test the settings of measured audio, more like glasses with audio playback and focused microphones, without visual elements.
The audio team at Facebook’s Reality Lab Research also wants to make this audio technology an accessive technology that listens to and enhances conversations and improves speech clarity while filtering background noise or other people’s speech.
Some of Facebook’s concepts of 3D audio and audio sensing capabilities remind CNET reporter Scott Stein of the various ambient noise filters he tried years ago at Doppler Labs, on its Here One earbuds. Facebook’s study used beamforming to focus audio, as well as space-aware prototypes worn by testers with directional microphones. The company has created a custom-made in-ear display to create the audio, with the aim of eventually building the technology into future products.
Facebook hasn’t explicitly explained the process yet, but a blog post thursday promised that the future audio could be used to navigate AR-optimized Live Maps, which Facebook is currently assembling to map the world. Abrash promises that the company’s advances in space audio have produced realistic 3D audio that looks like it’s from the real world rather than headphones, but the current research involves complex ear scans and a specific, spatially mapped room. The company is working on an algorithm that could make the technology more accessible by using simple camera ear scans at home.