Bloomberg’s Mark Gurman tells the story behind the development of Apple’s AR/VR glasses. By the end of 2018, Apple is planning to build a powerful headset with virtual reality (VR) and augmented reality (AR). At the time, Jony Ive, Apple’s head of design, objected to some fundamental aspects of the product and urged Apple to change course.
The headset will be the first big new product since the Apple Watch and the first device from the Technology Development Group, a secret division that specializes in VR and AR. TDG is led by a mysterious executive, Mike Rockwell. Rockwell worked for Dolby Labs and media editing software company Avid Technology Inc., before being hired by Apple’s hardware executive, Dan Riccio, in 2015.
He started building a team at the end of 2015 and eventually developed a team of 1,000 engineers and began developing two products designed to disrupt vr and AR. A device codenamed N301 takes the benefits of VR and AR to the fullest – the first is a full-scale digital experience for gaming and consumer content, with information such as text messages and maps superimposed on viewers. Another device, the N421, is an AR-only lightweight pair of glasses, but more complex.
The N301 was originally designed to be a super-powerful system, and graphics and processing speeds are unheard of in wearables. The processing power is so powerful, but the heat is so great that it’s not possible to fit the technology into a compact helmet. The Rockwell team plans to sell a fixed base, prototyped like a small Mac that can be wirelessly connected to a headset. In an earlier version of Rockwell, the headset could also run independently, albeit with less performance.
Ive is skeptical about selling a headset that requires a separate, fixed device to achieve full functionality. He recommended that Rockwell and his team continue to develop headsets that can be fully embedded. Rockwell countered that the standoff lasted months, arguing that the wireless docking station would deliver super-powerful performance and beat all the competitors in the market.
Rockwell is highly respected at Apple and is known for his sharpness, intelligence and efficiency. He was supported by Craig Federighi’s software development team and Johny Srouji Chip Development. Apple spends more than $15 billion a year on research and development — not the first time the company has reinvested huge resources into an ambitious and potentially potentially risky project.
Just a few months before Rockwell joined Apple, Apple began developing an electric car that rivals Tesla. Apple has hired hundreds of engineers and reassigned more internal staff. But by the end of 2016, the project had begun to lay off workers, largely abandoning vehicle development in favor of the underlying self-driving technology. Within the company, the project was considered a disaster.
Rockwell’s team is still in good shape, and while it works with other parts of Apple, insiders believe it enjoys unusual independence. Located in sunnyvale, California, about a 15-minute drive from its headquarters, TDG has its own hardware, software, operations, and content teams — made up of the company’s most talented people.
As for the impasse between Rockwell and Ive, CEO Tim Cook ended up with the design director. Although the head-wearing device now under development is not as technically ambitious as originally thought, it is quite advanced. The design features an ultra-high-resolution screen that makes it almost impossible for users to distinguish between virtual and real worlds. People who have used the prototype say the movie-grade speaker system will make the experience more realistic. (The technology used in the base is not completely wasted.) Some technologies have been reused to Mac self-developed ARM processors to replace Intel processors. )
However, removing the base means that the graphics are not as powerful as they used to be, and it may also make the experience less realistic than originally hoped. For Ive, who left last year last year after nearly three decades at the company, the more realistic experience has potential problems: he doesn’t want Apple’s technology to take people out of the real world. According to people familiar with the matter, he prefers the concept of N421 glasses, which will allow users to be realistic while delivering maps and information to their vision.
The prototype of the N301 looks like a smaller Oculus Quest, mostly with a fabric body, but less plastic than Quest. Apple’s engineering team is still testing different head types to find the best fit. The price is not certain. Oculus Quest is priced at $399, while Microsoft’s company-focused Hololens 2 mixed reality helmet and Magic Leap AR goggles are priced at $3,500 and $2,295, respectively.
The N301 will have its own App Store, play games and watch live streams, as well as serve as a hyper-high-tech communications device for virtual meetings. Siri, Apple’s voice assistant, supports control of the headset, but the device is also testing the physical remote control. Apple has reassigned some engineers who are working on The Siri interface to Rockwell’s team.