On February 24, 2019, the day before the official opening of MWC 2019 in Barcelona, Spain, Microsoft launched its next-generation product, HoloLens, a global offering of its augmented reality device, HoloLens 2, a pro-Microsoft CEO, Satya Nadella. The importance of this product to Microsoft can be seen in the launch of the platform. However, although Microsoft announced the price of the product at the time and opened the experience on the MWC booth, it has not announced when it will be available.
Alex Kipman, Microsoft Technology Fellow and Father of HoloLens
More than eight months later, the HoloLens is finally on sale.
HoloLens 2 shipping, $27388
On November 7th, Us time, Microsoft CEO Satya Nadella announced at the 2019 Fast Company Innovation Festival that HoloLens 2 was officially shipped to customers for $3,500 in the U.S. and also to France, Germany and Germany. countries such as the United Kingdom, New Zealand, Australia, etc.
Of course, the Chinese market hasn’t been left behind; Microsoft also announced on October 8th that HoloLens 2 has been officially shipped to the Mainland China, with equipment priced at RMB27,388 (approximately US$3925) and HoloLens 2 offering two-year quality to users in mainland China Guaranteed commitment.
Compared with the first-generation HoloLens sales model, the new generation HoloLens 2 will be launched in mainland China will be based on the original Microsoft China official mall online model, the addition of offline sales model. HoloLens 2 offline sales will be divided into two-tier sales models, including china’s total distributors and China-regional resellers. In addition, HoloLens 2’s global business partner, DMP, is Unity Beauty Software (Shanghai) Co., Ltd.
Clearly, microsoft’s position on the HoloLens 2 is no longer a consumer product in terms of sales model.
Compared to the holoLens first generation, the HoloLen 2 is undoubtedly a big improvement. In terms of visual experience, HoloLens 2 is more immersive. Microsoft has created a new visual display system for HoloLens 2 that updates the way users interact with holograms, combining built-in AI and semantic understanding with a new TOF depth sensor that allows users to directly manipulate the hologram with the same gestures they use to interact with real-world objects.
Not only that, but the feel-good experience for users of the HoloLens 2 has changed. Designs such as lightweight carbon fiber materials, new thermal guide technology, and a new latch-on fit system completely upgrade the HoloLens 2 wearing experience. In addition, it is worth noting that the user does not need to remove the glasses to use the HoloLens 2, as it can be adjusted by sliding over the glasses.
At the same time, Microsoft has a lot of work on the HoloLens 2 in terms of openness, customization and so on. In addition to Microsoft’s own HoloLens 2, Microsoft’s Hybrid Reality Partner Ecosystem offers a wide range of HoloLens-based products that provide value for a wide range of industries and uses, such as healthcare, construction, and industrial manufacturing. For example, in a particular industry, Holoens 2 can be combined with the worker’s hard hat to make the application needs of the more realistic scenario stoain.
As a result, Microsoft announced the launch of the Microsoft HoloLens Custom Project to help customers and partners customize HoloLens 2 to meet the needs of the environment. For example, Microsoft has partnered with Trimble to launch a HoloLens 2-based Trimble XR 10, a new headset security device that ensures that workers access holographic information at their work site in a secure and controlled environment.
Together with HoloLens 2, Microsoft has also launched a new hybrid reality app, Dynamics 365 Guides, that empowers employees to learn through practice, and is the third Dynamics 3 that can be used simultaneously with the previous generation of HoloLens and the latest HoloLens 2 devices. 65 apps (the previous two were Dynamics 365 Remote Assist and Dynamics 365 Layout).
In addition, the combination of HoloLens 2 and Azure is a bright spot. Based on Azure, Microsoft has also launched two Azure Hybrid Reality services, Azure Spatial Anchors and Azure Remote Rendering, designed to help each developer and every enterprise build a cross-platform, Enterprise-class mixed reality applications for different industry backgrounds.
Marc Pollefeys, director of Microsoft’s Mixed Reality and Artificial Intelligence Zurich lab, says a key advantage of holographic computing for smart cloud power is that it can share information with other people with HoloLens or similar devices. As devices that support mixed reality capabilities, including HoloLens and properly configured phones, tablets, and laptops, the graphic environment information is consolidated and shared in the cloud, these views will become more detailed, more accurate, and powerful over time.
Julia White, Microsoft’s global vice president, agrees that Azure includes a pre-built service that can be used on HoloLens and any other mixed reality device, including smartphones and tablets running iOS and Android operating systems, to write applications for such experiences. She said
This collaborative experience is not limited to HoloLens. And the cost, operational difficulty, and skill requirements of developers when writing these amazing applications are greatly reduced. ”
For example, cross-device and platform features can optimize the game experience for Minecraft: Earth. Combining popular video games with mixed reality, one player can build and place a durable virtual structure in the real world so that other players can interact with it on their devices.
In response, Julia White explains, “We’re all involved because it’s cloud-based and can be interpreted and interpreted by all types of devices.” ”
Thus, combined with these two mixed reality services, HoloLens collaborates with the Azure Smart Cloud with its role in smart ends, helping Microsoft implement its “Smart Cloud and Smart Side” strategy. According to Microsoft, its hybrid reality partners in related areas are joining an ecosystem of HoloLens 2 and Azure to gain the space, voice, and visual intelligence needed for mixed reality, as well as reliable cloud services for storage, security, and application insights.
AI elements in HoloLens 2
Even as an augmented reality device, HoloLens 2 is fully integrated with AI at a technical level.
In fact, when people first wear the new HoloLens 2, the device automatically begins to recognize them: it measures everything from the precise shape of the user’s hand, the precise pitch – a feature implemented by the HoloLens science team in Cambridge, UK, and includes a senior scientist. Jamie Shotton.
To achieve natural interaction with HoloLens 2, Jamie Shotton and his colleagues developed and trained artificial intelligence models and deployed them on devices to track people’s hand movements and lines of sight, such as the way people can perceive holograms floating in front of their eyes and reach out to resize or reposition them.
To build a hand tracking system, the team created a dome camera device with an inward direction to record people’s various hand shapes. The team then uses offline cloud processing technology to build 3D models that represent everyone’s hand shapes and actions. With this 3D model, the team was able to use computer graphics to render realistic composite hand images and composite labels, making the model suitable for a variety of hand shapes, postures, and movements. According to Jamie Shotton, you can effectively generate an infinite amount of training data. ”
Using this data, the team trained a compact deep neural network, an artificial intelligence algorithm for HoloLens’s on-board processorthat that efficiently processes every frame of the depth sensor from the device. When a new user wears the HoloLens 2, the neural network is used to build a personalized 3D model that fits the size of the user’s palm, enabling accurate tracking of the natural interaction swish for holograms.
At the same time, the team used a similar approach to build and train eye tracking models, paying close attention to the pitch spacing – that is, the distance between the center spacing of the eye pupil, which varies from person to person and affects how a person looks close to or far away. The final eye-tracking model allows HoloLens 2 to accurately display holograms in front of customers, allowing them to interact and manipulate holoLens with their hands and eyes. As Jamie Shotton says:
Eye tracking, hologram and real-world matching, human interaction, all of which, if not precision, are not at all instinctive and natural interactions.
Of course, AI’s support for holoLens 2 is also reflected in a more central hardware chip level – simply put, it has a second-generation holographic processing unit, HPU 2.0.
With HPU 2.0 plus, HoloLens 2 can be used to implement SLAM (Synchronous Positioning and Map Building), which enables people to manipulate and interact with holograms without worrying about so-called delays. Typically, it takes hundreds of milliseconds to transfer data to the cloud, process, and return to the edge. Jamie Shotton says:
When you press a button on a hologram, or use your eye ball to browse text on a hologram, even a few tens of milliseconds can produce significant perception differences, and running time is critical.
In addition, privacy protection is another reason for local AI computation on devices, such as the iris scanning information used by HoloLens 2 to verify the identity of customers, which people may not want to send to the cloud. For many other types of data, however, sending it to the cloud is beneficial: once the data reaches the cloud, customers can leverage Azure AI and hybrid reality services to combine the data on their devices with data from the entire universal computing structure.
New positioning for HoloLens 2
Whether it’s price or sales, the HoloLens 2 represents Microsoft’s new positioning for the product. So how is this positioning established?
In response, Lei Feng interviewed Julia White, Microsoft’s global vice president, during The Microsoft Build 2019 Conference.
When the first generation of HoloLens was born, it was very advanced in product form, in fact, even Microsoft itself did not fully understand where it can be applied in the future. But after years of observation, we quickly realized that it was clear that it was an enterprise-class device, not a gaming product, and that it was too expensive, after all, for thousands of dollars, but if a manufacturer could increase its performance by 1% through HoloLens, it would even be willing to buy thousands of units. HoloLens. In the enterprise-class application case, HoloLens is actually a hybrid of Mixed Reality with IoT and AI, creating the applications that businesses are looking for.
Julia White also made it clear in the interview that she doesn’t think the HoloLens itself will be a gaming device, but that other handheld devices, such as smartphones, will definitely move toward MR games, as cloud services have made it possible to build MR apps. Users can also use their phones to play MR games.
Of course, Julia White is more focused on the link between The HoloLens 2 and other devices, she says:
As a technology category, Mixed Reality has a huge imagination space, not just about HoloLens, it’s also on the user’s phone, and most users have a phone; Today, with MR services in Microsoft Azure (e.g. Azure Spatial Anchors and Azure Remote Rendering), Dynamics 365 MR Apps (Dynamics 365 Remote Assi st and Dynamics 365 Layout and Dynamics 365 Guides), users can easily and cost-effectively build MR applications for any device – before that, building MR was very difficult, and the developers themselves had to be very hands-on and understand a lot (e.g. games technology, holograms, system technology), but now you only need to subscribe to Azure cloud services to create great apps. As a result, Mixed Reality will become more popular with the help of cloud services, which is why Unity and Real Engines are involved.
As for the HoloLens 2, there are already business people who are optimistic. CTO Li Zezhou, a Chinese company that offers AI smart marketing and logistics solutions for consumer goods and retail companies, sees technology, saying that the many possibilities that HoloLens presents in the work scenario are impressive, he says:
Our company itself is to do product identification, the future in the field of commodity identification, how to use mixed reality to help brands of the industry staff in the store wearing HoloLens directly to find a shop, we are trying; How the shelves are better placed, and whether the shelves are repaired, etc., these HoloLens applications are very avant-garde scenes… This will allow our company or other companies in the field of product identification to build a lot of confidence, let us believe that with more updated technology to help the industry to do a better overall store audit.
So, the final question is, when HoloLens 2 is on the market for $3,500 and RMB27388, how much can it win the approval and favor of the enterprise user community?