In Satya Nadella’s first autobiography, Refresh, the current Microsoft CEO admits that one of the main sources of empathy is that his first son suffered cerebral palsy and permanent disability as a result of preterm birth. That’s one of the reasons Microsoft is pushing Microsoft Adaptive technology to make life more equitable for people with disabilities.
Following the successful launch of the Xbox Adaptive gamepad, the World Intellectual Property Office (WIPO) recently announced another new microso Adaptive technology patent. Under the patent, Microsoft plans to install a kinect-like depth sensor on a helmet and convert distance and direction information into sound, using the “sensory substitution” feature to inform players of the ball’s location.
The patent highlights the term “Sensory Substitute”, which means converting certain types of human sensory signals or modal patterns into another modal concept. For example, the first mode can cover visual signals, and the second mode can include audio signals. Other modes include tactile-based signals, temperature, vestibular signals, taste, odor, and other signals that can be perceived by biological sensory organs
The description in the Microsoft patent reads:
In the general overview, the FIG.1 diagram is an example and method of feeling substitution. Among them are the first player with a visual impairment, 110, who is batting. The second player 120 (with normal vision) is pitching towards the first player 110.
To illustrate the movement of the ball between the first player 110 and the second player 120, the ball 150 is shown in the snapshot as it is shown as a dashed line to indicate the movement of the ball over time. In addition, for illustrative purposes only, the waveform 160 is displayed directly above the stroke depicted by the ball 150 or above the flight.
If the ball itself is making a sound (e.g. beep, whistle, chi, music, or any other basically continuous sound) and repeating the sound, the sound appears to move upward in the first player 110 as the frequency approaches. The waveform 160 represents a possible shift in the frequency of the sound associated with the ball 150 when the ball moves toward situ at the first player 110, as the first player 110 might hear.
Understandably, humans are often able to handle complex and rapidly changing sound patterns, even in noisy environments or backgrounds, and these changes are easy to discern.
While this idea may seem fanciful, Microsoft has demonstrated its commitment to creating inclusive technology, so in the end, we may see this special helmet finally on the market. The complete patent can be seen here.