Bioengineers at the University of California, Los Angeles, have designed a glove that can be used to translate American Sign Language into English voice in real time using a smartphone app. The goal of the project is to facilitate direct communication between people who use sign language and those who do not understand sign language, without the need for a translator there.
The team also believes the glove can help more people learn sign language. The system is a pair of gloves with thin, stretchable sensors that extend upward along the length of five fingers. The sensor is made of conductive yarn and can receive hand movements and finger positions that represent single letters, numbers, words, and phrases in sign language.
The device converts finger movements into electrical signals and sends them to a dollar-sized coin board worn on the wrist. The board transmits signals wirelessly to a smartphone, which translates it into spoken language at a rate of about one word per second. The researchers also placed sensors on the faces of people who tested gloves. Sensors are placed between eyebrows and on the side of the mouth to capture facial expressions as part of American Sign Language.
Previous wearable systems have had limited effect on translating sign language because they are bulky or uncomfortable to wear, the researchers said. The lightweight wearable system created by the UCLA team is made of cheap material and can be used for a long time. Electronic sensors are also flexible and affordable.
The team said the system performed well in tests of four deaf people who used American sign language. Each participant made 15 gestures, and a customized machine learning algorithm turned the gesture into a representation of letters, numbers, and words. The system can recognize 660 gestures, including each letter and number 0 to 9. However, its business model needs to be improved to support a wider vocabulary and faster translation times.