Researchers from Cornell University and the University of Wisconsin-Madison have announced the development of a new wrist device that can continuously track the entire human hand in 3D. The device, called FingerTrak, converts many positions of the person into 3D information, including information about the location of 20 finger joints.
FingerTrak uses three to four miniature, low-resolution thermal imaging cameras to read the contours on the wrist. The device could be used to translate sign language or for virtual reality, the researchers said. It can also be used in mobile medicine, human-machine interaction, and more. By looking at the contours of the wrists, the technology can reconstruct the finger’s position with high precision in 3D, the project’s scientists said. This is the first system to reconstruct a complete hand position based on the contourofe of the wrist.
In the past, wrist-mounted cameras were too bulky and intrusive for everyday use. Despite its size, most can only rebuild some discrete gestures. The groundbreaking FingerTrak system is a lightweight bracelet that allows your hands to move freely. Instead of relying on a camera to capture the position of the finger directly, the device focuses on a combination of thermal imaging and machine learning to virtually reconstruct hand movements.
Each of the four miniature thermal imaging cameras used on the device is pea-sized and can take multiple silhouette images to form a hand profile. The system can combine these images and use deep neural networks to 3D reconstruction of virtual hands. The technology captures the position of the entire hand, even if it is holding an object in the hand.