Carnegie Mellon University researchers teach robots to pick up transparent or reflective objects.

Carnegie Mellon University researchers have been able to teach robots to pick up transparent or reflective objects. This has been a challenge for robots in the past, and researchers have solved this problem by teaching robots to infer shapes from color images. Picking up transparent and reflective objects has long hindered the development of robots, but the new system is expected to alleviate the problem.

The coolest thing about the new technology the team developed is that it doesn’t require complex sensors, lots of training or human guidance. Instead, it relies heavily on color cameras embedded in the robot’s arm. Researcher David Held says the infrared light was previously exposed to an object by a depth camera to determine its shape, a technique that is effective for opaque objects. The challenge for transparent and reflective objects is that light either passes directly through or falls across the surface, making it impossible for the depth camera to calculate the exact shape.

However, the color camera can see transparent and reflective objects, as well as opaque objects. The researchers developed a color camera system that recognizes shapes based on color. Using this technique, researchers were able to train the system to mimic a depth system and implicitly infer shapes to grasp objects. The team paired the image with a color image of the same object using a depth camera image for an opaque object. After training, the color camera system is applied to transparent shiny objects. Based on these images, combined with the information provided by the depth camera, the system can successfully capture challenging objects.

The researchers found that sometimes the arm makes mistakes, but it performs better than any other system in grabbing transparent or reflective objects. The system is still more effective than transparent or reflective objects when grabbing opaque objects. The system is also capable of grabbing cluttered objects.

Carnegie Mellon University researchers teach robots to pick up transparent or reflective objects.