Jacobs School of Engineering

Helping a Robotic Gripper Identify Objects

Using machine learning and other advanced data science techniques, engineering researchers hope their newly developed gripper will be able to identify the objects it’s manipulating, rather than just model them.

UC San Diego engineers have designed and built a robotic gripper that can pick up and manipulate objects – even fragile items such as lightbulbs – without needing to see them. The gripper’s three fingers, each wrapped in smart skin, are designed to mimic the movements the human hand performs when you reach into your pocket and feel for your keys.

This gripper, from the lab of Mechanical Engineering Professor Michael T. Tolley, is unique in that it brings together three different capabilities: it can sense objects; it can twist objects; and it can generate 3D digital models of the objects it’s interacting with. This allows the gripper to operate in low light and low visibility conditions.

But unlike a human, the gripper can’t identify the objects just by touching or holding them. With machine learning and other advanced data science techniques, that could change. By adding these techniques to data processing, researchers hope the gripper will be able to identify the objects it’s manipulating, rather than just model them.

Developing dexterous robots that “see” with their hands fits into the larger research goal of creating safe and useful robots that work with humans in real time, in the real world. These robotic systems will adapt, evolve and create their own solutions based on the people and situations they encounter. And they’ll be secure, even while collecting and handling unprecedented amounts of data. Learning to capture and analyze new and existing data streams – and learning to leverage the resulting insights in order to take actions in real time, is an example of how research teams with world-class data science expertise are poised to improve many domains of society, including medical robotics, precision medicine, renewable energy management, and more.

To get there, researchers at Jacobs School of Engineering and across the Contextual Robotics Institute are integrating the latest advances in data science and machine learning with discoveries in hardware, software, cognitive science, design, materials, security and much more.