Students and faculty researchers at NYU’s Courant Institute of Mathematical Sciences and a researcher from Meta AI recently published a new design framework for in-home robots called Dobb-E after nearly three years of working on the project.
Led by fourth-year Ph.D. student Mahi Shafiullah, the design aims to make home robotics more accessible. The robot learns from footage of household tasks such as opening drawers, pulling chairs and rotating stove knobs taken on an iPhone, and then attempts to replicate the same action shown in the videos. After 15 minutes of learning from a video, the 51-pound robot is able to initiate simple tasks like opening cabinets and moving objects with a human-like grasping mechanism attached to its body.
In an interview with WSN, Shafiullah said he has always been drawn to the relationship between humanity and technology. Shafiullah moved to the United States from Bangladesh in 2015 to attend the Massachusetts Institute of Technology for his bachelor’s degree in math and computer science, and also earned a master’s degree in engineering.
“What I really wanted to get out of it is just to inspire more people, more AI scientists, roboticists, engineers, even new computer science students — to change the paradigm of how to think,” Shafiullah said. “I believe very strongly that at the end of the day, the point of human technology is to make human lives better. If we’re not doing that, then what are we doing?”
Lerrel Pinto, an assistant professor at Courant, said the process of manufacturing home robots is comparable to how cars became more affordable once they were mass-produced in an interview with WSN. Pinto’s lab focuses on “general purpose robots” that can operate in unfamiliar environments and learn from trial and error, like humans.
“We want the community to keep building and improving our system, and the only way we can have a community of people build is if we open-source everything,” Pinto said. “Our hope is that because we’re open-sourcing everything, people are interested in using our tools, and when they’re interested in using our tools, they can scale it a lot more than us.”
Shafiullah and Pinto said that they predict home robots will be accessible to the general public within the next five to 10 years. They are working toward addressing privacy concerns by storing data locally in the device, and also said private data in people’s homes can be protected by measures like turning personal photos around so that they are not visible.
In a written statement to WSN, Matei Ciocarlie — a mechanical engineering professor at Columbia University who is not involved in the Dobb-E project — said he thinks the new framework is “a terrific effort” and that he hopes the adaptable learning Shafiullah and his team add to Dobb-E can help open up new possibilities.
“The home has long been considered such a difficult environment that many roboticists won’t even consider it,” Ciocarlie wrote. “However, Dobb-E has gotten over that hill and is performing real experiments in real homes, which is impressive. The novel learning approaches we now have at our disposal as a field might provide the impulse needed to be able to handle such variability.”
Contact Gabrielle Panelo at [email protected].
