Scalable Data Collection Drives Next-Generation Home Robotics

Key Quotes

"I would say I think we're kind of in between the GPT moment and the Chat GPT moment like in the context of LLMs what it means is that it seems like we have a recipe that can be scaled but we haven't scaled up yet and we haven't scaled up so much so that we can have a great consumer product out of it so this is where I mean like GPT which is like a technology and Chat GPT which is a product."

Tony Zhao explains that the current state of AI robotics is analogous to the early stages of Large Language Models (LLMs), where the underlying technology is promising but not yet refined into a user-friendly product. This suggests that while the foundational elements for advanced robotics exist, significant development is still needed to create a mass-market consumer product.

"Previously, you know classical robotics have this sense plan act modular approach where there's a human designing the interface between each of the modules and those are need to be designed for each specific task and each specific environment. In academia that means for every task that means a paper. So a paper is you design a task, design an environment, and you design interfaces, and then you produce engineering work for that specific task. But once you move on to the next task, you throw away all your code, all your work, and you start over again."

Cheng Chi describes the limitations of traditional robotics, highlighting its task-specific and modular nature. This approach required extensive human design for each new task and environment, leading to a lack of reusability and slow overall progress in the field.

"The problem is that, in the field, it's known to be very finicky. So when it comes to researchers, when I start into the field, people are like the researcher themselves, the specific researcher needs to collect the data so that there's exactly one way to do everything, otherwise the robot either like your model training will diverge or the robot will behave some weird way. And the diffusion model really allows us to capture multiple modes of behavior for the same observation in a way that's still preserved training stability."

Tony Zhao discusses the challenges of imitation learning in robotics, specifically how traditional methods were sensitive to data collection variations. He explains that diffusion policy, a specific algorithm, overcomes this by enabling the model to learn from diverse behaviors while maintaining training stability, thus allowing for more scalable data collection.

"I realized that all this information you can get from a GoPro. You can track the movement of the GoPro in space, and you can, you know, track the motion of the gripper and also finger through images as well. And that's why I've built this UMI gripper, a 3D printed. At the time, like the project had three PhD students, we just took the grippers everywhere."

Cheng Chi details the innovation behind the UMI gripper, which leveraged readily available technology like GoPros to collect essential robotic data. This approach, developed by a small team, significantly scaled data collection by capturing hand and gripper movements from video, demonstrating a more accessible method for generating large datasets.

"The core motivation for us is how can we build a useful robot as soon as possible. So whenever we see something that we can accelerate it with simplification, we'll go simplify that. So one example of that is the hand that we designed, which has three fingers. We kind of combined the three of the fingers that we have together. And the reasoning there is just that most of the time when we use those fingers, we use it together."

Tony Zhao explains Sunday Robotics' design philosophy, emphasizing simplification to accelerate the development of a useful robot. He uses the example of their three-fingered hand, which combines fingers to reduce complexity and cost while still performing most common tasks effectively.

"The number one is really figuring out the training recipe at scale. We as a field just entered the realm of scaling and we just got amount of data that we need. And I think now is a perfect time to start do research and actually figure out what exact training recipe we need to actually, you know, get robust behaviors."

Tony Zhao identifies a key technical challenge: determining the optimal "training recipe" for large-scale robotics. He suggests that with the recent increase in available data, the field is now positioned to conduct research focused on developing robust training methodologies.