Simple AI has recently introduced HiFi-UMI, a revolutionary robot-free data-production system meticulously designed for robot manipulation learning. This innovative system, equipped with three policy backbones, has shown remarkable success rates comparable to those achieved with traditional in-domain real-robot teleoperation data.
Published in the Tech Report by Simple AI, HiFi-UMI aims to enhance robot manipulation learning efficiency through the release of HiFi-UMI-2K, a comprehensive 2,000-hour dataset licensed under Creative Commons Attribution 4.0. The Tech Report and dataset are available on arXiv and Hugging Face platforms, respectively.
The advancement in robot manipulation learning hinges on the availability of quality data. While real-robot teleoperation provides accurate trainable trajectories, its scalability is limited due to the dependency on the physical robot, teleoperation setup, and skilled operators. In contrast, robot-free handheld demonstrations offer a scalable and cost-effective alternative. However, the challenge lies in post-training policies for real-robot deployment, which traditionally rely on a small fraction of real-robot teleoperation data.
The key focus of the report is to explore whether enhancing the fidelity of robot-free demonstration data can eliminate the need for real-robot data during post-training. HiFi-UMI boasts four essential fidelity features, including precise pose accuracy, synchronized cross-sensor timing, native measurement of inter-gripper poses, and wide-angle per-hand sensing. Each captured demonstration undergoes meticulous trajectory reconstruction and simulation replay validation to ensure accuracy.
Evaluation across three policy backbones and four bimanual tabletop tasks revealed that policies post-trained exclusively on HiFi-UMI data exhibited success rates on par with those post-trained on in-domain real-robot teleoperation data. The report demonstrates the potential of fidelity-enhanced robot-free data for deployment-oriented training in specific tasks and models.
By open-sourcing HiFi-UMI-2K, Simple AI aims to provide the research community with a shared, high-fidelity dataset for further exploration. The report emphasizes that these findings are specific to the tested models, tasks, and experimental conditions, highlighting approximate parity between HiFi-UMI and teleoperation data.
Simple AI encourages collaborations with research groups and industry partners interested in leveraging high-fidelity data for robot learning. HiFi-UMI represents a significant component of Simple AI’s efforts in developing general-purpose embodied intelligence systems for human living spaces.
For more information, access the Tech Report, dataset, and project page via the provided links. Contact Grant Xin at Simple AI for media inquiries and collaborations.
About the Author
