Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed an innovative Real-to-Sim-to-Real model to enhance robotic learning in diverse real-world conditions. This model, named RialTo, is designed to train robots to perform everyday tasks efficiently in specific environments.

While the goal of many developers is to create robots that can operate universally under all conditions, MIT CSAIL’s team focused on making robots adept at functioning in particular settings. The RialTo method improves robot policies by 67% compared to traditional imitation learning, even with the same number of demonstrations. This approach allows robots to handle tasks such as opening toasters, placing books on shelves, and opening drawers and cabinets.

Marcel Torne Villasevil, a research assistant at MIT CSAIL and lead author of a new paper on the work, emphasized the importance of robots performing exceptionally well under disturbances, distractions, varying lighting conditions, and changes in object poses within a single environment. “We propose a method to create digital twins on the fly using the latest advances in computer vision,” Torne explained. “With just their phones, anyone can capture a digital replica of the real world, and the robots can train in a simulated environment much faster than the real world, thanks to GPU parallelization.”

RialTo: Building Policies from Reconstructed Scenes

RialTo’s process involves scanning the chosen environment using tools like NeRFStudio, ARCode, or Polycam. Once the scene is reconstructed, users upload it to RialTo’s interface for detailed adjustments and adding necessary joints to the robots. The redefined scene is then exported to a simulator, where real-world demonstrations are replicated, providing valuable data for reinforcement learning (RL).

“This helps in creating a strong policy that works well in both the simulation and the real world,” said Torne. An enhanced algorithm using reinforcement learning guides this process to ensure the policy’s effectiveness when applied outside the simulator.

Testing and Performance

MIT CSAIL’s testing showed that RialTo produced strong policies for various tasks in both controlled lab settings and unpredictable real-world environments. The researchers tested the system’s performance under three increasing levels of difficulty: randomizing object poses, adding visual distractors, and applying physical disturbances during task executions.

Zoey Chen, a computer science Ph.D. student at the University of Washington who was not involved in the paper, noted that RialTo addresses both the safety constraints of real-world RL and the efficient data constraints for data-driven learning methods. “RialTo has the potential to significantly scale up robot learning and allows robots to adapt to complex real-world scenarios much more effectively,” Chen added.

When paired with real-world data, RialTo outperformed traditional imitation-learning methods, especially in situations with many visual distractions or physical disruptions. Despite these promising results, the system currently requires three days for full training. The team hopes to improve the underlying algorithms using foundation models to speed up this process.

Future Directions

Training in simulation has its limitations, such as sim-to-real transfer and simulating deformable objects or liquids. The MIT CSAIL team plans to enhance robustness against various disturbances while improving adaptability to new environments. “Our next endeavor is to use pre-trained models, accelerate the learning process, minimize human input, and achieve broader generalization capabilities,” said Torne.

Torne co-authored the paper with senior authors Abhishek Gupta, assistant professor at the University of Washington, and Pulkit Agrawal, assistant professor in the Department of Electrical Engineering and Computer Science (EECS) at MIT. The work was supported by the Sony Research Award, the U.S. government, Hyundai Motor Co., and assistance from the WEIRD (Washington Embodied Intelligence and Robotics Development) Lab.

By Impact Lab