Researchers from Johns Hopkins University (JHU) and Stanford University have achieved a major milestone in robotic surgery: teaching a robotic system to perform complex surgical tasks as skillfully as human surgeons, simply by training it using videos of real surgical procedures. This development could significantly accelerate the path to fully autonomous robots in the operating room.
The study was conducted using the da Vinci Surgical System, a robotic platform that is already used in many surgeries today. This system, typically controlled remotely by a surgeon, features robotic arms that manipulate instruments for delicate tasks such as dissection, cutting, suction, and vessel sealing. Known for its precision, the da Vinci system gives surgeons enhanced control and a more detailed view of the surgical site, but the latest model can cost over $2 million, excluding accessories and training expenses.
In this new study, the team trained the da Vinci Surgical System to autonomously execute three common tasks in surgery: manipulating a needle, lifting body tissue, and suturing. Using a machine learning technique called imitation learning, the researchers fed the robot hundreds of surgical procedure videos recorded from wrist cameras mounted on da Vinci systems during real surgeries. The robot learned by imitating the actions of human surgeons in these videos.
The results were impressive. Not only was the robot able to perform these tasks as well as human surgeons, it also developed the ability to correct its own mistakes. “If it drops the needle, it will automatically pick it up and continue,” said Axel Krieger, an assistant professor at JHU and co-author of the study. “This isn’t something I explicitly taught it to do—it just learned it on its own.”
The researchers used a combination of imitation learning and the same machine learning architecture that powers popular AI models like OpenAI’s ChatGPT. However, instead of processing text as ChatGPT does, this AI model generates kinematic data—a specialized “language” used to describe movement through mathematical formulas and equations. This kinematic data guides the movements of the robot’s arms during surgery, enabling the system to perform delicate tasks with great precision.
Krieger believes this approach could revolutionize the way robots are trained for surgery. Traditionally, coding each surgical task by hand is a slow and labor-intensive process, with every step needing to be manually programmed. In contrast, the imitation learning method used in this study allows the robot to quickly learn from videos, drastically reducing training time. According to Krieger, “We only have to collect imitation learning data for different procedures, and we can train a robot to learn it in just a couple of days. This accelerates the goal of autonomy while reducing medical errors and improving the accuracy of surgery.”
This breakthrough could be a game-changer for the future of surgery. While there are already robotic systems, such as Corindus’s CorPath, that assist in certain steps of complex procedures, these systems are typically limited in scope. The ability to teach a robot to perform a broader range of tasks autonomously could lead to more comprehensive robotic systems that can handle entire surgeries with minimal human intervention.
Krieger points out that traditional hand-coding methods are time-consuming and inefficient. “Someone might spend a decade trying to model just the suturing step for a single type of surgery,” he explained. With imitation learning, however, robots can rapidly acquire a broader range of surgical skills without requiring years of painstaking coding.
This latest work builds on earlier advancements in robotic surgery, such as the development of the Smart Tissue Autonomous Robot (STAR) by Krieger and his team at JHU in 2022. STAR used a 3D endoscope and machine learning-based tracking to autonomously suture two ends of a pig’s intestine, demonstrating the potential for fully autonomous robots in complex surgical tasks.
The researchers at JHU are now working to extend this new imitation learning approach to teach robots how to perform entire surgeries, potentially paving the way for fully autonomous robots in the operating room. While it will likely be years before we see robots completely replacing human surgeons, this innovation represents a significant step forward in making surgery safer, more efficient, and more accessible worldwide.
With this new method, robotic surgery is no longer just a futuristic concept—it’s becoming a practical, scalable solution that could help improve patient outcomes, reduce medical errors, and ease the burden on healthcare systems.
By Impact Lab
