Robot that watched surgery videos performs with skill of human doctor

Meet your next surgeon!

Wow, the da Vinci Surgical System has come a long way since it was introduced in the year 2000.

"... The successful use of imitation learning to train surgical robots eliminates the need to program robots with each individual move required during a medical procedure and brings the field of robotic surgery closer to true autonomy, where robots could perform complex surgeries without human help. ...

The team, which included Stanford University researchers, used imitation learning to train the da Vinci Surgical System robot to perform three fundamental tasks required in surgical procedures: manipulating a needle, lifting body tissue, and suturing. In each case, the robot trained on the team's model performed the same surgical procedures as skillfully as human doctors. ...

The researchers fed their model hundreds of videos recorded from wrist cameras placed on the arms of da Vinci robots during surgical procedures. These videos, recorded by surgeons all over the world, are used for post-operative analysis and then archived. Nearly 7,000 da Vinci robots are used worldwide, and more than 50,000 surgeons are trained on the system, creating a large archive of data for robots to "imitate." ..."

From the abstract:

"We explore whether surgical manipulation tasks can be learned on the da Vinci robot via imitation learning. However, the da Vinci system presents unique challenges which hinder straight-forward implementation of imitation learning. Notably, its forward kinematics is inconsistent due to imprecise joint measurements, and naively training a policy using such approximate kinematics data often leads to task failure. To overcome this limitation, we introduce a relative action formulation which enables successful policy training and deployment using its approximate kinematics data. A promising outcome of this approach is that the large repository of clinical data, which contains approximate kinematics, may be directly utilized for robot learning without further corrections. We demonstrate our findings through successful execution of three fundamental surgical tasks, including tissue manipulation, needle handling, and knot-tying."

Robot that watched surgery videos performs with skill of human doctor | Hub "Breakthrough training system utilizing imitation learning opens 'new frontier' in medical robotics"

Surgical Robot Transformer (SRT): Imitation Learning for Surgical Tasks (open access)