Good news!
"Researchers at Stanford University and Simon Fraser University recently introduced TWIST (teleoperated whole-body imitation system), a new system that allows humanoid robots to closely imitate the whole-body motions of human users in real-time, successfully completing various real-world tasks.
This system ... leverages motion capture (MoCap) data, along with reinforcement learning and imitation learning approaches. ..."
From the abstract:
"Teleoperating humanoid robots in a whole-body manner marks a fundamental step toward developing general-purpose robotic intelligence, with human motion providing an ideal interface for controlling all degrees of freedom. Yet, most current humanoid teleoperation systems fall short of enabling coordinated whole-body behavior, typically limiting themselves to isolated locomotion or manipulation tasks.
We present the Teleoperated Whole-Body Imitation System (TWIST), a system for humanoid teleoperation through whole-body motion imitation.
We first generate reference motion clips by retargeting human motion capture data to the humanoid robot.
We then develop a robust, adaptive, and responsive whole-body controller using a combination of reinforcement learning and behavior cloning (RL+BC).
Through systematic analysis, we demonstrate how incorporating privileged future motion frames and real-world motion capture (MoCap) data improves tracking accuracy.
TWIST enables real-world humanoid robots to achieve unprecedented, versatile, and coordinated whole-body motor skills--spanning whole-body manipulation, legged manipulation, locomotion, and expressive movement--using a single unified neural network controller. ..."
Whole-body teleoperation system allows robots to perform coordinated tasks with human-like dexterity
