Amazing stuff!
When will the first robot play a Stradivari or Amati violin? 😊
"... To improve the way a multi-fingered robotic hand performs complex tasks, researchers from China developed a two-pronged approach using only a basic webcam and low-cost sensors. Their process is outlined in a paper published in the journal Science Robotics.
During the first stage, the robot's AI brain was pretrained by watching a large library of videos of humans performing tasks using their bare hands and gloves. This taught the robot how visual information (what a hand looks like near an object) and tactile information (when a finger touches a surface) work together. ..."
From the abstract:
"Achieving human like dexterity with anthropomorphic multi-fingered robotic hands requires precise finger coordination. However, dexterous manipulation remains highly challenging because of high-dimensional action-observation spaces, complex hand-object contact dynamics, and frequent occlusions. To address this, we drew inspiration from the human learning paradigm of observation and practice and propose a two-stage learning framework by learning visual-tactile integration representations via self-supervised learning from human demonstrations.
We trained a unified multitask policy through reinforcement learning and online imitation learning. This decoupled learning enabled the robot to acquire generalizable manipulation skills using only monocular images and simple binary tactile signals.
With the unified policy, we built a multifingered hand manipulation system that performs multiple complicated tasks with low-cost sensing.
It achieved an 85% success rate across five complex tasks and 25 objects and further generalized to three unseen tasks that share similar hand-object coordination patterns with the training tasks."
Visual-tactile pretraining and online multitask learning for humanlike manipulation dexterity (no public access)
Representative failure modes observed in both simulation and real-world experiments.
No comments:
Post a Comment