Robots use Cornell's RHyME AI to learn new skills by watching just one video

RHyME AI helps robots learn tasks with 50% more success using one video.

: Researchers at Cornell University have developed the RHyME AI framework that allows robots to learn new skills by watching a single demonstration video, marking a significant departure from traditional, time-intensive training methods. Known as Retrieval for Hybrid Imitation under Mismatched Execution, this approach overcomes the rigid instructions required by prior robot learning methodologies by letting robots tap into a memory bank of past actions to construct new task strategies. This method requires only 30 minutes of training data, enabling robots to complete tasks over 50% more successfully in lab tests when compared to conventional methods. Notable contributors to the project include doctoral student Kushal Kedia and assistant professor Sanjiban Choudhury, with support from prominent organizations such as Google and OpenAI.

In a breakthrough for robotics and artificial intelligence, researchers at Cornell University have developed a new AI framework known as RHyME, which stands for Retrieval for Hybrid Imitation under Mismatched Execution. This cutting-edge system allows robots to learn new tasks by watching a single demonstration video, representing a remarkable shift from the lengthy, detailed training processes robots have historically required. Usually, this training involves meticulous step-by-step instruction, occupying countless hours. The new methodology enables robots to rapidly develop the capacity to handle unforeseen obstacles or changes in task execution, a significant leap in robotics utility, particularly in unpredictable environments.

RHyME AI addresses a core challenge in robotics: translating human cues and actions into robotic execution. Unlike humans, who naturally adapt their motions to changing circumstances, robots historically required exact, matching instructions to achieve desired outcomes. Previous methods falter as minor deviations in task execution between humans and robots often led to failures. RHyME tackles this by accessing previously observed actions from a memory bank, allowing the robot to search for familiar aspects within these stored experiences. For instance, upon receiving a demonstration such as placing a mug in a sink, a robot analyzes past actions like holding a cup or setting an item down, forming a holistic strategy to perform unfamiliar tasks.

This AI-driven approach not only introduces flexibility into robotic learning systems but also drastically enhances their effectiveness. Traditional training methods often require thousands of hours of task-specific data. However, with RHyME, the requisite training time is reduced to approximately 30 minutes. In empirical studies, robots utilizing RHyME demonstrated a more than 50 percent increase in task completion success compared to counterparts trained via conventional methodologies. This revelation suggests broader applications for RHyME, potentially revolutionizing industries reliant on robotics by fast-tracking robot training processes.

The team spearheading this research includes doctoral student Kushal Kedia and assistant professor Sanjiban Choudhury, with valuable contributions from collaborators Prithwish Dan, Angela Chao, and Maximus Pace. The study's significant findings and implications will be presented at the forthcoming IEEE International Conference on Robotics and Automation in Atlanta. The project has garnered support from industry giants such as Google and OpenAI and receives funding from prestigious institutions including the US Office of Naval Research and the National Science Foundation.

As artificial intelligence continues to evolve, frameworks like RHyME highlight the potential for AI to bridge the gap between human and robotic capabilities, fostering advancements enabling robots to autonomously learn and adapt to diverse tasks and environments. This promising trajectory positions RHyME not only as a milestone in the field of robotics but as a precursor to future innovations that could further broaden the scope of robotic applications across various domains.

Sources: TechSpot, Cornell University

roboticsAutomation