Meta, the company behind Facebook and Instagram, has made significant advancements in the development of AI robots capable of performing “challenging sensorimotor skills.” In a press release on Friday, the company revealed that it had created a way for robots to learn from real-world human interactions, using a dataset of thousands of hours of wearable camera video. This dataset, called Ego4D, shows people performing everyday tasks such as cooking, sports, cleaning, and crafts.
To train the robots, the company has developed an artificial visual cortex, which mimics the region of the brain responsible for converting vision into movement in humans. The robots are taught using CortexBench, a set of 17 different sensorimotor tasks in simulation, including locomotion, navigation, and manipulation.
In a demonstration of its capabilities, Meta’s FAIR team used adaptive (sensorimotor) skill coordination (ASC) on a Boston Dynamics‘ Spot robot to rearrange objects in a 185-square-meter apartment and a 65-square-meter university lab. The robot was able to use its learned notion of what houses look like to identify and move out-of-place objects to the correct location. Meta claims that ASC achieved near-perfect performance, succeeding on 59 of 60 episodes and overcoming obstacles such as hardware instabilities, picking failures, and moving obstacles.
Link to the paper: Adaptive Skill Coordination (ASC)
Link to the paper: Visual Cortex
Access to the Visual Cortex code
These developments represent a significant step forward in the development of AI robots capable of performing complex tasks in the real world. The ability to learn from human interactions and perform challenging sensorimotor skills opens up new possibilities for the use of robots in a range of fields, from manufacturing and construction to healthcare and service industries. As Meta continues to refine its technology, we can expect to see even more impressive demonstrations of the capabilities of AI robots in the future.
