Despite stunning advances in AI in recent years, robots remain stubbornly stupid and limited. Those found in factories and warehouses typically go through carefully choreographed routines without much ability to sense their environment or adapt on the fly. The few industrial robots that can see and grasp objects can only do a limited number of things with minimal dexterity due to a lack of general physical intelligence.
More broadly capable robots could take over a much wider range of industrial tasks, perhaps after minimal demonstrations. Robots will also need more general skills to deal with the enormous variability and messiness of human homes.
The general excitement about advances in AI has already translated into optimism about big new leaps in robotics. Elon Musk's car company, Tesla, is developing a humanoid robot called Optimus, and Musk recently suggested it would be widely available for $20,000 to $25,000 and could perform most tasks by 2040.
Previous attempts to teach robots to perform challenging tasks focused on training a single machine for a single task, because learning seemed intransferable. Recent academic work has shown that with sufficient scale and coordination, learning can be transferred between different tasks and robots. A 2023 Google project called Open X-Embodiment involved sharing robot learning between 22 different robots in 21 different research labs.
A key challenge with the strategy that Physical Intelligence is pursuing is that there is not the same scale of robot data available for training as there is for large language models in the form of text. So the company must generate its own data and devise techniques to improve learning from a more limited data set. To develop π0, the company combined so-called vision language models, which are trained on both images and text, with diffusion modeling, a technique borrowed from AI image generation, to enable a more general form of learning.
To ensure that robots can take over any robotic job someone asks of them, such learning will have to be scaled up significantly. “There's still a long way to go, but we have something that you can think of as a scaffolding that illustrates things to come,” says Levine.
