OpenAI is apparently ramping up its own robotics efforts as well. Last week, Caitlin Kalinowski, who previously led the development of virtual and augmented reality headsets at Meta, announced on LinkedIn that she will be working on hardware, including robotics, at OpenAI.
Lachy Groom, a friend of OpenAI CEO Sam Altman and an investor and co-founder of Physical Intelligence, joins the team in the conference room to discuss the business side of the plan. Groom is wearing an expensive looking hoodie and looks remarkably young. He emphasizes that physical intelligence has sufficient potential to pursue a breakthrough in robot learning. “I just got off the phone with Kushner,” he says, referring to Joshua Kushner, founder and managing partner of Thrive Capital, who led the startup's seed investment round. He is of course also the brother of Donald Trump's son-in-law Jared Kushner.
A few other companies are now chasing a similar breakthrough. One called Skild, founded by roboticists at Carnegie Mellon University, raised $300 million in July. “Just as OpenAI built ChatGPT for language, we are building a general-purpose brain for robots,” said Deepak Pathak, CEO of Skild and assistant professor at CMU.
Not everyone is confident that this can be achieved in the same way that OpenAI cracked the language code of AI.
There simply is no repository on the Internet of robot actions comparable to the text and image data available for training LLMs. Achieving a breakthrough in physical intelligence may require exponentially more data.
“Words in order are, dimensionally speaking, a small toy compared to all the movement and activity of objects in the physical world,” says Illah Nourbakhsh, a roboticist at CMU who is not involved with Skild. “The degrees of freedom we have in the physical world are so much more than just the letters of the alphabet.”
Ken Goldberg, an academic at UC Berkeley who works on applying AI to robots, warns that the excitement surrounding the idea of a data-driven robot revolution and humanoids is reaching hype-like proportions. “To achieve the expected performance levels, we need 'good old-fashioned engineering', modularity, algorithms and metrics,” he says.
Russ Tedrake, a computer scientist at the Massachusetts Institute of Technology and vice president of robotics research at the Toyota Research Institute, says the success of LLMs has caused many roboticists, including himself, to rethink his research priorities and focus on finding ways to pursue robotic learning. a more ambitious scale. But he admits enormous challenges remain.
