Michal Kosinski is a Stanford research psychologist with a flair for current topics. He sees his work not only as advancing knowledge, but also as alerting the world to potential dangers caused by the effects of computer systems. His most famous projects involved analyzing the ways in which Facebook (now Meta) gained a shockingly deep insight into its users from all the times they clicked 'Like' on the platform. Now he's moved on to studying surprising things AI can do. For example, he has conducted experiments showing that computers can predict a person's sexuality by analyzing a digital photo of his or her face.
I got to know Kosinski through my writing on Meta, and I contacted him again to discuss his latest paper, published this week in the peer-reviewed Proceedings of the National Academy of Sciences. His conclusion is startling. Major language models like OpenAI's, he argues, have crossed a line and use techniques analogous to actual thinking, which was once considered solely the domain of flesh-and-blood humans (or at least mammals). In particular, he tested OpenAI's GPT-3.5 and GPT-4 to see if they had mastered the so-called 'Theory of Mind'. This is the ability of humans, developed in childhood, to understand the thought processes of other people. It is an important skill. If a computer system cannot correctly interpret what people are thinking, understanding of the world will be impoverished and many things will go wrong. As models Doing If they have a theory of mind, they are one step closer to matching and exceeding human capabilities. Kosinski has put LLMs to the test and now says his experiments show that in GPT-4 in particular, a theory of ghost-like abilities “may have emerged as an unintended byproduct of LLMs' improved language skills… They indicate the arrival of more powerful and socially skilled AI.”
Kosinski sees his work in AI as a natural outgrowth of his earlier dive into Facebook Likes. “I didn't really study social networks, I studied people,” he says. When OpenAI and Google started building their latest generative AI models, he says, they thought they were training them to deal primarily with language. “But they actually trained a human mind model, because you can't predict what word I'm going to say next without modeling my mind.”
Kosinski is careful not to claim that LLMs have fully mastered the theory of mind – not yet. In his experiments, he presented a few classic problems to the chatbots, some of which they were able to solve very well. But even the most advanced model, GPT-4, failed a quarter of the time. The successes, he writes, put GPT-4 on par with 6-year-old children. Not bad, considering the early state of the field. “As we observe the rapid progress of AI, many wonder if and when AI could achieve ToM or consciousness,” he writes. Besides that radioactive c-word, that's a lot to chew on.
“If the Theory of Mind emerged spontaneously in those models, this also suggests that other skills may subsequently emerge,” he tells me. “These capabilities allow them to better train, influence and manipulate us.” He worries that we aren't really prepared for LLMs who understand how people think. Especially when they reach the point where they understand people better than people.
“We humans do not simulate personality – us to have personality,” he says. “So I'm a bit stuck with my personality. These things model personality. The advantage is that they can have any personality they want at any time. When I tell Kosinski it sounds like he's describing a sociopath, he lights up. “I use that in my conversations!” he says. “A sociopath can put on a mask – they're not really sad, but they can play a sad person.” This chameleon-like power could make AI a superior con artist. Without regrets.
