The text anticipated several modern AI safety concerns, including the possibility of machine consciousness, self-replication, and humans losing control of their technological creations. These themes later appeared in works such as those of Isaac Asimov The inevitable conflict by Frank Herbert Dune novels (Butler may have inspired the term “Butlerian Jihad”), and the Matrix movies.
A model of Charles Babbage's Analytical Engine, a calculating machine invented in 1837 but never built during Babbage's lifetime.
Credit: THE AGOSTINI PICTURE LIBRARY via Getty Images
Butler's letter delved deeply into the taxonomy of machine evolution, discussing mechanical 'genera and sub-genera' and referring to examples such as how watches had evolved from 'cumbersome clocks of the thirteenth century' – suggesting that mechanical species, like some early vertebrates, become smaller as they become more advanced. He expanded on these ideas in his 1872 novel Erewhonwhich depicted a society that had banned most mechanical inventions. In his fictional society, citizens destroyed all the machines invented in the last 300 years.
Butler's concerns about the evolution of the machine received mixed reactions, Butler said in the foreword to the second edition of Erewhon. Some reviewers, he said, interpreted his work as an attempt to satirize Darwin's theory of evolution, although Butler denied this. In a letter to Darwin in 1865, Butler expressed his deep appreciation for The origin of specieswriting that it “profoundly fascinated” him and explaining that he had defended Darwin's theory against critics in the New Zealand press.
What makes Butler's view particularly remarkable is that he was writing in a very different technological context, when computer equipment barely existed. While Charles Babbage had proposed his theoretical Analytical Engine in 1837 – a mechanical computer using gears and levers that was never built in his lifetime – the most advanced calculators of 1863 were little more than mechanical calculators and slide rules.
Butler extrapolated the simple machines of the Industrial Revolution, where mechanical automation transformed manufacturing, but nothing like modern computers existed. The first working program-controlled computer would not appear for another seventy years, making his predictions about machine intelligence remarkably progressive.
Some things never change
The debate Butler started continues today. Two years ago, the world was grappling with what you might call the “great AI takeover scare of 2023.” OpenAI's GPT-4 had just been released and researchers were evaluating its “power-seeking behavior,” which echoed concerns about potential self-replication and autonomous decision-making.
