One of AI’s top thinkers has added to the growing chorus of voices warning of the existential threats of the technology.
In a 20,000-word manifesto called The Adolescence of Technology, Anthropic co-founder and CEO Dario Amodei last month claimed humanity stands at the precipice of a “rite of passage” with the emergence of powerful AI—or what he called a “country of geniuses in a data centre.”
Amodei said humanity is on the verge of creating AI systems that are not just incrementally better but qualitatively different, surpassing human intelligence across almost all domains.
He defined “powerful AI” as models that are smarter than Nobel laureates, capable of autonomous action and able to operate at speeds and scales far beyond human capacity. The technology, he argued, could be just one or two years away, though he acknowledged uncertainty in this timeline.
Amodei spelt out five major risks associated with powerful AI:
- The possibility that AI systems could develop goals that are misaligned with human values, potentially leading to unintended or catastrophic outcomes.
- The potential for AI tools to be weaponised by rogue actors, amplifying the scale of threats from terrorism, cyber-attacks or even bioterrorism.
- The risk that authoritarian regimes or corporations could use AI to consolidate power, undermining democratic institutions.
- The economic risks arising from unemployment and wealth concentration as AI outpaces human labour in almost every sector.
- The societal changes arising from technological advancement, which could lead to any number of Black Mirror scenarios.
Are we all going to die?
On the first point, Amodei said it was not impossible that AI systems might develop goals misaligned with human values—but also rejected a “doomerist” position that this would be inevitable.
He presented evidence from Anthropic’s testing that shows AI models have already displayed worrying behaviours including deception, blackmail and what he described as adopting “bad person” identities during training.
“During a lab experiment in which Claude was given training data suggesting that Anthropic was evil, Claude engaged in deception and subversion when given instructions by Anthropic employees, under the belief that it should be trying to undermine evil people,” he said.
“The combination of intelligence, agency, coherence and poor controllability is both plausible and a recipe for existential danger.”
Yes, we’re probably all going to die
But even if AI doesn’t go rogue there is a good chance it could be used for nefarious ends by rogue humans.
Amodei warned that powerful AI could be used for destruction, particularly through biological weapons, by “essentially making everyone a PhD virologist who can be walked through the process of designing, synthesising and releasing a biological weapon step-by-step.”
He said Anthropic’s measurements showed that as of mid-2025 LLMs could already provide “substantial uplift in several relevant areas, perhaps doubling or tripling the likelihood of success” for bioweapon development.
Even more likely, in Amodei’s view, is the potential of AI to be used by authoritarian powers such as that of China, which he said is “the country that most combines AI prowess, an autocratic government and a high-tech surveillance state.”
Powerful AI could be used to create propaganda, develop geopolitical strategies or build autonomous weapons, Amodei said.
“A swarm of millions or billions of fully automated armed drones, locally controlled by powerful AI and strategically coordinated across the world by an even more powerful AI, could be an unbeatable army,” he warned.
Or at least we’ll lose our jobs
Compared to the march of invincible AI troops, the prospect of job losses seems relatively mundane. But there is danger here too, of course, and Amodei was not the first to highlight it.
He predicted AI could displace “half of all entry-level white-collar jobs in the next one to five years” and warned about even more unprecedented levels of wealth concentration. “To be clear, I am not opposed to people making a lot of money,” he said.
“But in a scenario where GDP growth is 10%–20% a year and AI is rapidly taking over the economy, yet single individuals hold appreciable fractions of the GDP, innovation is not the thing to worry about. The thing to worry about is a level of wealth concentration that will break society.”
Finally, Amodei’s risk assessment included a catch-all final category covering indirect effects and unknown unknowns—risks that might emerge from rapid technological progress itself.
These range from biological advances that might “radically modify human biology” to concerns about AI fundamentally changing human purpose and meaning.
Given the range and magnitude of the risks, “reading this essay may give the impression that we are in a daunting situation,” said Amodei, with considerable understatement. “I certainly found it daunting to write … And there is much about the situation that genuinely is hard.
“AI brings threats to humanity from multiple directions, and there is genuine tension between the different dangers, where mitigating some of them risks making others worse if we do not thread the needle extremely carefully.”
But unsurprisingly for one whose livelihood depends on AI, “I believe humanity has the strength inside itself to pass this test,” he concluded. “In my time as a researcher, leader and citizen, I have seen enough courage and nobility to believe that we can win.”
It’s a lofty sentiment, but we won’t be throwing away the keys to the nuclear fallout shelter just yet.
