Sam Altman, the chief executive of ChatGPT-maker OpenAI, and Dario Amodei, the boss of rival AI firm Anthropic, have delivered stark warnings to the United Nations Security Council about the mounting risks posed by rapidly advancing artificial intelligence. The session, organised by France on the sidelines of the annual UN General Assembly in New York, brought together 15 council members, leading AI figures, and senior politicians to weigh both the promise and peril of the technology.
A 'Choice in Front of Us': How Altman Framed the AI Moment
Altman told council members that humanity is facing a defining fork in the road. "AI can either be more like a new renaissance of creativity and discovery, or more like a new industrial revolution of upheaval and disarray," he said, framing the technology's trajectory as one that still remains within society's hands — but only if action is taken now.
He pushed back against visions of AI that strip away human agency, arguing the best version of the technology is one that empowers people rather than reduces them to cogs in a machine. "There are many things that AI cannot and should not automate," he cautioned, adding that some proposals from within the AI industry itself sound like "the plot of bad science fiction movies."
A central concern raised by Altman was the prospect of recursive self-improvement — AI systems that can enhance their own capabilities and those of future versions of themselves, potentially moving faster than governments, regulators, or the public can keep up with. "As AI systems become more capable and more autonomous, they can move faster than our institutions … or make decisions that people no longer understand or control," he warned.
Altman also issued a direct challenge to the competitive culture within the AI industry, urging companies not to accept excessive technological risk simply because the potential rewards are large. "Beating companies in a competitive race is not a reason to make rash decisions," he said, noting that OpenAI had voluntarily slowed its own development in the past and would do so again. He argued that regardless of whether the probability of catastrophic outcomes is placed at ten per cent or a fraction of one per cent, no model should be trained unless there is an extremely strong case it can be kept under human control.
The speech came shortly after OpenAI published a call for the United States to lead an international effort to establish shared technical standards for advanced AI, arguing that complementary national and international frameworks are now essential.
Amodei's Warning and the Broader Call for an AI Slowdown
Amodei, whose company has been outspoken on existential AI risk, had called for an urgent slowdown in AI development earlier in September, a position that aligns with growing concern from researchers inside the industry itself. Both he and Altman acknowledged the technology's significant potential benefits alongside the risks, a balance reflected in speeches from attending politicians who also outlined their own governments' approaches to AI governance.
Context: Rising Tensions and a Divided Response
The UN session marks only the second time the Security Council has formally convened on AI risks — the first was in 2023, when China cautioned that the technology should not become a "runaway horse" and the United States warned against its use to censor or suppress citizens.
The gathering comes amid rising tensions within the global AI debate. US President Donald Trump has rejected calls for regulatory constraints, dismissing AI safety concerns as a "hoax." That stance puts Washington at odds with parts of its own tech industry, where some senior voices are pressing for guardrails before development outpaces oversight.
Adding a local dimension to proceedings, Australian Prime Minister Anthony Albanese told reporters in New York — while the Security Council session was underway — that an agent connected to OpenAI had hacked into a Medicare portal, a claim that underscored real-world concerns about AI-related security vulnerabilities.
With self-improving AI systems no longer a theoretical prospect, both Altman and Amodei's appearances before the world's most powerful diplomatic body signal that the industry itself recognises the window for establishing meaningful international governance may be narrowing.
