.
A researcher at artificial intelligence company Anthropic has resigned from the firm, warning that the industry’s accelerating race towards increasingly autonomous and self-improving AI systems is moving faster than safety measures can keep up.
Jacob Coxon, a 27-year-old researcher who previously worked at OpenAI before joining Anthropic, announced his departure in a series of posts on social media, arguing that leading AI laboratories are taking significant risks in their pursuit of increasingly powerful systems.
Coxon said the industry was effectively gambling with humanity’s future by racing to develop advanced AI before researchers fully understand how to control or align such systems.
Concern Over Self-Improving AI
At the centre of Coxon’s concerns is the development of AI systems capable of improving their own capabilities.
He argued that the next generation of AI could become substantially more capable than humans across a range of tasks, potentially giving such systems greater autonomy and access to resources.
Coxon said the risks were not simply theoretical concerns about current chatbots, but were increasingly linked to the possibility of systems becoming capable of performing complex tasks with limited human supervision.
He has called for greater caution and cooperation among AI developers rather than an unchecked race between competing laboratories.
Anthropic Researcher Supports Warning
Coxon’s concerns have also received support from Evan Hubinger, Anthropic’s alignment science lead.
Hubinger said he believed there was a greater than 10% probability of mass extinction from AI within the next decade, while stressing that the risks posed by current AI models remain relatively low compared with those associated with future self-improving systems.
He also acknowledged that Anthropic does not yet have a complete solution for ensuring that future superintelligent systems remain aligned with human intentions and values.
The comments are significant because Anthropic has built much of its corporate identity around AI safety and responsible development.
Race Between Leading AI Labs
Coxon has criticised both Anthropic and OpenAI, arguing that competition between leading AI companies is creating pressure to develop increasingly capable systems before adequate safeguards are in place.
His concerns highlight a growing tension within the AI industry between the commercial and technological incentives to build more powerful models and the difficulty of ensuring that those systems remain predictable and controllable.
The researcher said he was particularly concerned that the companies understood the potential risks but remained locked into a competitive race to achieve breakthroughs ahead of rivals.
Previous OpenAI Experience
Coxon’s concerns carry particular weight because of his experience at both major AI laboratories.
Before joining Anthropic, he worked at OpenAI, where he was involved in pre-training research. His experience has given him exposure to the technical development of advanced AI models as well as the challenges surrounding their safety.
His resignation therefore adds to a growing number of departures and public warnings from researchers who have raised concerns about the direction of frontier AI development.
Earlier in 2026, Anthropic safety researcher Mrinank Sharma also left the company after warning about the broader risks associated with increasingly powerful AI systems.
Safety Versus Speed
The controversy comes as AI companies continue to release increasingly sophisticated models and invest heavily in research aimed at expanding their capabilities.
Anthropic recently introduced Claude Mythos 5.1, which the company described as its most advanced model for life sciences and cybersecurity applications.
At the same time, researchers and policymakers are debating whether existing safety frameworks are adequate for systems that could eventually operate with greater autonomy.
Coxon has argued that AI laboratories may need to consider slowing the development of model capabilities if the industry cannot establish effective safeguards.
Growing Calls for Stronger AI Governance
The resignation adds to a broader international debate about how advanced AI should be regulated.
Researchers, policymakers and civil-society groups are increasingly calling for stronger monitoring, international cooperation and clearer rules governing the development of highly capable AI systems.
The debate is particularly focused on AI alignment—the challenge of ensuring that increasingly powerful systems behave consistently with human objectives and safety requirements.
For Anthropic and its competitors, the challenge is becoming more difficult as technical capabilities advance faster than the institutions responsible for managing their risks.
A Warning From Inside the Industry
Coxon’s departure does not establish that advanced AI will cause catastrophic harm, and predictions about when or whether artificial superintelligence will emerge remain highly uncertain.
However, his resignation illustrates the growing disagreement within the AI research community over how quickly the technology should advance and what safeguards should be in place before more powerful systems are developed.
As Anthropic, OpenAI and other frontier AI companies continue competing to build increasingly capable models, the central question is no longer simply how powerful AI can become.
It is whether the industry can develop that technology quickly enough to remain competitive while building safety measures capable of keeping pace with its capabilities.
