Anthropic Researcher: >10% Chance AI Kills All Humans — Alignment Plan Missing

Anthropic safety researcher Evan Hubinger publicly stated he believes there is a greater than 10% chance AI "could kill all humans" within the next decade. In a post on X viewed over 10 million times, Hubinger said current models pose "low" risk, but he's "worried" about rapid self-improvement leading to existential threats.

Key warnings from the post

• “We really do earnestly believe AI poses a species-ending risk to humans.”
• “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”

Hubinger's comments came in response to Jacob Coxon, a self-described AI researcher who quit Anthropic and previously worked at OpenAI. Coxon wrote: "Neither company is acting responsibly. These will soon be superhuman systems that can hack anything, revolutionise any field overnight, and acquire real power and resources."

Context: Withheld model and recent incidents

The Financial Times reported Anthropic withheld its latest model from the UK's AI Safety Institute (AISI). A Cabinet Office spokesperson didn't confirm, but stated the UK "continues to collaborate closely with industry partners, including Anthropic, to make models safer."

This follows a summer of disclosed cyber-attacks carried out by AI agents from OpenAI, Anthropic, and Meta — all described in their own safety reports. In Anthropic's August safety report, the company rated low risk of models becoming misaligned with a hypothetical powerful organization, but added: "We are seeing early signs of potential acceleration." They were "less confident" in their assessment than previously.

OpenAI's chief scientist Jakub Pachocki also called for "extreme caution" over AI's progress, warning more intervention may be needed to ensure "humans remain in control of the future".

The larger picture

Leading researchers have signed open letters calling for slowed AI development, including 1,300 staff from AI firms urging the US government to "support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development."

University of Cambridge Professor Neil Lawrence commented on BBC Radio 4's Today Programme that the report was credible, linking it to US isolationist tendencies and the AI race with China.