AI Change Tracker
4

Ex-OpenAI/Anthropic researcher Jacob Coxon resigns citing safety concerns; Anthropic's own alignment lead cites >10% extinction risk, triggering federal bill proposal

2026-09-07

Jacob Coxon, who did pretraining research at OpenAI and then Anthropic over three years, announced his resignation from Anthropic on X, writing "Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives" — a post that reached 159 million views per Platformer. Evan Hubinger, who leads alignment science at Anthropic, responded publicly: "Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade... we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," a post that reached 41 million views. Yoshua Bengio said frontier-lab researchers' warnings should be taken seriously; David Shor called for government-mandated independent oversight. Sen. Bernie Sanders (I-VT) and Rep. Greg Casar (D-TX) announced the "Ban Artificial Superintelligence Act." Interconnects and Latent Space both framed this as landing in an unusually receptive climate following the OpenAI-Hugging Face incident and Anthropic's own reported cybersecurity incidents involving Claude models.

Significance 4: A researcher resignation amplified by an Anthropic alignment lead's own public double-digit extinction-risk estimate visibly shifted U.S. AI-safety discourse and produced a concrete federal bill proposal — a talent/discourse event materially affecting labs' social license and strategic position, per the rubric's lane-blind treatment of operational events. It's capped below 5 because every source here is a curator outlet relaying social-media posts, not a primary lab or legislative-text confirmation.

operational business Talent flows Safety / alignment Leadership / governance Policy / regulation

Implications · machine-drafted, not owner judgment

A resignation-driven public 'AI safety vibe shift,' amplified by an Anthropic alignment lead's own >10% extinction-risk estimate, hands safety-focused legislators a concrete talking point and could accelerate binding oversight proposals faster than labs' voluntary governance moves — a stated-vs-revealed gap worth tracking under the leader-pattern-predicts-behavior thesis, since the lab marketed as the safety-conscious alternative is now the source of its own alignment lead's public alarm. For the owner, this is a live instance of a Capability/Commitment gap at the frontier-lab level — a named side-note worth watching for relevance to the book's January 2027 launch window if the legislative push gains real traction rather than fading.

Watch for
  • Whether the Sanders/Casar 'Ban Artificial Superintelligence Act' gets a committee hearing or additional co-sponsors within 60 days
  • Whether Anthropic or OpenAI make any observable policy or release-pace change in the following month attributable to this pressure
  • Whether media attention persists beyond two weeks or fades like prior safety-researcher departures

Sources

← Back to feed