AI Evaluator Forum publishes AEF-1 baseline standard as Anthropic proposes expanded 'Pacing the Frontier' framework
2026-09-11
The AI Evaluator Forum published AEF-1, a proposed baseline standard for independent third-party AI evaluations covering access, conflicts of interest, funding relationships, recusal, and transparency; OpenAI, Anthropic, and xAI cosigned it. Separately, Anthropic's Dario Amodei wrote a personal blog post describing a follow-up to Anthropic's earlier 'Pacing the Frontier' proposal built on three mechanisms: embedded third-party evaluators (citing METR) with employee-like access — desks, badges, laptops — to Anthropic's offices and training pipelines; democratic coordination among frontier AI companies in democratic countries; and global coordination including with authoritarian governments. Anthropic said it is unilaterally committing to the embedded-evaluator step now. Cohere CEO Aidan Gomez publicly argued against a small number of Silicon Valley companies becoming AI 'gatekeepers.' Coverage is via Latent Space's AINews digest of Twitter activity from Sept 11-14.
Significance 3: A cosigned industry evaluation standard and a named lab's unilateral governance commitment would ordinarily rate a 4, but the only source in this run is a single curator's digest of social-media activity with no primary Anthropic or AEF statement in hand, so significance is held down pending corroboration.
Implications · machine-drafted, not owner judgment
If Anthropic's embedded-evaluator commitment holds, it would be the most concrete, hard-to-reverse governance step any frontier lab has taken this year, giving METR-style evaluators a standing seat inside a lab's training pipeline rather than after-the-fact access — a real test of the leader-pattern-predicts-behavior thesis's walkback pattern. Cohere's pushback signals the standard could calcify into an incumbents' club that smaller or open labs are shut out of, which would reinforce rather than reduce the 'gatekeeper' dynamic Gomez is warning about. This is exactly the stated-vs-revealed governance gap the leader-pattern thesis tracks.
- A primary Anthropic post or AEF statement confirming AEF-1 and the embedded-evaluator commitment.
- Evidence METR or another evaluator has actually received the described office/badge access inside Anthropic.
- Whether a second frontier lab (Google DeepMind, Meta) joins or publicly declines AEF-1.
Sources
- curator AEF-1 standard emerges for Third Party Evaluators, as Xai, OpenAI, and Anthropic all cosign retrieved 2026-09-15
- primary Who Gets to Define the Rules for AI? AI Needs Evidenced Standards, Not A Cartel retrieved 2026-09-15