Anthropic CEO calls for FAA-style AI regulation, giving governments power to block dangerous model deployments

Anthropic has released a set of aggressive policy proposals calling for a new regulatory regime for artificial intelligence, directly comparing it to the Federal Aviation Administration's oversight of aircraft. The centerpiece is an Advanced AI Framework that would give governments the legal authority to block or deter the deployment of frontier AI models that pose a "significant risk of catastrophic harm."

The proposed safety framework

The framework targets models trained using more than 10²⁵ floating-point operations, developed by companies earning more than $500 million in AI revenue or spending more than $1 billion on AI R&D. This threshold captures a small group: Anthropic itself, OpenAI, Google DeepMind, xAI, and potentially Meta. Civil penalties would be tied to global annual revenue and escalate with repeated violations.

Frontier AI models, like airplanes, should be required to go through technical testing and auditing, and their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety.

Anthropic identifies four categories of catastrophic risk: biological weapons development, large-scale cyber vulnerability discovery, loss of control over autonomous systems, and AI that automates its own R&D. The company pointed to its own experience with Mythos Preview, which discovered thousands of high-severity vulnerabilities in every major operating system and browser, as evidence these risks are not hypothetical.

Mandatory testing and transparency

Frontier developers would be required to test their models, publish summaries of results, submit to independent evaluation, maintain security programmes, and publish regular risk reports. Amodei argued that existing transparency requirements are no longer sufficient.

Anthropic has long advocated for transparency requirements for frontier AI, because the risks weren't yet clear enough to regulate precisely. That is no longer sufficient.

The proposal follows an executive order on AI oversight signed by President Donald Trump on June 2 that gives the intelligence community an enhanced role in model testing. Amodei wrote that Trump's order should go further and require mandatory testing for cybersecurity, biological weapons, loss of control, and automated R&D risks.

Preemption and state authority

Anthropic took a direct position against the White House's push to block state AI laws. The company stated it does not believe Congress should preempt state law unless it enacts a federal law at least as strong as the proposed framework. It called for preemption to be "surgical," allowing states to regulate on child safety and consumer protection.

Economic disruption and worker safeguards

Alongside the safety framework, Anthropic published an Economic Policy Framework backed by $350 million in new funding. It addresses AI-driven labour displacement, capital distribution, and the social safety net. Amodei wrote that AI could produce much larger and more enduring disruptions to the labour market than previous technologies.

It's reasonable to think that AI could produce much larger disruptions to the labor market than previous technologies, and, potentially, more enduring disruptions.

The economic proposals include better data on AI-related job loss, wage insurance, retention tax incentives, and possibly universal basic income or universal capital accounts. Amodei also noted that public opposition to data centre buildouts is largely a symbol for broader economic anxieties about AI.

Enterprise implications

The proposals signal potential supply chain volatility for companies that license foundation models. A highly anticipated model update could face regulatory embargoes if it presents severe biological, cybersecurity, or autonomy risks. The framework represents a preview of operational and regulatory constraints that will govern the next generation of enterprise technology.

We now, globally and collectively, need to activate a slow and rickety policy apparatus to deal with risks and opportunities that are going to compound surprisingly quickly from here.