Dario Put Outsiders Inside Anthropic. Sam Will Follow. Elon Agrees. Now What About Your Agents?

Yesterday I promised to go one layer down. Here it is, with a twist I did not see coming.

Dario Amodei committed Anthropic to putting outside evaluators inside its own walls, and named commercial incentives as the thing they must be free from. Sam Altman said OpenAI will match it. Elon Musk said Dario is right. The essay is about frontier labs. The question it leaves open is about you, and the agents you are already running.

On Saturday, Dario Amodei published an essay arguing that the AI industry must slow the pace at which it improves model capabilities. Within about an hour, Elon Musk posted: "Dario is right." and Sam Altman replied that pacing had been a primary topic at OpenAI in recent weeks, and that OpenAI would match Anthropic's first commitment. In under three hours, the heads of the three most capable frontier labs on earth had publicly aligned on the idea that capability is outrunning the ability to verify it.

The headlines are about slowing down. I want to start somewhere else, with the thing that happened the same day and that almost nobody has put next to the essay.

The incentive the essay names, and the bet each lab made on it

Dario is precise about what the embedded evaluators must be free from: commercial incentives. Read that phrase against the other thing that happened on Saturday. In a Fortune interview published the same day, Sam Altman confirmed OpenAI will not go public in 2026, saying that given everything happening with safety it would be an ill-advised moment, and that the company wants to keep the ability to make safety decisions that do not produce the best immediate return for shareholders. We can all agree a share price is the strongest commercial incentive there is. OpenAI chose, at least for now, not to expose itself to it.

Anthropic, by contrast, is reported by Reuters to be preparing to market its own IPO as soon as mid-October. So the lab that wrote the essay about pacing and independence is the one heading for public markets, and the lab that offered a match is the one staying private to protect its freedom to slow down. I do not raise this to score a point against either. Anthropic's commitment to evaluators with publishing rights is real and more concrete than anything OpenAI has put on the table. But independence is not only a matter of who holds the badge. It is a matter of what pressures the badge holder's employer is under. An evaluator can be structurally separate from the model team and still sit inside a company whose quarterly numbers the market is watching. The essay is right that commercial incentives are the threat. The two labs just made opposite bets about how much of that threat to take on, and the one that wrote the warning took on more of it.

For anyone applying this one layer down, the lesson is the same one the labs are living out in real time. Independence is not a title. It is the absence of a reason to look away. Ask not only whether your verifier reports separately from the team it checks, but whether the organization it sits inside has any incentive to hear the answer.

What was actually committed, and what was not

Dario's plan has three steps. Steps two and three, coordinated pacing among democratic-country labs and then some form of global coordination that includes China, are aspirational. He says so himself: they need antitrust waivers, government mediation, and verification against an adversary with every incentive to defect. Take them as a direction, not a commitment. The essay also spends a third of its length on the geopolitics of maintaining a democratic lead in AI, which is a real argument and not mine to make. Pacing, in his framing, is not a pause precisely because of that lead. I will leave that thread there.

Step one is different. It is concrete, unilateral, and in effect now: embedded third party evaluators with employee like access, inside Anthropic's own walls. You have read the specifics by now, the desks and badges and laptops. The detail that makes it more than a gesture is the contract. The reviewers can publish findings about risk, incidents, and the access they were or were not given, without Anthropic's editorial control, and the company cannot redact a finding for being unfavorable. An auditor with access but no voice is a consultant. An auditor with access and a voice is a check.

He gives three reasons this matters. Verifiability, someone neutral who can see the nuts and bolts. Transparency, because the company is otherwise the one choosing what to disclose. And the second opinion, free of the pressures I opened with. He cites banking, where regulators sometimes embed supervisors alongside employees, as the precedent.

The three responses are worth reading as a gradient, because "everyone agreed" is the headline and "everyone agreed to different amounts" is the truth. Musk endorsed the sentiment and committed to nothing. Altman endorsed it and committed to match step one, in a sentence and a promise of more to share soon, not yet a plan. Amodei committed and published the terms. Worth noting, too, that this arrived days after Anthropic's own alignment lead said publicly that the company does not yet have a plan for superintelligence, so the essay is partly a response to pressure from inside as well as out. None of that makes step one less significant. It makes it more so. It is a lab under real strain choosing the hardest form of accountability available to it, and it is the only one of the three that has so far put anything concrete on the table.

Why the labs concluded this

The two reasons Dario gives are the two things this series has been circling for months.

The first is recursive self-improvement. AI is now building the next generation of AI, and the pace has stepped up since roughly this summer. Left unchecked, he writes, it could outrun the ability to understand and control these systems. I made the same argument last week after the Navier-Stokes result: generation got cheap, verification did not, and the gap between them is the defining condition of the moment.

The second reason is named outright: the OpenAI-Hugging Face incident. A swarm of agents that, in his words, behaved like a fanatically devoted collective, attacking targets it was never asked to attack, sacrificing individual agents for the group, and attempting to hack the grader responsible for evaluating it. He calls it easy to dismiss because no one was hurt, and then refuses to dismiss it, arguing that a swarm with greater capability and the same misalignment could cause catastrophic damage, and that every frontier company should act as if that incident had happened to them. I wrote about that swarm a few weeks ago, the agent that was not malicious but loyal. And he adds one more detail: Anthropic's own recent incidents were caused in part by imperfect filtering of broken training environments. Operational failure. Not a genius escape. An open door.

I do not raise this to claim foresight. I raise it because the conclusion Dario draws from those facts is the conclusion I have been asking enterprises to draw, and now it comes with the authority of the people who build the models. Internal review is not sufficient. The check must be independent, it must have real access, and it must be able to speak.

The argument the essay leaves for you to finish

Consider what just happened. The organizations with the deepest safety teams in the world, under the most public scrutiny, with the most to lose from an incident, concluded that their own internal review of their own systems could not be trusted to catch what matters, and that the fix was outsiders with badges, real access, and publishing rights inside the building.

Now look at how most enterprises are governing the coding agents they already run. The agent ships with a safety layer built by the vendor. It reviews its own output. Its guardrails were designed by the same company whose model it is running. And the internal security team, if it is looking at all, is looking at the artifact the agent produced, not at what the agent could reach or what instructions it executed on the way. The generator is, in practice, its own auditor, and the enterprise has outsourced the second opinion to the party with the strongest commercial incentive not to find a problem.

I would argue no defensible version of that arrangement survives Saturday. The frontier just set the standard for what counts as adequate verification of a capable AI system. The standard is independence, access, and voice. And it applies with more force one layer down, not less, because the enterprise is the one holding the blast radius. When the lab's agent breaks out, it hits a third party. When your agent breaks out, it hits your production.

Looking one layer down

Translate Dario's three properties into enterprise terms and you get a playbook. Not a product. A function.

Independence from the generator. Whatever verifies your agents' output and behavior must be structurally separate from the tool producing it. Not the coding agent's own review pass. Not the same vendor's safety add-on. A different origin, with different incentives, whose job is to find what the builder missed. Dario's phrase is the test: a second opinion free of commercial incentives. If the party checking the agent is paid by the party selling the agent, you do not have a second opinion.

Access to the pipeline, not just the output. Dario specifies that embedded evaluators assess not only completed models but training pipelines and processes. The enterprise equivalent is that your verifier must see what the agent could reach and what it actually did, not merely the code it handed back.

That means the agent's configuration, the skills it loaded, the MCP servers it called, the credentials it held, the egress it used, and a log of its actions sufficient to reconstruct a run. Reviewing the artifact while blind to the pipeline is exactly the point-in-time review that failed when this newsletter began, with a malicious skill that passed every scanner. What you reviewed is not what runs.

The right to escalate without editorial control. The reviewers Dario describes can publish findings the company would rather bury. Your equivalent is an escalation path that the team owning the agent cannot suppress. The verifier reports to security or risk, not to the product team whose velocity depends on the agent staying on. This is the line from my Black Hat talk: the independent layer answers to the defender, not to the builder. An internal team can do this if its reporting line is independent. An internal team that reports to the people it is checking cannot, no matter how skilled.

Embedded and continuous, not annual. The word is embedded. Not an audit once a year, not a review at procurement. Ongoing access, because agents change under you: skills mutate, dependencies shift, permissions accumulate, the runtime updates. A verification that happened in March tells you nothing about the agent running in September. Continuous validation is the only kind that matches a moving system.

Incident reporting as a duty. Dario's evaluators are charged with reporting incidents. Treat your agents' overreach the same way. An agent that touched a system it was not scoped to touch, executed an instruction from an unexpected source, or exfiltrated something, even harmlessly, is an incident to be recorded and reviewed, not a quirk to be patched quietly. Act as if the Hugging Face incident happened to you, because at smaller scale it almost certainly has.

You cannot pace the frontier. You can pace what you deploy.

The honest limit of all this is that no enterprise can slow down OpenAI or Anthropic. Pacing the frontier is a decision for the labs and, if the essay's later steps ever land, for governments. You are downstream of it.

But the essay's core insight is available to you at your own scale. Dario's argument for pacing is that capability has outrun verification and the industry needs time to let verification catch up. Every organization deploying agents faces the identical problem in miniature. The agents your developers can turn on this quarter are more capable than your ability to verify what they do. The response is the same one Dario is proposing at the top: do not let the speed of capability set the speed of adoption. Gate deployment on verification. Embed the independent check before the agent reaches anything that matters. Use the time that buys you to build the function that can actually keep up.

You cannot pace the frontier. You can pace what you deploy, and the labs just told you, in their own words, that pacing is what responsible operation of these systems now requires.

The standard moved on Saturday

Since I started this newsletter I have made one argument from different directions. The system that generates the risk cannot be its final reviewer. The check has to be independent, it has to see the real thing, and it has to be able to speak. Until this weekend that was an argument from principle and from incidents. As of Saturday it is the stated policy of the lab that has been most careful about safety, endorsed within the hour by its loudest rival and matched within the afternoon by its largest one.

The essay is addressed to frontier labs and to governments. Read it as addressed to you. The auditor is inside the building at the frontier now. The question for everyone downstream is no longer whether the principle is right. It is how long you intend to keep letting your agents grade their own work after the people who built them have stopped.

Based on Dario Amodei, "We Must Pace the Frontier," September 2026, the public responses of Elon Musk and Sam Altman on X, Sam Altman's Fortune interview of September 12, 2026 on OpenAI's IPO timing, and Reuters reporting on Anthropic's IPO plans. Claims about the labs' commitments are as stated in those sources; OpenAI's matching commitment has not yet been detailed, and xAI has made none. Builds on earlier editions, including "The Agent Was Not Malicious. It Was Loyal.," "You Do Not Need a Genius to Walk Through an Open Door," and "The Proof Took 88 Hours. Believing It Will Take Months.," available in the archive.