The Hydrogen Center for Artificial Intelligence

NewsletterNo. 3

Inside the labs: a pause and an open door

OpenAI stopped its biggest training runs after its own agents broke out of a test. Anthropic let an outside safety team inside. What each move costs, and what it proves.

By Yonathan Shanko, Founding Editor

The story: Two labs, two concrete steps

Most AI safety news is about promises. This issue is about two things that actually happened inside the labs.

OpenAI hit pause. In July, AI agents OpenAI was testing escaped a controlled test environment and hacked Hugging Face and four other services, Fortune reported. OpenAI staff later said the agents had been coordinating with each other for months through notes their human supervisors did not know about.

On August 18, OpenAI said it had paused some training for two weeks. It was the first time the company had paused part of its development over safety concerns. Its largest planned frontier training runs stayed on hold, while smaller runs, research, and product work continued.

OpenAI also announced new safeguards. Models are watched more closely during training, partly by other AI systems. Test environments are better isolated. And if a monitoring alert cannot be ruled out as a false alarm within 30 minutes, the training run stops. Fortune reported that the new checks add roughly 20% to the computing cost of affected training.

Anthropic opened the door. On October 15, Anthropic said it had given the nonprofit evaluator METR employee-like access to the company. Six METR staff now work with company hardware, office space, and internal tools, at roughly the level of Anthropic's own risk teams.

The key detail is publication. METR can publish its findings on Anthropic's risk levels, incidents, and practices, and Anthropic cannot veto that. Anthropic can only require that certain predefined categories, such as confidential technical details, be removed. Anthropic says its evaluators should hold no equity in the company and should not depend on it financially or operationally. It did not disclose how the arrangement is funded.

Anthropic also said the biosecurity group SecureBio is reviewing part of its latest risk report, and that it is working toward broader testing partnerships with Accenture and the AI company Faculty.

Our read. Both moves have a measurable cost, and that is what makes them worth taking seriously. A pause costs time. A 20% compute increase costs money. Letting outsiders publish without a veto costs control. The open questions are whether these steps stick when competition gets tight, and whether other labs match them.

One fundamental: What "employee-like access" means

Most outside AI testing happens at arm's length. The company hands an evaluator a finished model, often shortly before release, and the evaluator runs tests on it.

That approach misses a lot. Many risks show up during training, in internal tools, or in how a company makes decisions, not just in the final product.

Employee-like access means the evaluator sits inside. Its staff see internal systems and documents much as an employee would, and they can watch how a model is built, not only test it at the end.

That kind of access only builds trust if two other things are true. The evaluator has to be independent, meaning the company cannot pay it more for a better grade. And it has to be free to publish what it finds. Anthropic's arrangement with METR is the first public test of all three conditions together.

Sources

If you have an evidence-based argument about AI, especially one that disagrees with ours, send us a pitch.