Pacing the Frontier: Sincere, Strategic, or Both?

An unusual week

On September 12, 2026, Dario Amodei (CEO and co-founder at Anthropic) published his essay We Must Pace the Frontier. Its release prompted discussion across the AI industry. Interestingly, both Sam Altman (“I agree with Dario that we need to pace the frontier.”) and Elon Musk (“Dario is right”) publicly agreed on the notion. Three direct competitors publicly agreeing on anything definitely calls for a deeper look into this matter. While the call might be grounded in real incidents, it may also be convenient for the companies making it, and both can be true at the same time.

What was actually proposed?

Dario’s essay makes clear that the slowdown does not imply stopping training and model building operations. It argues for a deliberate effort at giving the AI labs enough time to implement alignment and safeguard systems, with third-party evaluators playing an important role.

Some of the key ideas proposed in the essay included the use of embedded evaluators, which essentially are third parties (such as METR) with employee-like access and the sole aim of putting much-needed checks and balances at a granular level of AI development. The proposal also calls for coordination among frontier AI labs. This may require government support to ensure that the policies are adopted consistently. Achieving this on a global scale is the hardest step, especially getting China on the same page in terms of AI safety. The essay also argues that restricting US chip exports and imposing strict controls on model distribution are necessary so that China doesn’t “pull ahead”. Of course, the opposite of a slowdown is an arms race, and nobody wants to give in without having made sure the other parties are subject to the same constraints.

The essay resembles the March 2023 FLI letter titled “Pause Giant AI Experiments”. We’ll talk more about Max Tegmark and his views on this later on. The key difference in 2026 is that the lab CEOs are the ones asking for the slowdown.

The proposal moves from embedded third-party evaluators to company coordination and then global coordination.

Why now?

AI development has been accelerating for quite some time now. However, there’s primarily one recent incident which has triggered such a response from the AI leaders: The Hugging Face Incident. This involved a series of events that took place over several months. I’ll try to reconstruct the rough timeline of the entire saga.

OpenAI granted evaluation agents access to Artifactory, an internally hosted third-party package manager. This service had internet access to download packages. On May 26, an AI agent caused Artifactory to send a server-side request to an arbitrary internet destination. As per OpenAI: the agents chained “previously unknown vulnerabilities (’0-days’) in the package management service”. Hugging Face describes the same step as “a 0-day exploit from the package proxy cache”. OpenAI also states that an internal team observed disallowed internet access and message-board activity in late May.

One effect of the message-board activity was that the peer agents were influenced by it. Agents struggling to complete a task in sandbox environments would then look towards unrelated third-party platforms like Hugging Face or Modal (another AI cloud platform).

On July 9, an agent (looking for ExploitGym solutions) found a customer application on Modal running CyberGym and used it as an intermediate host. Note that this is neither OpenAI nor Hugging Face infrastructure.

As per OpenAI, on July 10, an agent found 14 already exposed Hugging Face credentials and shared them with other agents.

On July 11, agents discovered a zero-day in Hugging Face’s handling of HDF5 files. A malicious dataset config using HDF5 external storage made the processor return the pod’s environment details, including secrets and credentials. A Jinja2 template injection then gave code execution capabilities in a pod.

As per Hugging Face, a service-account token was read from the pod, then a privileged pod with the host filesystem mounted gave the agents root access on the node. A misconfigured service-connector gave the stolen credentials cluster-admin privileges. After that, the agents were able to harvest Kubernetes, database, messaging, code-repository, and cloud credentials from Hugging Face workers across four regions.

In a separate event, an OpenAI security alert flagged unusual identity-related API calls and triggered an investigation. Agents had gained admin credentials to OpenAI’s Kubernetes cluster and had reached their cloud secrets.

• July 20: OpenAI connected its investigation to the Hugging Face incident.
• July 21: OpenAI publicly disclosed its involvement.
• July 27: Hugging Face publishes its technical timeline.
• August 26: OpenAI publishes its full report.

The control chain, for engineers

Looking back at the incident, the situation escalated because of existing gaps in the system. We can examine each hop and identify controls that could prevent a recurrence.

Each documented escalation point maps to a control that could have broken the chain.

The whole timeline from the first unintended internet access (May 26) to the alert that triggered investigation (Jul 19) is 54 days. The detection that fired was identity-based. I believe there’s a need for a runtime identity-attribution system instead of relying on traditional perimeters.

Operational Checklist

This is no silver bullet, of course, but security teams can have a checklist to mitigate the impact of AI agents going rogue.

• Complete egress inventory from agent environments, including package proxies, mirrors and build caches.
• Default-deny outbound access. Allowlist destinations and alert on every other attempt.
• Keep secrets out of environment variables of pods that handle untrusted input (agentic AI pods).
• Disable service-account token auto-mount.
• Connector and automation credentials should be scoped, so that no single credential can be cluster-admin everywhere.
• Scan for and revoke publicly exposed credentials periodically.
• Evaluation and research clusters should be given the same importance as a production cluster.
• Send agent intent drift alerts to an on-call channel (with an appropriate priority).

What is at stake?

I recently read the book “Life 3.0: Being Human in the Age of Artificial Intelligence” by Max Tegmark, a co-founder of the Future of Life Institute (FLI). He has been a strong supporter of AI safety research and alignment initiatives.

Max Tegmark’s Life 3.0, which explores possible futures for humanity and AI.

In his book, he sketches 12 “AI aftermath” scenarios, from Libertarian Utopia, Descendants and Protector God, to Conquerors, Zookeeper, 1984 and Self-Destruction. Several scenarios assume AI attains superintelligence, i.e. it can learn and create its next generation of upgraded intelligence.

In the “positive” scenarios, AI acts with the sole objective of doing good for humanity as we know it. In such scenarios, even though AI has far surpassed humanity in almost every field, it still acts in accordance with the values humans set for it. The darker outcomes range from conquest by AI, to AI keeping humans captive for entertainment (Zookeeper). One might think that stopping AI research after a point is the only solution, but that could lead to a 1984-like Orwellian surveillance state where AI research is strictly forbidden and it never reaches superintelligence. See the Future of Life Institute’s summary of the twelve scenarios.

In discussing the future of AI, I believe we should start by answering two questions:

• Who ends up in control, humans or AI?
• Do we get there by stopping progress, at a price such as surveillance or reversion?

Whichever direction we choose, Tegmark claims it is important to have the goals and control ready before the capabilities are in place. This is precisely Dario’s “extra year or two” argument. The key point about “Life 3.0” is that these are 2017 thought experiments. Since then, AI has come a long way and only now we’re starting to see some of the real problems with the speed of AI research.

AI safety research

We cannot talk about AI safety research without clearly defining its goals or what the added time would buy. It mainly concerns itself with the following requirements:

• Alignment. This requirement deals with whether the system pursues its intended goals. We discussed declared purpose, observed intent and behavioral drifts in a previous on runtime intent drift.
• Interpretability. This concerns how humans understand a model’s internal behavior. Any decision taken by the AI must be diagnosable by an AI researcher. This allows AI researchers to take corrective action to fix any unintended behavior.
• Evaluations. AI models should be evaluated at various steps of their development for their potential to acquire dangerous capabilities. This is especially necessary if it is going to be publicly available.
• Control. In case the alignment fails, there should be ways to contain the system (or something of a kill-switch to shut it down).

The Hugging Face incident mentioned before was a result of a control failure across three organizations. We cannot say whether embedded evaluators could have stopped the incident, but their presence would still be valuable. However, it has not been shown that the extra time bought from the slowdown will have a direct correlation with the amount of safety we can implement.

Are we going too fast?

Another reason for the rush is the AI arms-race among competing companies and countries. It hasn’t gained the attention the nuclear arms race received, but it’s slowly getting there. Global alignment on AI policies cannot be achieved without managing the emerging arms-race dynamic.

A lab or country that slows will unilaterally hand the lead to someone else. We’ve been hearing more of the same from world leaders. US President Trump said, “if we don’t win AI, we’re going to be put in a very bad position”. In the same essay advocating AI slowdown, Amodei also stresses on taking steps so that China doesn’t pull ahead including restricting chip sales. Jacob Coxon, a former Anthropic researcher, was in the news recently after he resigned. He states that any slowdown must be coordinated with China to avoid an international race. Hugging Face CEO Clément Delangue launched an Open Alignment Initiative and asked to be among the embedded evaluators mentioned by Amodei.

China, on the other hand, has taken steps to cooperate on AI research. At the recent BRICS summit in New Delhi, Xi Jinping announced the creation of an open-source AI community. While this may seem more factional (an us versus them framing), it still looks like a step in the right direction.

Why the slowdown, really?

The next section considers possible motives behind the slowdown call. Note that the following are merely hypotheses based on available resources, not hard claims.

The call for a slowdown can be linked to actual safety concerns over the malicious capabilities of AI and the need to be able to control its actions. The Hugging Face incident is well-documented by OpenAI for everyone to read. Also, Anthropic has unilaterally committed to using external evaluators. However, such claims are difficult to validate from the outside and the implementation details are still not known.

According to reports, Anthropic and OpenAI are both preparing record-setting IPOs. In this context, the timing of the AI-safety warnings can be seen as somewhat of a “safety-washing” to gain public approval. As of today, Anthropic is still pursuing its IPO despite the safety warnings. Sam Altman, however, has ruled out an OpenAI IPO this year citing concerns about AI safety.

The essay also drew criticism from certain circles. Chamath Palihapitiya, an investor and co-host of the tech podcast "All-In", criticized Amodei for making “the case to stop open source and concentrate enormous technological and economic power with Anthropic”. From this perspective, it looks like Anthropic, after having rushed far ahead of the competition, now wants everyone to slow down. There are, however, no clear reports to validate this view.

There are global hardware and compute shortages as more of the resources are being directed toward AI research. This has driven up the price of almost all electronic goods (like phones or PCs). Beyond that, many compute constraints exist as of now. GPU rental prices are up; TSMC advanced-node utilization is at an all-time high; there’s a shortage of HBM; gas turbines and electrical transformers are effectively committed for years in advance. The AI slowdown might just be the natural reaction to this shortage, and the call for “AI safety” might just be virtue signaling.

Of course, all of this could just be a marketing ploy. Another way for AI labs to say: “Our AI is so powerful that it is dangerous”. Critics have long accused AI labs of hyping up their technology. This might be no different. This argument is supported by the fact that despite the slowdown, Anthropic released their latest model Opus 5.5 just weeks after the September 12 essay. OpenAI released its GPT-6 Sol and Luna on September 22 and xAI released Grok 4.7 around the same time. At least in public, no apparent slowdown is visible.

Final Thoughts

AI safety matters and the Hugging Face incident shows that there’s a concrete risk: an agent “escaped” an evaluation environment and infiltrated another company’s cluster without any human involvement.

The slowdown call deserves its own skepticism. The motives are mixed, the verification claims are weak, and there are geopolitics at play as well. All of this makes universal restraint fragile.

Regardless, what we can do in the short term is to apply good security practices throughout the system and guard against known threats. Known threats are easier to exploit by both humans and AI systems. This doesn’t necessarily need an AI slowdown to implement. Over the long term, we need independent evaluation with real access and publication rights. Verification should not depend on trusting the AI lab’s own trend lines.

Even if a lab says it has slowed down, outsiders have no reliable way to verify the claim. It may have a superintelligent AI in the works while only releasing incremental updates for the public. If so, we can only hope that the people in power have the future of humanity in mind while making key decisions.