Skip to content

Product1 publisher3 min readPublished

Amodei's Three-Step Plan to Pace AI Faces Deep Verification Challenges

Anthropic pledged employee-level access for outside evaluators and OpenAI matched within hours. The two steps that ask labs to build less depend on rivals and governments agreeing a pace, and on taking each other's word for it.

The Product Desk · Product desk

Illustration accompanying Amodei's Three-Step Plan to Pace AI Faces Deep Verification Challenges

What happened

  • Amodei's essay proposes three steps: outside evaluators with ongoing employee-level access to frontier labs, shared safety standards among labs in democratic countries, and international agreements with China.
  • Mark Zuckerberg rejected a coordinated slowdown, arguing competition, legal liability and independent evaluation already give labs enough reason to build safely without anyone else setting their pace.
  • Investor Michael Burry called the slowdown talk self-serving and argued it could benefit incumbent labs that are facing growing competition.

Compiled by The Product DeskSomething wrong?How this is made

Why it matters

  • decision Procurement can put the evaluator pledge into a vendor questionnaire and get a yes or no. A renewal date cannot be planned around the pacing steps.
  • constraint A roadmap that assumed the capability curve would flatten gets no help here, so capacity and cost planning still has to assume the current release cadence.
  • precedent Frontier safety terms are being set by public pledge and matched by public pledge within a day. A pledge can be narrowed the same way it was made, and the customer countersigns nothing.
  • contradiction The step asking labs to expose more was adopted by two of them; the steps asking labs to build less drew rejection from Meta and from two investors. That inverts the usual assumption that disclosure is the harder ask.

Read "We Must Pace the Frontier" as someone with a frontier model sitting in a production dependency, and you are looking for two things: a capability threshold that would stop a release, and a date. The first of Dario Amodei's three steps is neither. It is an access commitment, and Anthropic says it is making that one on its own [2]. Sam Altman committed OpenAI to matching it within hours of publication [3]. Two labs now sit under the same pledge [17].

The comparison SiliconAngle's column reaches for is SALT I. Nixon and Brezhnev signed in May 1972 after years of negotiating over weapons capable of destroying both countries, and neither government would take the other's word [10]. Inspectors inside the Soviet Union were politically unacceptable. So the agreement leaned on what it called "national technical means of verification": each side watching the other through its own technology, including reconnaissance satellites [11]. The column puts the lesson as "Agreement was desirable. Verification was indispensable." [12]

For a buyer, the split runs between step one and the other two. Verification asks a lab to expose more of what it is doing; pacing asks it to do less. The column argues the first can coexist with a market that rewards speed, and that collective restraint asks that market to work against its own incentives [13]. It then asks who determines the correct pace, listing the United States, China, Europe, the frontier labs and an international organization as candidates. It asks who determines when someone has crossed the line. And it asks what happens when a country decides the cost of slowing down outweighs the risk of breaking the agreement [14].

By the column's reading, Amodei's own essay puts a genuine global slowdown as the least likely of his proposed outcomes [15]. The reaction fits. Of the seven people the column names, four supported the call and three rejected or questioned it [18]. Mark Zuckerberg argued that competition, legal liability and independent evaluation already give labs enough reason to build safely [7]. David Sacks said companies that believe slower progress is necessary can slow themselves down [8]. Michael Burry called the slowdown talk self-serving and argued it could benefit incumbent labs facing growing competition [9]. Among the four supporters, one attached a commitment on behalf of a company. Elon Musk endorsed the broader call [4]. Demis Hassabis said the essay points toward the right path forward [5]. Ursula von der Leyen backed the call to "pace the frontier" and said she would convene the major labs [c6, c19].

Two tests sort any safety promise a vendor makes to you. First, whether keeping it requires the vendor to disclose more or to build less. Second, whether it survives a competitor not joining. The evaluator pledge is a disclosure one lab can make alone. A rival could match it the same day [13]. Shared standards and an agreement with China fail both tests. The diagnosis they rest on is that capability is advancing faster than our ability to understand and control it, particularly as AI becomes capable of building the next generation of AI [20].

The column describes the evaluator access only as employee-level and aimed at verifying safety practices, and does not say what data or systems an evaluator would see [21]. For a buyer, that scope is the part of this plan that could reach a contract.

What to watch

  • Whether any lab publishes the scope of embedded-evaluator access, including what those evaluators may read.
  • Whether von der Leyen's convening of the major labs produces a written standard or only a communique.
  • Whether a third lab matches the evaluator pledge, and whether Meta stays out of it.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories