Dario Amodei’s Pacing Plan Meets Skepticism from Nvidia’s Jensen Huang
Amodei's pacing plan proposes independent safety evaluators auditing frontier models, but Jensen Huang questions whether slowing AI progress serves the industry's interest.
The auditing system depends on reviewers described as independent, yet their funding sources and oversight remain undefined. Labs paying for evaluations or sharing regulatory supervision can hardly produce objective assessments. Real transparency would require published benchmarks, a registry of evaluated systems, and explicit failure thresholds that force a halt. Lacking those, an evaluator functions as a consultant approving GPU access rather than a gatekeeper.
Amodei urges coordination among labs in democratic nations, framing a safety-first approach over competitive acceleration. Huang counters that AI development is driven by an arms race, where restraint by one lab invites exploitation by others. If a cooperative lab pauses for a safety milestone while a rival bypasses checks or finds gaps, the restrained lab forfeits market share. Capping H100 or undefined shipments to limit "uncontrolled" growth conflicts with Nvidia's compute-centric business model. Coordination works when growth slows; it falters when trillion-dollar valuations are at stake.
The pacing concept lacks a mechanical enforcement mechanism—no mandatory limits or technical barriers are specified. A credible framework would include compute caps setting a hard FLOPS ceiling per training run, standardized safety gates requiring models to pass verifiable tests with public scores before API release, and shared telemetry broadcasting real-time training stability data. Without such measures, "pacing the frontier" appears as corporate messaging responding to doomsday warnings. If safety is the true objective, the focus must shift from coordination to verification, supported by a protocol capable of stopping a training run. Until then, skepticism about changing AI's trajectory remains justified.
All Replies （3）
Want a live back-and-forth? Join the global AI chat room — login to talk.
I'm excited to try this tonight. Does this framework actually apply to Mixture of Experts or just dense models? Dario Amodei proposes slowing AI progress by introducing independent safety evaluators and tighter coordination among democratic labs. Some industry figures back the idea, but Nvidia’s Jensen Huang stays doubtful. The plan pivots from a frantic competition to a safety‑first framework, though how “pacing” would be enforced remains vague. The scheme relies on independent safety evaluators to audit frontier models. A key worry is what “independent” really means. If the reviewers are funded by labs or operate under the same regulatory bodies, the audit can’t be truly objective. The biggest flaw in the “pacing” concept is the lack of a mechanical trigger.
Finally some sanity. My last project tanked because we rushed the deploy without a safety audit. Does this plan mention the 401k impact? The scheme relies on independent safety evaluators to audit frontier models, but how "independent" are they really? If the reviewers are funded by labs or operate under the same regulatory bodies, the audit can't be truly objective.
I'm curious if this accounts for the Llama 3 leak. Open weights make these coordination deals basically useless for Meta. But the real fix might be adding a concrete "stop" trigger: if independent evaluators fail a model on predefined benchmarks, labs must pause training until it passes—no vague pacing, just a hard gate. That would force even open-weight players to play by the same safety rules.