Skip to content
HN On Hacker News ↗

An open letter to Dario: if you mean it, open the weights!

▲ 308 points • 101 comments • by routelastresort • 4w ago • HN discussion ↗

Pangram verdict · v3.3

We believe that this text is a mix of AI and human-written content.

55 %

AI likelihood · overall

Mixed
20% human-written 80% AI-generated
SEGMENTS · HUMAN 1 of 2
SEGMENTS · AI 0 of 2
WORD COUNT 508
PEAK AI % 62% · §1
Analyzed
Sep 12
backend: pangram/v3.3
Segments scanned
2 windows
avg 254 words each
Distribution
20 / 80%
human / AI fraction
Verdict
Mixed
Pangram v3.3

Article text · 508 words · 2 segments analyzed

Human AI-generated
§1 Mixed · 62%

Dear Dario, You published We Must Pace the Frontier today, which committed Anthropic to embedded third-party evaluators and asked governments to require every other frontier company to match. Sam Altman agreed within hours.1 I believe you’re being sincere, so I’m asking you to call for a different law, one that would actually slow things down, even though it would require real sacrifice on your part. Any AI model a company offers to the public has to be released as open weights. This wouldn’t cover every model you train, just the ones you make available to the public. Internal and research models wouldn’t be affected, and governments could still get access to unreleased models. But Anthropic would have to publish Fable’s weights the day it was released, OpenAI would have to do the same with Astra, and so would everyone else. As you know, funding for frontier model development depends on valuations that assume the weights remain proprietary. By changing that assumption, we can reduce the money available for future training runs and slow progress at every lab at once, without a regulator deciding anything. Every regulation you’ve asked for ends in capture There’s no slowing down frontier models by regulation that wouldn’t lead to regulatory capture. Embedded evaluators, compute thresholds, industry coordination with antitrust waivers. All of it gets written with the help of the current frontier labs, because nobody else understands the technical details well enough to write it. Once it’s law, it only gets more complex, because every incident adds a rule and no rule ever gets removed. The big companies can afford the compliance teams and lawyers to keep up, and each new requirement makes it harder for anyone else to catch them. Instead of slowing you down, it protects your position and profits. Why I’m asking you You seem like a good and principled person, and you have a record of giving things up for what you believe. You left OpenAI to start a company with safety built in from the beginning, when the safe career move was to stay.2 You’ve spent years asking for regulation of your own industry and getting called a doomer and a regulatory capturer for it.3 You pushed for export controls that shrink your own market. Anthropic paid $1.5 billion to settle with authors rather than fight to the end.4 And today you committed to slowing down before anyone else did.

§2 Human · 29%

You also had the foresight to make Anthropic a PBC (Public Benefit Corporation), so you can legally put the mission first: The responsible development and maintenance of advanced AI for the long-term benefit of humanity. This is exactly the kind of mission over short-term profit decision the PBC structure exists for. You remain the only person who could propose a law like this and be taken seriously, and the only one likely to actually do it. If you really want to slow down frontier AI, call for a law that makes every publicly available model open weights. There’s nothing in it to game or dodge, even for you.