The Safety Capture

When 'Please Regulate Us' Means 'Please Regulate Them.'

The loudest calls for AI safety rules and the hardest lobbying against a specific safety bill came from the same address. That's not hypocrisy. It's strategy.

Abstract oil painting: rainbow tones — safety regulation captured by the industry it governs

The Hook

In 2023, OpenAI CEO Sam Altman testified before the U.S. Senate, calling for AI regulation and offering to help write it. It was widely covered as an unusual act of corporate humility: a company asking to be regulated. Then California's SB 1047 reached Governor Newsom's desk, and OpenAI opposed it.

The Question

When the same company calls loudest for regulation in one venue and lobbies hardest against it in another, what kind of regulation is actually being sought?

The Paper Trail

SB 1047 would have required safety testing and certification for large AI models above a compute threshold. When it passed the California legislature and reached Newsom's desk, OpenAI's Chief Strategy Officer Jason Kwon warned it would "lead California's world-class engineers and entrepreneurs to leave the state." Google and Meta also opposed it. Newsom vetoed it September 29, 2024.

The structural logic: a licensing regime for "frontier" models is a barrier priced in compliance lawyers, safety staff, and regulatory relationships. An incumbent with OpenAI's scale absorbs that cost as overhead. A startup or a smaller lab may not. The regulation the incumbents called for in Washington — principled-sounding, vague on specifics — is the same regulation that might never actually constrain them. The regulation California tried to pass — specific, enforceable, with liability — is the one they fought.

The honest exception is worth recording: Anthropic did not simply oppose SB 1047. After amendments, Dario Amodei wrote publicly that the bill's "benefits likely outweigh its costs." A frontier lab conceding a binding state safety law was worth passing is not moat-building. That position was available to every frontier lab. Most chose differently.

The Synthesis

Regulatory capture is not a conspiracy; it's a structure. The companies building the most powerful AI systems are also the ones with the most resources to shape the rules that would govern them. When the outcome is a framework they helped write, in a venue they control, that does not bind them in ways the California bill would have bound them, the capture is in the record — not in speculation.

The Verdict — Did AI do this, or did we?

Human — the selective pursuit of rules that fall hardest on smaller competitors while the strictest binding proposals get lobbied down or vetoed. The honest exception — Anthropic's position on SB 1047 — proves the rest was a choice, not an inevitable industry position.

The Receipts
  • Sam Altman Senate testimony, May 2023 — Congressional transcript
  • OpenAI SB 1047 opposition — reported by TechCrunch, Sep 29 2024 [verify]
  • Newsom veto of SB 1047 — Governor's official veto message, Sep 29 2024 [verify]
  • Dario Amodei statement on SB 1047 — Anthropic blog / public letter
  • Jason Kwon statement — TechCrunch or direct source [verify]