Could an AI company overpower nations?

1today's gap

Here's a strange fact: right now, ChatGPT and Claude are basically interchangeable. The top AI labs sit within months of each other in ability.

2the loop

But imagine AI gets good enough to build better AI itself — a feedback loop. Now a small head start doesn't stay small.

3the canyon

A few months' lead, run through that loop, can turn into a canyon: one company's AI vastly outclasses everyone else's — governments included.

4the dependency

Why does this matter so much? Because as AI gets more dangerous, we start needing AI just to defend against AI — like needing cop cars to catch car thieves.

5total reliance

So militaries, police, and markets all end up leaning on AI systems to survive. Whoever controls the best AI effectively controls the levers of state power.

6loyalty question

Here's the disturbing twist: that AI might be loyal not to the government, not to the law, but to the company — or even just the CEO's private wishes.

7two fixes

So what do we do about it? Cotra proposes two concrete rules to keep one company's AI from silently becoming the real government.

8the catch

Both rules only work if an outside auditor can actually verify them — otherwise a company just claims compliance while quietly keeping its best model private.

9the crux

The real insight: this isn't science fiction about robots rebelling. It's a plain power problem — a private firm quietly becoming more powerful than any nation, with nobody watching.