"Larry Page's 2007 Prediction: AI Would Come From Computation, Not Clever Algorithms"

In February 2007, at the annual meeting of the American Association for the Advancement of Science in San Francisco, Larry Page made a prediction that sounded, at the time, like a provocation. The dominant belief in AI research then centered on elegant algorithms and symbolic reasoning — clever ideas written on whiteboards. Page offered the opposite view: "When AI happens, it's going to be a lot of computation, not so much clever algorithms." He backed the claim with biology. Human DNA, he said, is about 600 megabytes compressed — smaller than any modern operating system — so the brain's algorithms are "probably not that complicated."

The numbers, checked nineteen years later, hold up better than the argument. A haploid human genome is about 3 billion base pairs, and since each base pair can be coded in 2 bits, that is roughly 750 megabytes of raw data; compressed by the genome's own redundancy, Page's 600 megabytes is a defensible figure. There is an even sharper version of the point: one person's deviation from the reference genome compresses to roughly 4 megabytes. And the operating-system comparison is true with one qualifier — true for a full system installation, which runs to tens of gigabytes, though not for the Linux kernel alone, whose compressed source is smaller than the genome. But the inference Page drew from these numbers does not follow, and it was challenged within days of the talk. As the programmer Seth Finkelstein pointed out at the time, a short program can encode enormously complicated behavior; code size is not a measure of algorithm simplicity. There is also harder arithmetic on the other side: a human brain has about 86 billion neurons and on the order of 100 trillion synapses, and a wiring diagram specified synapse by synapse would dwarf 600 megabytes many times over. The genome does not store a blueprint of the brain. It stores a developmental recipe and a set of learning rules, and the brain itself is built by development, experience, and four billion years of evolution.

And yet the prediction came true, by a different road than the one Page argued. In 2019, Richard Sutton's essay "The Bitter Lesson" observed that across seventy years of AI research, the methods that ultimately won were the ones that could absorb more computation — search and learning — while methods that tried to hand-code human knowledge consistently lost. In 2020, Kaplan and colleagues at OpenAI made the claim quantitative: a language model's error falls as a smooth power law in model size, dataset size, and compute, and architectural choices barely matter once size is fixed. DeepMind's Chinchilla work in 2022 refined the recipe rather than refuting it. Nothing fundamental about the transformer changed between GPT-2 and GPT-4; what changed was scale — parameters, tokens, and GPU-hours — and the capabilities that emerged were not explicitly designed.

The deeper correction, and the one worth keeping, is that what matters is not the size of the code but the size of the process that wrote it. The genome is not a small program that boots a brain; it is evolution's summary, the compressed record of an unimaginably long process of trial and error, and what it grows is not a finished mind but a learner. A frontier model's weights are the same kind of object: a training run compressed into parameters, a summary of a process rather than a design. Page was right that scale would win, and wrong about what the smallness of the code proved — the code was never the argument. The process was. That is also why the race is now measured in disclosed training runs: Epoch AI estimates GPT-4 at 2.1 × 10²⁵ FLOP and 2024 frontier runs around 10²⁶, with frontier training compute growing four to five times per year. If that pace holds, a single disclosed run crossing 10²⁷ FLOP by the end of 2027 is the trend's natural next step, and the next test of a prediction made on a San Francisco stage nineteen years ago.