Blog

The chess Elo analogy: AI progress looks slow right until it doesn't

There is one chart I keep coming back to when I think about AI progress. It's not about AI. It's about chess.

For about forty years, the best chess engines gained roughly 40 Elo points a year, while the best humans gained about 2. AI Impacts looked at the SSDF rating list from 1984 to 2018 and found a plain linear trend, with no jump bigger than a few years. A straight line.

But the effect of that line was not straight at all.

In 1997 Deep Blue beat Kasparov, and people still argued about whether the match conditions favored the machine. In 2002 and 2003 three human–computer matches ended in draws. Humans could still fight. Then in 2005 Hydra beat Michael Adams, world number seven, with five wins and one draw in six games. In 2006 Deep Fritz beat world champion Kramnik 4–2. That was the last major human–computer match. Since then engines on ordinary hardware, including phones, beat even the strongest players. Top engines are rated above 3000 Elo; the human record is still under 2900.

So, about ten years: from "humans sometimes win" to "why would we even play". The engine curve never bent. The outcome flipped because a straight line crossed the human range.

I think AI's economic impact may work the same way. Progress looks roughly linear and the world looks roughly normal — not because the tech is stuck, but because models are still climbing through the human range. Once they cross it in some domain, the result won't be "a bit better than last year". It will be "humans stop competing".

Two caveats I can't dismiss. The curve could flatten right before the crossing, and in some domains we may be close to that point already. And the analogy works best for goals you can write down: lower the loss, pass the benchmark. Open-ended work, where nobody can write the objective, may be a different curve.

I keep thinking about the 2003–2006 gap. Three years of "it's a draw", and then it was over. If you were betting on chess as a human skill in 2003, the straight line was right there on the chart. It just didn't feel like a warning yet.

References