The same answer, far cheaper

In March 2023, getting GPT-4’s score on MMLU, a broad test of general knowledge, cost $37.50 per million tokens. By February 2025, Gemini 2.0 Flash matched that score for $0.18. That is about 200 times cheaper in under two years.

Epoch AI, a research group that tracks these prices, found the same pattern elsewhere. For a hard science test called GPQA Diamond, the cheapest model reaching GPT-4’s level got about 40 times cheaper a year. Across the milestones they studied, the fall ran from 9 to 900 times a year.

The trend is older than GPT-4. In November 2021, a GPT-3-level score on MMLU cost $60 per million tokens. In October 2024, a small model called Gemini 1.5 Flash 8B did it for $0.07, about 850 times less.

Small models catch up

Epoch AI names two well-known reasons for the fall: models getting smaller and hardware getting cheaper to run. Some other causes are hard to see from public information. The first reason is easy to spot: a new small model soon matches an old large one. GPT-4 scored well on a coding test called HumanEval at $37.50 per million tokens in March 2023. By July 2024, Llama 3.1 8B, a small open model, matched it for ten cents.

A small model has fewer weights. To write each token, a chip reads every weight once, so fewer weights means less to read. Each token comes out sooner, and the same chip serves more of them.

Slimmer weights

There is a second trick. Each weight is a number, and a number can be stored with more or fewer digits. The usual format uses 16 bits per weight. Squeezing each one into 4 bits makes the model a quarter of the size.

On one H100 chip, an 8-billion-weight model at 16 bits tops out near 209 tokens a second. At 4 bits the ceiling rises to about 838, four times as many. The cost is a little accuracy.

A current example

Claude Haiku 5.5 shows the trend still running. Released on October 7, 2026, it costs $0.10 per million input tokens and $0.50 per million output tokens for most prompts. The Haiku model before it cost $1 and $5. Anthropic says Haiku 5.5 costs around 75% less to run on average.

Haiku 5.5 also counts the same text as about 30% more tokens, so the saving on a real document is a bit smaller than the price sheet shows.

A cheaper token is only one side of the story. The study from Part 2 still expects AI datacenters at six large firms to use two to two and a half times as much electricity in 2030 as in 2024.

In one sentence: a fixed level of skill gets cheaper very fast, and one big reason is that smaller and slimmer models soon match what only big ones could do.