The cost of AI performance is falling at an exceptional rate

Good news!

"... according to a new report from Epoch AI, a research nonprofit. Epoch tracked how much it costs for AI models to achieve given scores on five tests, covering mathematics, hard sciences, and games of skill such as chess puzzles. Across all five tests, Epoch estimates that the minimum cost of reaching a given score has fallen about 47 percent per quarter since 2023, a roughly 13-fold decline each year. According to Epoch’s analysis, that’s about four times faster than the fall in DNA sequencing costs, one of the fastest price declines in history."

"Key takeaways

• AI has gotten cheaper more quickly than any other transformative technology in history. The cost of achieving a given level of AI performance has fallen about 47% per quarter since 2023, or 13× per year. That price drop is four times faster than DNA sequencing, six times faster than compute, 18 times faster than lithium batteries, and (in the century up to 1973) 54 times faster than electricity.
• Coarser evidence suggests that the price of thought has been falling at least this fast since the dawn of commercial LLM inference in November 2021, when OpenAI fully released GPT-3.
• The speed of price drop varies by domain: slower on game-based puzzles, at 39–43% per quarter, faster on math problems, at 50–52% per quarter.
• The cost of a given level of performance often falls fastest right after that level is first achieved, that is, when it is state of the art (SOTA). We see this pattern on three of our five main benchmarks of AI capability. Averaging across all five, cost falls 66% per quarter (75× per year) for performance that has just debuted as SOTA. Two years later, prices fall half as fast, at 32% per quarter (4.7× per year).

..."

Doomslayer: Progress Roundup - by Malcolm Cochran

The plunging price of thought "Over the past three years, the cost of a given level of AI performance has fallen about 47% per quarter, faster than for any other transformative technology in history."

The price of artificial thought may have fallen faster than for any other transformative technology in history