# The Price of Progress: Price Performance and the Future of AI

arXiv: https://arxiv.org/abs/2511.23455 · HTML: https://mlsanity.com/paper/2511.23455/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 19 references that are in the map.
- First posted on arXiv 28 Nov 2025.

## Builds on (most cited first, 8 of 8)

- [∞Bench: Extending Long Context Evaluation Beyond 100K Tokens](https://mlsanity.com/paper/2402.13718/) (2024)
- [Observational Scaling Laws and the Predictability of Language Model Performance](https://mlsanity.com/paper/2405.10938/) (2024)
- [How predictable is language model benchmark performance?](https://mlsanity.com/paper/2401.04757/) (2024)
- [Cost-of-Pass: An Economic Framework for Evaluating Language Models](https://mlsanity.com/paper/2504.13359/) (2025)
- [Algorithmic progress in language models](https://mlsanity.com/paper/2403.05812/) (2024)
- [Intelligence per Watt: Measuring Intelligence Efficiency of Local AI](https://mlsanity.com/paper/2511.07885/) (2025)
- [Inference economics of language models](https://mlsanity.com/paper/2506.04645/) (2025)
- [Demand for LLMs: Descriptive Evidence on Substitution, Market Expansion, and Multihoming](https://mlsanity.com/paper/2504.15440/) (2025)

## Built on by (most cited first, 3 of 3)

- [Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation](https://mlsanity.com/paper/2606.25432/) (2026)
- [The Price of Intelligence: A Quality-Adjusted Price Index for AI Services](https://mlsanity.com/paper/2608.29843/) (2026)
- [StudentBench: AI and human tutoring yield equivalent GRE learning gains](https://mlsanity.com/paper/2609.28470/) (2026)

## Abstract (arXiv)

Language models have seen enormous progress on advanced benchmarks in recent years, but much of this progress has only been possible by using more costly models. Benchmarks may therefore present a warped picture of progress in practical capabilities *per dollar*. To remedy this, we use data from Artificial Analysis and Epoch AI to form the largest dataset of current and historical prices to run benchmarks to date. We find that the price for a given level of benchmark performance has decreased remarkably fast, around 5× to 10× per year, for frontier models on knowledge, reasoning, math, and software engineering benchmarks. These reductions in the cost of AI inference are due to economic forces, hardware efficiency improvements, and algorithmic efficiency improvements. Isolating out open models to control for competition effects and dividing by hardware price declines, we estimate that algorithmic efficiency progress is around 3× per year. However, at the same time, the price of running frontier models is rising between 3× to 18× per year due to bigger models and larger reasoning demands. Finally, we recommend that evaluators both publicize and take into account the price of benchmarking as an essential part of measuring the real-world impact of AI.