# Hype, Sustainability, and the Price of the Bigger-is-Better Paradigm in AI

arXiv: https://arxiv.org/abs/2409.14160 · HTML: https://mlsanity.com/paper/2409.14160/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Sep 2024.

## Built on by (most cited first, 30 of 35)

- [OverThink: Slowdown Attacks on Reasoning LLMs](https://mlsanity.com/paper/2502.02542/) (2025)
- [Table Foundation Models: on knowledge pre-training for tabular learning](https://mlsanity.com/paper/2505.14415/) (2025)
- [Query-Level Uncertainty in Large Language Models](https://mlsanity.com/paper/2506.09669/) (2025)
- [Beyond Max Tokens: Stealthy Resource Amplification via Tool Calling Chains in LLM Agents](https://mlsanity.com/paper/2601.10955/) (2026)
- [Understanding Adam Requires Better Rotation Dependent Assumptions](https://mlsanity.com/paper/2410.19964/) (2024)
- [Ground-Truthing AI Energy Consumption: Validating CodeCarbon Against External Measurements](https://mlsanity.com/paper/2509.22092/) (2025)
- [LoRTA: Low Rank Tensor Adaptation of Large Language Models](https://mlsanity.com/paper/2410.04060/) (2024)
- [Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor](https://mlsanity.com/paper/2506.14652/) (2025)
- [Do Depth-Grown Models Overcome the Curse of Depth? An In-Depth Analysis](https://mlsanity.com/paper/2512.08819/) (2025)
- [Why Companies "Democratise" Artificial Intelligence: The Case of Open Source Software Donations](https://mlsanity.com/paper/2409.17876/) (2024)
- [TensorSocket: Shared Data Loading for Deep Learning Training](https://mlsanity.com/paper/2409.18749/) (2024)
- [The Elephant in the Coreference Room: Resolving Coreference in Full-Length French Fiction Works](https://mlsanity.com/paper/2510.15594/) (2025)
- [Small is Sufficient: Reducing the World AI Energy Consumption Through Model Selection](https://mlsanity.com/paper/2510.01889/) (2025)
- [Large Language Models as Search Engines: Societal Challenges](https://mlsanity.com/paper/2512.08946/) (2025)
- [Limits to AI Growth: The Ecological and Social Consequences of Scaling](https://mlsanity.com/paper/2501.17980/) (2025)
- [Do Chinese models speak Chinese languages?](https://mlsanity.com/paper/2504.00289/) (2025)
- [Imposing AI: Deceptive design patterns against sustainability](https://mlsanity.com/paper/2508.08672/) (2025)
- [The Environmental Impacts of Language Model Training Keep Rising Now is the Time to Catch Impacts on the Rebound](https://mlsanity.com/paper/2510.09022/) (2025)
- [Distilling Lightweight Domain Experts from Large ML Models by Identifying Relevant Subspaces](https://mlsanity.com/paper/2601.05913/) (2026)
- [Lift What You Can: Green Online Learning with Heterogeneous Ensembles](https://mlsanity.com/paper/2509.18962/) (2025)
- [Irresponsible AI: big tech's influence on AI research and associated impacts](https://mlsanity.com/paper/2512.03077/) (2025)
- [Dataset Diversity Metrics and Impact on Classification Models](https://mlsanity.com/paper/2603.15276/) (2026)
- [CogScale: Scalable Benchmark for Sequence Processing](https://mlsanity.com/paper/2605.19758/) (2026)
- [A second-order-like optimizer with adaptive gradient scaling for deep learning](https://mlsanity.com/paper/2410.05871/) (2024)
- [Inteligencia Artificial para la conservaci'on y uso sostenible de la biodiversidad, una visi'on desde Colombia (Artificial Intelligence for conservation and sustainable use of biodiversity, a view from Colombia)](https://mlsanity.com/paper/2503.14543/) (2025)
- [Deprecating Benchmarks: Criteria and Framework](https://mlsanity.com/paper/2507.06434/) (2025)
- [Help the machine to help you: an evaluation in the wild of egocentric data cleaning via skeptical learning](https://mlsanity.com/paper/2510.23635/) (2025)
- [Energy Scaling Laws for Diffusion Models: Quantifying Compute in Image Generation](https://mlsanity.com/paper/2511.17031/) (2025)
- [Energy-Efficient Federated Learning via Adaptive Encoder Freezing for MRI-to-CT Conversion: A Green AI-Guided Research](https://mlsanity.com/paper/2512.03054/) (2025)
- [The Global Landscape of Environmental AI Regulation: From the Cost of Reasoning to a Right to Green AI](https://mlsanity.com/paper/2603.00068/) (2026)

## Abstract (arXiv)

With the growing attention and investment in recent AI approaches such as large language models, the narrative that the larger the AI system the more valuable, powerful and interesting it is is increasingly seen as common sense. But what is this assumption based on, and how are we measuring value, power, and performance? And what are the collateral consequences of this race to ever-increasing scale? Here, we scrutinize the current scaling trends and trade-offs across multiple axes and refute two common assumptions underlying the 'bigger-is-better' AI paradigm: 1) that performance improvements are driven by increased scale, and 2) that all interesting problems addressed by AI require large-scale models. Rather, we argue that this approach is not only fragile scientifically, but comes with undesirable consequences. First, it is not sustainable, as, despite efficiency improvements, its compute demands increase faster than model performance, leading to unreasonable economic requirements and a disproportionate environmental footprint. Second, it implies focusing on certain problems at the expense of others, leaving aside important applications, e.g. health, education, or the climate. Finally, it exacerbates a concentration of power, which centralizes decision-making in the hands of a few actors while threatening to disempower others in the context of shaping both AI research and its applications throughout society.