mlsanityData to 14 Oct 2026

AlphaZero Neural Scaling and Zipf's Law: a Tale of Board Games and Power Laws

arXiv 2412.11979 ↗First posted 16 Dec 2024cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Neural scaling laws are observed in a range of domains, to date with no universal understanding of why they occur. Recent theories suggest that loss power laws arise from Zipf's law, a power law observed in domains like natural language. One theory suggests that language scaling laws emerge when Zipf-distributed task quanta are learned in descending order of frequency. In this paper we examine power-law scaling in AlphaZero, a reinforcement learning algorithm, using a model of language-model scaling. We find that game states in training and inference data scale with Zipf's law, which is known to arise from the tree structure of the environment, and examine the correlation between scaling-law and Zipf's-law exponents. In agreement with the quanta scaling model, we find that agents optimize state loss in descending order of frequency, even though this order scales inversely with modelling complexity. We also find that inverse scaling, the failure of models to improve with size, is correlated with unusual Zipf curves where end-game states are among the most frequent states. We show evidence that larger models shift their focus to these less-important states, sacrificing their understanding of important early-game states.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 36 of its 58 references that are in the map. First posted on arXiv 16 Dec 2024.

Builds on

36 of 58 references.

References · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2020Scaling Laws for Neural Language Modelscited by 6,586
  4. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  5. 2022Emergent Abilities of Large Language Modelscited by 2,512
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  8. 2021Scaling Vision Transformerscited by 999
  9. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  10. 2020Scaling Laws for Autoregressive Generative Modelingcited by 522
  11. 2021Explaining Neural Scaling Lawscited by 363
  12. 2023Inverse Scaling: When Bigger Isn't Bettercited by 207
24 more
  1. 2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
  2. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 186
  3. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  4. 2024Stop Regressing: Training Value Functions via Classification for Scalable Deep RLcited by 144
  5. 2023The Quantization Model of Neural Scalingcited by 138
  6. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  7. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 130
  8. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 125
  9. 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
  10. 2022A Solvable Model of Neural Scaling Lawscited by 116
  11. 2021Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regimecited by 96
  12. 2021Learning Curve Theorycited by 85
  13. 20244+3 Phases of Compute-Optimal Neural Scaling Lawscited by 68
  14. 2021Scaling Scaling Laws with Board Gamescited by 63
  15. 2020A Neural Scaling Law from the Dimension of the Data Manifoldcited by 52
  16. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 42
  17. 2023Scaling laws for single-agent reinforcement learningcited by 41
  18. 2022Scaling Laws for a Multi-Agent Reinforcement Learning Modelcited by 38
  19. 2023Scaling Laws for Associative Memoriescited by 35
  20. 2024Offline Actor-Critic Reinforcement Learning Scales to Large Modelscited by 30
  21. 2022Error Scaling Laws for Kernel Classification under Source and Capacity Conditionscited by 17
  22. 2024A Resource Model For Neural Scaling Lawcited by 9
  23. 2022Evaluation Beyond Task Performance: Analyzing Concepts in AlphaZero in Hexcited by 8
  24. 2021Investment vs. reward in a competitive knapsack problemcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 7.

7 papers.

  1. 2025How Do Large Language Monkeys Get Their Power (Laws)?cited by 43
  2. 2025From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scalingcited by 5
  3. 2025From Zipf's Law to Neural Scaling through Heaps' Law and Hilberg's Hypothesiscited by 3
  4. 2025SkewRoute: Training-Free LLM Routing for Knowledge Graph Retrieval-Augmented Generation via Score Skewness of Retrieved Contextcited by 1
  5. 2025Complexity Scaling Laws for Neural Models using Combinatorial Optimizationcited by 1
  6. 2025Small transformer architectures for task switchingcited by 1
  7. 2025From generative AI to the brain: five takeawayscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning
  2. 2024Observational Scaling Laws and the Predictability of Language Model Performance
  3. 2025How to Upscale Neural Networks with Scaling Law? A Survey and Practical Guidelines
  4. 2023Scaling Laws for Imitation Learning in Single-Agent Games
  5. 2025The Art of Scaling Reinforcement Learning Compute for LLMs
  6. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
  7. 2026SLDBench v2: Active Scaling Law Discovery
  8. 2025Superposition Yields Robust Neural Scaling