mlsanityData to 7 Oct 2026

Coupled Scaling: A Representational Accessibility Framework for Neural Scaling Laws

arXiv 2609.03533 ↗First posted 3 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We ask when two learning systems trained on the same task under a common resource protocol should share a scaling rate and when their rates should differ. Coupled Scaling answers this through representational accessibility: the task-relevant geometry that a specified architecture-optimization system can reach and how that geometry is acquired as resources grow. In an orthogonal model, unsupported target energy sets the asymptotic floor, while unacquired supported energy sets the finite-budget residual. When acquisition can skip high-value directions, the largest fully acquired prefix no longer determines a unique exponent. For target powers aj asymp j-b, b>1, and prefix log-growth rate 0<ρ≤1, the sharp attainable interval is [ρ(b-1),minb-1,ρ b], and every rate in this interval is realized by a fixed acquisition order on a common support. A rank-window condition identifies when the product law α=ρ(b-1) applies, while a fixed-kernel specialization expresses the same task-system dependence through the task-weighted spectral tail. We then audit released capability trajectories to diagnose measurement and comparison effects and propose a staged test in which independently measured geometry predicts held-out loss, scaling rates, and cross-task reversals.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 39 references that are in the map. First posted on arXiv 3 Sep 2026.

Builds on

33 of 39 references.

References · 33

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2018On Lazy Training in Differentiable Programmingcited by 671
  3. 2017Deep Learning Scaling is Predictable, Empiricallycited by 556
  4. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  5. 2021Explaining Neural Scaling Lawscited by 353
  6. 2024Repeat After Me: Transformers are Better than State Space Models at Copyingcited by 208
  7. 2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
  8. 2023Zoology: Measuring and Improving Recall in Efficient Language Modelscited by 188
  9. 2020Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networkscited by 166
  10. 2023The Quantization Model of Neural Scalingcited by 137
  11. 2024A Dynamical Model of Neural Scaling Lawscited by 105
  12. 2022Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?cited by 103
21 more
  1. 2022Broken Neural Scaling Lawscited by 102
  2. 2022A Solvable Model of Neural Scaling Lawscited by 101
  3. 2022Data Scaling Laws in NMT: The Effect of Noise and Architecturecited by 55
  4. 2024How Feature Learning Can Improve Neural Scaling Lawscited by 55
  5. 2025Superposition Yields Robust Neural Scalingcited by 30
  6. 2025Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regimecited by 18
  7. 2026SMELT: Scaling Laws for Compute-Matched MoE Looped Transformerscited by 12
  8. 2026What do Language Models Learn and When? The Implicit Curriculum Hypothesiscited by 8
  9. 2026Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retentioncited by 8
  10. 2026Optimal scaling laws in learning hierarchical multi-index modelscited by 6
  11. 2025Scaling Laws and Symmetry, Evidence from Neural Force Fieldscited by 4
  12. 2026Towards Robust Scaling Laws for Optimizerscited by 4
  13. 2026A Qualitative Test-Risk Mechanism for Scaling Behavior in Normalized Residual Networkscited by 4
  14. 2026Neural Scaling Universality: If Exponents Are Fixed, Time to Understand Coefficientscited by 4
  15. 2026Spectral Lens: Activation and Gradient Spectra as Diagnostics of LLM Optimizationcited by 3
  16. 2026Same Architecture, Different Capacity: Optimizer-Induced Spectral Scaling Lawscited by 2
  17. 2026Data Scaling as Progressive Coverage of a Predictive Contribution Spectrumcited by 1
  18. 2026On the Optimizer Dependence of Neural Scaling Lawscited by 1
  19. 2026Spectral Reach: Understanding Neural Scaling as Progress into the Spectral Tailcited by 1
  20. 2026Mechanisms of Width Scaling in Normalized Residual Networks: The Effective Alignment Dimensioncited by 1
  21. 2026Scaling Domain Data Repetition in LLM Pretrainingcited by 1

Built on by

Not cited yet.