# One Spectrum, Two Resources: Data-Memory Scaling in Autoregressive Prediction

arXiv: https://arxiv.org/abs/2609.13500 · HTML: https://mlsanity.com/paper/2609.13500/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2026.

## Abstract (arXiv)

How much learned memory is needed to benefit from more data? We show that the two resources are governed by one predictive-energy spectrum in a positive-entropy autoregressive retrieval source. Each coordinate contributes its query probability times the squared radius of its unknown logit. Writing μ for the resulting energy spectrum, we prove the minimax law mathfrak R^*rm value(n,B)asympR Phiμ(n-1)+Phiμ(τB), Phiμ(t)=intminx,t,μ(mathrm dx), for n prediction blocks and a learned state with at most 2B values. Data set the resolution 1/n; memory sets the level τB reached by optimal bit allocation. The complete curve also recovers the positive spectrum. Energy-dimension pairing is essential: two causal sources with identical block-energy and block-dimension marginals have different data and memory exponents. A masked query-key attention head learns the route and values, realizing the law with explicit routing, format, and arithmetic errors. Further results give exponent-adaptive allocation, finite-precision realization, and compute-precision laws under two-sided arithmetic assumptions. Experiments recover the data-memory collapse and coupling exponents, explain the routing and allocation mechanisms, and examine weight-only quantization across six pretrained-model scales.