How Big Should a Wireless Foundation Model Be?
Not on the map. Why?
Abstract (arXiv)
Wireless foundation models are rapidly emerging as a key enabler of AI-native communication systems, yet a fundamental question remains unanswered: how large should these models be? We present a principled, physics-grounded answer, showing that the intrinsic dimensionality (dNL, the nonlinear manifold dimension of the channel) acts as the fundamental bottleneck, defining the scaling ceiling once a data-sufficient regime is reached. This dimensionality is not a design choice but a physical constraint: Maxwell's equations, finite scatterers, and antenna aperture inherently constrain wireless propagation environments to a limited number of degrees of freedom -- spanning 5-35 across both real-world OTA measurements and 3GPP-standardized channel models we evaluate -- orders of magnitude below the ~1,000-dimensional semantic space of language. As a consequence, we propose a scaling framework for wireless AI: taking NTN satellite channels as a representative case (dNL ~= 14), scaling gains diminish rapidly beyond ~30 million parameters, entering a stochastic asymptote above 70M where a further 1.6x increase (96M->150M) yields only 0.52 dB. Beyond this ceiling, inference-time adaptation via pilot-aided test-time training (TTT) is far more effective: a compact 12M-parameter model surpasses a static 96M model by 9.9 dB (NMSE, SNR = 20 dB) / 7.6 dB (MCM, SNR = 10 dB) at one-eighth the parameters. With dNL distributions validated across real-world indoor massive MIMO measurements, our scaling laws and TTT gains are demonstrated through NTN satellite simulations, reframing wireless AI design: channel geometry -- not model size -- fundamentally governs the scaling laws of physical-layer wireless AI.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 13 references that are in the map. First posted on arXiv 8 May 2026.
Builds on
8 of 13 references.
References · 8
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
- 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 250
- 2019DeepMIMO: A Generic Deep Learning Dataset for Millimeter Wave and Massive MIMO Applicationscited by 123
- 2024WiFo: Wireless Foundation Model for Channel Predictioncited by 29
- 2022A Distributed Massive MIMO Channel Sounder for "Big CSI Data"-driven Machine Learningcited by 12
- 2026WiMamba: Linear-Scale Wireless Foundation Modelcited by 3
Built on by
Not cited yet.