Infrared Organization and Critical Cognitive Field Formation in Transformer Dynamics
Not on the map. Why?
Abstract (arXiv)
Large language models exhibit remarkable emergent behaviors, yet the physical mechanism governing their collective dynamics remains poorly understood. Cognitive Field Theory predicts that learning reorganizes the collective relaxation spectrum, thereby modifying memory self-energy, long-memory dynamics, and collective susceptibility through the infrared organization of slow relaxation modes. Here we test this framework directly in Transformer dynamics. Using publicly available Pythia language models, we extract relaxation spectra from layer Jacobians throughout training, prompt ensembles, network depth, and model scale, allowing the collective observables of Cognitive Field Theory to be measured quantitatively. The measurements reveal pronounced infrared reorganization of the relaxation spectrum. Learning substantially redistributes spectral weight while preserving a nearly flat but weakly infrared-enhanced time-scale density of states, ( ρ(λ)~λβ, qquad βsimeq-0.1, ) with a corresponding memory kernel exhibiting robust long-memory scaling close to ( K(t)~frac1t. ) The collective observables further reveal a critical formation process: the memory self-energy reaches a transient maximum during early training before relaxing toward a metastable near-critical regime. Prompt-resolved and token-subspace measurements show that distinct local Jacobians recover a common macroscopic TDOS with shared infrared scaling, consistent with infrared fixed-point organization under coarse graining. The reproducibility of this infrared organization across training, prompt ensembles, network depth, and Transformer model scales supports infrared slow-mode organization as a robust collective principle of Transformer dynamics, providing a quantitative experimental realization of the collective observables predicted by Cognitive Field Theory.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 18 of its 26 references that are in the map. First posted on arXiv 12 Jul 2026.
Builds on
18 of 26 references.
References · 18
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
- 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
- 2018Universal Transformerscited by 647
- 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
- 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
6 more
- 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
- 2023Emergent and Predictable Memorization in Large Language Modelscited by 148
- 2023Continual Pre-Training of Large Language Models: How to (re)warm your model?cited by 137
- 2025PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runscited by 35
- 2026Cognitive Field Theory: Memory-Dressed Collective Dynamics of Intelligencecited by 3
- 2023Large Language Models Sometimes Generate Purely Negatively-Reinforced Textcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.