# Infrared Organization and Critical Cognitive Field Formation in Transformer Dynamics

arXiv: https://arxiv.org/abs/2607.10923 · HTML: https://mlsanity.com/paper/2607.10923/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 26 references that are in the map.
- First posted on arXiv 12 Jul 2026.

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [DeepSeek LLM: Scaling Open-Source Language Models with Longtermism](https://mlsanity.com/paper/2401.02954/) (2024)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Are Emergent Abilities of Large Language Models a Mirage?](https://mlsanity.com/paper/2304.15004/) (2023)
- [Linear Transformers Are Secretly Fast Weight Programmers](https://mlsanity.com/paper/2102.11174/) (2021)
- [Emergent and Predictable Memorization in Large Language Models](https://mlsanity.com/paper/2304.11158/) (2023)
- [Continual Pre-Training of Large Language Models: How to (re)warm your model?](https://mlsanity.com/paper/2308.04014/) (2023)
- [PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs](https://mlsanity.com/paper/2503.09543/) (2025)
- [Cognitive Field Theory: Memory-Dressed Collective Dynamics of Intelligence](https://mlsanity.com/paper/2601.10221/) (2026)
- [Large Language Models Sometimes Generate Purely Negatively-Reinforced Text](https://mlsanity.com/paper/2306.07567/) (2023)

## Built on by (most cited first, 2 of 2)

- [Infrared Universality of Collective Dynamics across Transformer and State-Space Architectures](https://mlsanity.com/paper/2608.18592/) (2026)
- [Beyond Episodic AI: Cognitive Field Networks for Biologically Inspired Persistent Cognition](https://mlsanity.com/paper/2609.16752/) (2026)

## Abstract (arXiv)

Large language models exhibit remarkable emergent behaviors, yet the physical mechanism governing their collective dynamics remains poorly understood. Cognitive Field Theory predicts that learning reorganizes the collective relaxation spectrum, thereby modifying memory self-energy, long-memory dynamics, and collective susceptibility through the infrared organization of slow relaxation modes. Here we test this framework directly in Transformer dynamics. Using publicly available Pythia language models, we extract relaxation spectra from layer Jacobians throughout training, prompt ensembles, network depth, and model scale, allowing the collective observables of Cognitive Field Theory to be measured quantitatively. The measurements reveal pronounced infrared reorganization of the relaxation spectrum. Learning substantially redistributes spectral weight while preserving a nearly flat but weakly infrared-enhanced time-scale density of states, ( ρ(λ)~λβ, qquad βsimeq-0.1, ) with a corresponding memory kernel exhibiting robust long-memory scaling close to ( K(t)~frac1t. ) The collective observables further reveal a critical formation process: the memory self-energy reaches a transient maximum during early training before relaxing toward a metastable near-critical regime. Prompt-resolved and token-subspace measurements show that distinct local Jacobians recover a common macroscopic TDOS with shared infrared scaling, consistent with infrared fixed-point organization under coarse graining. The reproducibility of this infrared organization across training, prompt ensembles, network depth, and Transformer model scales supports infrared slow-mode organization as a robust collective principle of Transformer dynamics, providing a quantitative experimental realization of the collective observables predicted by Cognitive Field Theory.