# Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction

arXiv: https://arxiv.org/abs/2609.34130 · HTML: https://mlsanity.com/paper/2609.34130/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 78 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability](https://mlsanity.com/paper/2103.00065/) (2021)
- [Limitations of the Empirical Fisher Approximation for Natural Gradient Descent](https://mlsanity.com/paper/1905.12558/) (2019)
- [Feature Learning in Infinite-Width Neural Networks](https://mlsanity.com/paper/2011.14522/) (2020)
- [The Principles of Deep Learning Theory](https://mlsanity.com/paper/2106.10165/) (2021)
- [Stabilizing Transformer Training by Preventing Attention Entropy Collapse](https://mlsanity.com/paper/2303.06296/) (2023)
- [Adaptive Gradient Methods at the Edge of Stability](https://mlsanity.com/paper/2207.14484/) (2022)
- [Neural Networks and Quantum Field Theory](https://mlsanity.com/paper/2008.08601/) (2020)
- [An exact mapping between the Variational Renormalization Group and Deep Learning](https://mlsanity.com/paper/1410.3831/) (2014)
- [On the Role of Attention Masks and LayerNorm in Transformers](https://mlsanity.com/paper/2405.18781/) (2024)
- [Tensor Programs IVb: Adaptive Optimization in the Infinite-Width Limit](https://mlsanity.com/paper/2308.01814/) (2023)
- [Mutual Information, Neural Networks and the Renormalization Group](https://mlsanity.com/paper/1704.06279/) (2017)
- [On the Expressivity Role of LayerNorm in Transformers' Attention](https://mlsanity.com/paper/2305.02582/) (2023)
- [Setting the Record Straight on Transformer Oversmoothing](https://mlsanity.com/paper/2401.04301/) (2024)
- [Towards quantifying information flows: relative entropy in deep neural networks and the renormalization group](https://mlsanity.com/paper/2107.06898/) (2021)
- [CoulGAT: An Experiment on Interpretability of Graph Attention Networks](https://mlsanity.com/paper/1912.08409/) (2019)
- [Power Law Graph Transformer for Machine Translation and Representation Learning](https://mlsanity.com/paper/2107.02039/) (2021)
- [Power-law statistics and universal scaling in the absence of criticality](https://mlsanity.com/paper/1503.08033/) (2015)
- [PLDR-LLM: Large Language Model from Power Law Decoder Representations](https://mlsanity.com/paper/2410.16703/) (2024)
- [PLDR-LLMs Learn A Generalizable Tensor Operator That Can Replace Its Own Deep Neural Net At Inference](https://mlsanity.com/paper/2502.13502/) (2025)
- [PLDR-LLMs Reason At Self-Organized Criticality](https://mlsanity.com/paper/2603.23539/) (2026)
- [Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inference](https://mlsanity.com/paper/2608.10288/) (2026)

## Abstract (arXiv)

This monograph develops a unified account of training and inference in Power Law Decoder Representation language models (PLDR-LLMs). Exact finite work identities decompose changes in the absolute energy of the row-centered learned map into parameter contributions, signed interactions, and numerical observation defects. Positive affine blocking retains restarts at the row-constant face, while the augmented AdamW state supplies the complete dynamical description. Predictive renormalization acts on the complete conditional training law for a single pass over distinct corpus target blocks, retaining optimizer memory, remaining data, schedule, and numerical policy. Autonomous reductions require closure; approximate reductions carry successor and emission errors. Finite-population covariance, matched physical clocks, matrix fluxes, and signed temporal energy connect row dynamics to model-wide observations. Absolute row collapse, relative row concentration, operator stabilization, and predictive accuracy are distinguished. Experiments reveal observer and optimizer dependence, reject the tested autonomous row-state candidates, and support finite conditional prediction and state-specific operator reduction. Independent single-pass families exhibit moving finite fluctuation regions without establishing a thermodynamic critical class. Conditional symmetry, head limits, covariance flows, and readout error budgets specify assumptions needed to transfer scaling laws to inference. The theory separates exact identities, conditional dynamical claims, and finite empirical findings, with proofs, selected formal checks, and compact numerical evidence.