mlsanityData to 14 Oct 2026

Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction

arXiv 2609.34130 ↗First posted 28 Sep 2026cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

This monograph develops a unified account of training and inference in Power Law Decoder Representation language models (PLDR-LLMs). Exact finite work identities decompose changes in the absolute energy of the row-centered learned map into parameter contributions, signed interactions, and numerical observation defects. Positive affine blocking retains restarts at the row-constant face, while the augmented AdamW state supplies the complete dynamical description. Predictive renormalization acts on the complete conditional training law for a single pass over distinct corpus target blocks, retaining optimizer memory, remaining data, schedule, and numerical policy. Autonomous reductions require closure; approximate reductions carry successor and emission errors. Finite-population covariance, matched physical clocks, matrix fluxes, and signed temporal energy connect row dynamics to model-wide observations. Absolute row collapse, relative row concentration, operator stabilization, and predictive accuracy are distinguished. Experiments reveal observer and optimizer dependence, reject the tested autonomous row-state candidates, and support finite conditional prediction and state-specific operator reduction. Independent single-pass families exhibit moving finite fluctuation regions without establishing a thermodynamic critical class. Conditional symmetry, head limits, covariance flows, and readout error budgets specify assumptions needed to transfer scaling laws to inference. The theory separates exact identities, conditional dynamical claims, and finite empirical findings, with proofs, selected formal checks, and compact numerical evidence.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 78 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

28 of 78 references.

References · 28

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2016Layer Normalizationcited by 5,669
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  6. 2017Decoupled Weight Decay Regularizationcited by 1,831
  7. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  8. 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 427
  9. 2019Limitations of the Empirical Fisher Approximation for Natural Gradient Descentcited by 217
  10. 2020Feature Learning in Infinite-Width Neural Networkscited by 190
  11. 2021The Principles of Deep Learning Theorycited by 173
  12. 2023Stabilizing Transformer Training by Preventing Attention Entropy Collapsecited by 161
16 more
  1. 2022Adaptive Gradient Methods at the Edge of Stabilitycited by 101
  2. 2020Neural Networks and Quantum Field Theorycited by 61
  3. 2014An exact mapping between the Variational Renormalization Group and Deep Learningcited by 56
  4. 2024On the Role of Attention Masks and LayerNorm in Transformerscited by 48
  5. 2023Tensor Programs IVb: Adaptive Optimization in the Infinite-Width Limitcited by 38
  6. 2017Mutual Information, Neural Networks and the Renormalization Groupcited by 33
  7. 2023On the Expressivity Role of LayerNorm in Transformers' Attentioncited by 33
  8. 2024Setting the Record Straight on Transformer Oversmoothingcited by 21
  9. 2021Towards quantifying information flows: relative entropy in deep neural networks and the renormalization groupcited by 14
  10. 2019CoulGAT: An Experiment on Interpretability of Graph Attention Networkscited by 6
  11. 2021Power Law Graph Transformer for Machine Translation and Representation Learningcited by 5
  12. 2015Power-law statistics and universal scaling in the absence of criticalitycited by 4
  13. 2024PLDR-LLM: Large Language Model from Power Law Decoder Representationscited by 4
  14. 2025PLDR-LLMs Learn A Generalizable Tensor Operator That Can Replace Its Own Deep Neural Net At Inferencecited by 3
  15. 2026PLDR-LLMs Reason At Self-Organized Criticalitycited by 2
  16. 2026Power law graph attention: exact generalization of scaled dot-product attention, empirical collapse at inferencecited by 1

Built on by

Not cited yet.