Latent Trajectory Dynamics in Large Language Models: A Manifold Evolution Framework with Empirical Validation
Not on the map. Why?
Abstract (arXiv)
Understanding how latent representations evolve during generation is a central open problem in large language model interpretability. We introduce Dynamical Manifold Evolution Theory (DMET), a phenomenological framework that models LLM generation as a controlled dynamical system evolving along a trajectory on a low-dimensional semantic manifold. DMET formalizes the structural correspondence between Transformer components and a first-order ODE governed by a semantic potential V, and characterizes trajectory geometry through three falsifiable proxy metrics: state continuity C, attractor clustering quality Q, and topological persistence P, targeting local smoothness, meso-scale basin structure, and global topological organization, respectively. Across six model architectures, four task types, and 1,080 experimental runs, all three metrics consistently predict text quality outcomes -- log-perplexity, grammaticality, and cross-sentence coherence -- after controlling for decoding parameters, with associations surviving Benjamini--Hochberg correction. Ablation and sanity-check experiments confirm that the effects arise from genuine trajectory structure rather than static distributional artefacts. Furthermore, online monitoring of C drives an adaptive decoding controller that reduces perplexity from 48.5 to 14.6 relative to a fixed-parameter baseline, demonstrating that latent dynamics characterization translates directly into actionable generation control.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 38 references that are in the map. First posted on arXiv 24 May 2025.
Builds on
15 of 38 references.
References · 15
- 2017Attention Is All You Needcited by 60,361
- 2023GPT-4 Technical Reportcited by 18,396
- 2018Neural Ordinary Differential Equationscited by 3,779
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
- 2019Augmented Neural ODEscited by 397
- 2020On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselinescited by 255
- 2019Visualizing and Measuring the Geometry of BERTcited by 216
- 2017Latent Space Oddity: on the Curvature of Deep Generative Modelscited by 174
- 2017VAMPnets: Deep learning of molecular kineticscited by 136
- 2018Stable Recurrent Modelscited by 66
3 more
Built on by
Citing papers per year: 2026: 1.
1 paper.