# Latent Trajectory Dynamics in Large Language Models: A Manifold Evolution Framework with Empirical Validation

arXiv: https://arxiv.org/abs/2505.20340 · HTML: https://mlsanity.com/paper/2505.20340/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 24 May 2025.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines](https://mlsanity.com/paper/2006.04884/) (2020)
- [Visualizing and Measuring the Geometry of BERT](https://mlsanity.com/paper/1906.02715/) (2019)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [VAMPnets: Deep learning of molecular kinetics](https://mlsanity.com/paper/1710.06012/) (2017)
- [Stable Recurrent Models](https://mlsanity.com/paper/1805.10369/) (2018)
- [The Origins of Representation Manifolds in Large Language Models](https://mlsanity.com/paper/2505.18235/) (2025)
- [Backward Lens: Projecting Language Model Gradients into the Vocabulary Space](https://mlsanity.com/paper/2402.12865/) (2024)
- [Topological Attention for Time Series Forecasting](https://mlsanity.com/paper/2107.09031/) (2021)

## Built on by (most cited first, 1 of 1)

- [Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning](https://mlsanity.com/paper/2603.01326/) (2026)

## Abstract (arXiv)

Understanding how latent representations evolve during generation is a central open problem in large language model interpretability. We introduce Dynamical Manifold Evolution Theory (DMET), a phenomenological framework that models LLM generation as a controlled dynamical system evolving along a trajectory on a low-dimensional semantic manifold. DMET formalizes the structural correspondence between Transformer components and a first-order ODE governed by a semantic potential V, and characterizes trajectory geometry through three falsifiable proxy metrics: state continuity C, attractor clustering quality Q, and topological persistence P, targeting local smoothness, meso-scale basin structure, and global topological organization, respectively. Across six model architectures, four task types, and 1,080 experimental runs, all three metrics consistently predict text quality outcomes -- log-perplexity, grammaticality, and cross-sentence coherence -- after controlling for decoding parameters, with associations surviving Benjamini--Hochberg correction. Ablation and sanity-check experiments confirm that the effects arise from genuine trajectory structure rather than static distributional artefacts. Furthermore, online monitoring of C drives an adaptive decoding controller that reduces perplexity from 48.5 to 14.6 relative to a fixed-parameter baseline, demonstrating that latent dynamics characterization translates directly into actionable generation control.