# Transformer Dynamics: A neuroscientific approach to interpretability of large language models

arXiv: https://arxiv.org/abs/2502.12131 · HTML: https://mlsanity.com/paper/2502.12131/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 17 Feb 2025.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 9 of 9)

- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Explaining Neural Scaling Laws](https://mlsanity.com/paper/2102.06701/) (2021)
- [How to use and interpret activation patching](https://mlsanity.com/paper/2404.15255/) (2024)
- [Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View](https://mlsanity.com/paper/1906.02762/) (2019)
- [What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation](https://mlsanity.com/paper/2404.07129/) (2024)
- [Interpreting Attention Layer Outputs with Sparse Autoencoders](https://mlsanity.com/paper/2406.17759/) (2024)
- [Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics](https://mlsanity.com/paper/1906.10720/) (2019)
- [Relational Composition in Neural Networks: A Survey and Call to Action](https://mlsanity.com/paper/2407.14662/) (2024)
- [Residual Stream Analysis with Multi-Layer SAEs](https://mlsanity.com/paper/2409.04185/) (2024)

## Built on by (most cited first, 11 of 11)

- [Block-Recurrent Dynamics in Vision Transformers](https://mlsanity.com/paper/2512.19941/) (2025)
- [Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning](https://mlsanity.com/paper/2603.01326/) (2026)
- [Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networks](https://mlsanity.com/paper/2511.19265/) (2025)
- [Dynamics of the Transformer Residual Stream: Coupling Spectral Geometry to Network Topology](https://mlsanity.com/paper/2605.14258/) (2026)
- [Explaining Large Language Models with gSMILE](https://mlsanity.com/paper/2505.21657/) (2025)
- [SPD-Faith Bench: Diagnosing and Improving Faithfulness in Chain-of-Thought for Multimodal Large Language Models](https://mlsanity.com/paper/2602.07833/) (2026)
- [Identity as Attractor: Geometric Evidence for Persistent Agent Architecture in LLM Activation Space](https://mlsanity.com/paper/2604.12016/) (2026)
- [Concept Attractors in LLMs and their Applications](https://mlsanity.com/paper/2601.11575/) (2025)
- [What Neuroscience Can Teach AI About Learning in Continuously Changing Environments](https://mlsanity.com/paper/2507.02103/) (2025)
- [Reasoning Errors Have a Region and a Direction in the Residual-Stream Trajectory of LLMs](https://mlsanity.com/paper/2608.05660/) (2026)
- [The Depth Flow of Token Representations Is Nonlinear and Does Not Descend Its Own Density](https://mlsanity.com/paper/2608.29706/) (2026)

## Abstract (arXiv)

As artificial intelligence models have exploded in scale and capability, understanding of their internal mechanisms remains a critical challenge. Inspired by the success of dynamical systems approaches in neuroscience, here we propose a novel framework for studying computations in deep learning systems. We focus on the residual stream (RS) in transformer models, conceptualizing it as a dynamical system evolving across layers. We find that activations of individual RS units exhibit strong continuity across layers, despite the RS being a non-privileged basis. Activations in the RS accelerate and grow denser over layers, while individual units trace unstable periodic orbits. In reduced-dimensional spaces, the RS follows a curved trajectory with attractor-like dynamics in the lower layers. These insights bridge dynamical systems theory and mechanistic interpretability, establishing a foundation for a "neuroscience of AI" that combines theoretical rigor with large-scale data analysis to advance our understanding of modern neural networks.