# Using Composition Operators to Linearize LLM Semantic Transformations

arXiv: https://arxiv.org/abs/2609.22143 · HTML: https://mlsanity.com/paper/2609.22143/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 42 references that are in the map.
- First posted on arXiv 25 Aug 2026.

## Builds on (most cited first, 18 of 18)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [TinyLlama: An Open-Source Small Language Model](https://mlsanity.com/paper/2401.02385/) (2024)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model](https://mlsanity.com/paper/2502.02737/) (2025)
- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [Similarity of Neural Network Models: A Survey of Functional and Representational Measures](https://mlsanity.com/paper/2305.06329/) (2023)
- [A Mathematical Guide to Operator Learning](https://mlsanity.com/paper/2312.14688/) (2023)
- [Neural population geometry: An approach for understanding biological and artificial neural networks](https://mlsanity.com/paper/2104.07059/) (2021)
- [Rigorous data-driven computation of spectral properties of Koopman operators for dynamical systems](https://mlsanity.com/paper/2111.14889/) (2021)
- [Residual Dynamic Mode Decomposition: Robust and verified Koopmanism](https://mlsanity.com/paper/2205.09779/) (2022)
- [The Multiverse of Dynamic Mode Decomposition Algorithms](https://mlsanity.com/paper/2312.00137/) (2023)
- [Beyond expectations: Residual Dynamic Mode Decomposition and Variance for Stochastic Dynamical Systems](https://mlsanity.com/paper/2308.10697/) (2023)
- [DisCoCLIP: A Distributional Compositional Tensor Network Encoder for Vision-Language Understanding](https://mlsanity.com/paper/2509.21287/) (2025)
- [Multi-Agent Constraint Factorization Reveals Latent Invariant Solution Structure](https://mlsanity.com/paper/2601.15077/) (2026)
- [Composition Operators, Matrix Representation, and the Finite Section Method: A Theoretical Framework for Maps between Shapes](https://mlsanity.com/paper/1705.00325/) (2017)

## Abstract (arXiv)

Machine learning learns functions: prompt to response, image to caption. What these functions are mathematically remains hard to say. We present a method to approximate these kinds of transformations using techniques from dynamical systems that fall under the umbrella of Koopmanism. We introduce the use of composition operators, which generalize the Koopman operator and, crucially, can map between distinct spaces, motivating the perspective that LLM transformations are rectangular infinite-dimensional operators. This formalism reveals useful structure: under natural assumptions on the prompt and response distributions, the LLM operator is an isometry, and misalignment between learned representations manifests as spectral pollution of its finite sections. We then outline a method of constructing finite-dimensional approximations of an LLM operator, and demonstrate how the singular value spectrum can be used to compare tasks and models.