# Language Models Use Trigonometry to Do Addition

arXiv: https://arxiv.org/abs/2502.00873 · HTML: https://mlsanity.com/paper/2502.00873/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Placed in Mathematical reasoning, one of 17 directions in LLM reasoning.
- First posted on arXiv 2 Feb 2025.

## Placed in

- LLM reasoning / [Mathematical reasoning](https://mlsanity.com/lr/mathematical-reasoning/)

## Built on by (most cited first, 30 of 70)

- [LLM Social Simulations Are a Promising Research Method](https://mlsanity.com/paper/2504.02234/) (2025)
- [When Models Manipulate Manifolds: The Geometry of a Counting Task](https://mlsanity.com/paper/2601.04480/) (2026)
- [The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?](https://mlsanity.com/paper/2507.08802/) (2025)
- [Do Sparse Autoencoders Capture Concept Manifolds?](https://mlsanity.com/paper/2604.28119/) (2026)
- [Towards Understanding Distilled Reasoning Models: A Representational Approach](https://mlsanity.com/paper/2503.03730/) (2025)
- [Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior](https://mlsanity.com/paper/2605.05115/) (2026)
- [FoNE: Precise Single-Token Number Embeddings via Fourier Features](https://mlsanity.com/paper/2502.09741/) (2025)
- [The Geometry of Reasoning: Flowing Logics in Representation Space](https://mlsanity.com/paper/2510.09782/) (2025)
- [Probing for Arithmetic Errors in Language Models](https://mlsanity.com/paper/2507.12379/) (2025)
- [The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs](https://mlsanity.com/paper/2502.19981/) (2025)
- [Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks](https://mlsanity.com/paper/2506.06489/) (2025)
- [How Many Features Can a Language Model Store Under the Linear Representation Hypothesis?](https://mlsanity.com/paper/2602.11246/) (2026)
- [Curveball Steering: The Right Direction To Steer Isn't Always Linear](https://mlsanity.com/paper/2603.09313/) (2026)
- [Sequential Group Composition: A Window into the Mechanics of Deep Learning](https://mlsanity.com/paper/2602.03655/) (2026)
- [Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts](https://mlsanity.com/paper/2605.01148/) (2026)
- [SOM Directions are Better than One: Multi-Directional Refusal Suppression in Language Models](https://mlsanity.com/paper/2511.08379/) (2025)
- [Understanding In-context Learning of Addition via Activation Subspaces](https://mlsanity.com/paper/2505.05145/) (2025)
- [Mechanistic Indicators of Understanding in Large Language Models](https://mlsanity.com/paper/2507.08017/) (2025)
- [Why Can't Transformers Learn Multiplication? Reverse-Engineering Reveals Long-Range Dependency Pitfalls](https://mlsanity.com/paper/2510.00184/) (2025)
- [LLMs Process Lists With General Filter Heads](https://mlsanity.com/paper/2510.26784/) (2025)
- [Transformers learn factored representations](https://mlsanity.com/paper/2602.02385/) (2026)
- [Layerwise Recall and the Geometry of Interwoven Knowledge in LLMs](https://mlsanity.com/paper/2502.10871/) (2025)
- [Modular Arithmetic: Language Models Solve Math Digit by Digit](https://mlsanity.com/paper/2508.02513/) (2025)
- [Establishing Construct Validity in LLM Capability Benchmarks Requires Nomological Networks](https://mlsanity.com/paper/2603.15121/) (2026)
- [Convergent Evolution: How Different Language Models Learn Similar Number Representations](https://mlsanity.com/paper/2604.20817/) (2026)
- [In-Context Algebra](https://mlsanity.com/paper/2512.16902/) (2025)
- [Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds](https://mlsanity.com/paper/2606.25234/) (2026)
- [Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit](https://mlsanity.com/paper/2506.06607/) (2025)
- [Addition in Four Movements: Mapping Layer-wise Information Trajectories in LLMs](https://mlsanity.com/paper/2506.07824/) (2025)
- [Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers](https://mlsanity.com/paper/2506.08966/) (2025)

## Abstract (arXiv)

Mathematical reasoning is an increasingly important indicator of large language model (LLM) capabilities, yet we lack understanding of how LLMs process even simple mathematical tasks. To address this, we reverse engineer how three mid-sized LLMs compute addition. We first discover that numbers are represented in these LLMs as a generalized helix, which is strongly causally implicated for the tasks of addition and subtraction, and is also causally relevant for integer division, multiplication, and modular arithmetic. We then propose that LLMs compute addition by manipulating this generalized helix using the "Clock" algorithm: to solve a+b, the helices for a and b are manipulated to produce the a+b answer helix which is then read out to model logits. We model influential MLP outputs, attention head outputs, and even individual neuron preactivations with these helices and verify our understanding with causal interventions. By demonstrating that LLMs represent numbers on a helix and manipulate this helix to perform addition, we present the first representation-level explanation of an LLM's mathematical capability.