# States Hidden in Hidden States: Implicit Discrete State Representations Emerge in LLMs' Hidden States

arXiv: https://arxiv.org/abs/2407.11421 · HTML: https://mlsanity.com/paper/2407.11421/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 44 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems](https://mlsanity.com/paper/2402.14008/) (2024)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://mlsanity.com/paper/2404.06395/) (2024)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Zephyr: Direct Distillation of LM Alignment](https://mlsanity.com/paper/2310.16944/) (2023)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [Linearity of Relation Decoding in Transformer Language Models](https://mlsanity.com/paper/2308.09124/) (2023)
- [TheoremQA: A Theorem-driven Question Answering dataset](https://mlsanity.com/paper/2305.12524/) (2023)
- [Implicit Representations of Meaning in Neural Language Models](https://mlsanity.com/paper/2106.00737/) (2021)
- [Do Large Language Models Latently Perform Multi-Hop Reasoning?](https://mlsanity.com/paper/2402.16837/) (2024)
- [Future Lens: Anticipating Subsequent Tokens from a Single Hidden State](https://mlsanity.com/paper/2311.04897/) (2023)
- [Do language models plan ahead for future tokens?](https://mlsanity.com/paper/2404.00859/) (2024)
- [Unified View of Grokking, Double Descent and Emergent Abilities: A Perspective from Circuits Competition](https://mlsanity.com/paper/2402.15175/) (2024)

## Built on by (most cited first, 1 of 1)

- [JAM: Controllable and Responsible Text Generation via Causal Reasoning and Latent Vector Manipulation](https://mlsanity.com/paper/2502.20684/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) exhibit emergent abilities that may reveal aspects of their internal mechanisms. We study one such capability: directly performing extended sequences of calculations without generating chain-of-thought solutions. The strongest models in our evaluation can directly output sums with up to 15 addends, where operands are sampled from 1 to 100. We hypothesize that models form Implicit Discrete State Representations (IDSRs) within their hidden states and use them for internal symbolic calculation. We test for these representations, characterize their formation from layer, digit, and sequence perspectives, and investigate their use in producing answers. We also find that these state representations are far from lossless in current open-source models, contributing to errors in final outputs. Our work offers an initial exploration of LLMs' symbolic calculation abilities and underlying mechanisms. Code and reproducibility artifacts are available at https://github.com/Junhaoo-Chen/IDSR.