# REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Model arXiv: https://arxiv.org/abs/2509.22518 ยท HTML: https://mlsanity.com/paper/2509.22518/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. - Builds on 31 of its 50 references that are in the map. - First posted on arXiv 26 Sep 2025. ## Placed in - LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/) ## Builds on (most cited first, 30 of 31) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024) - [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021) - [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025) - [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023) - [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025) - [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021) - [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024) - [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022) - [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023) - [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023) - [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019) - [A Diagram Is Worth A Dozen Images](https://mlsanity.com/paper/1603.07396/) (2016) - [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023) - [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018) - [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021) - [Visual Entailment: A Novel Task for Fine-Grained Image Understanding](https://mlsanity.com/paper/1901.06706/) (2019) - [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023) - [Interpreting CLIP's Image Representation via Text-Based Decomposition](https://mlsanity.com/paper/2310.05916/) (2023) - [Self-Attention Attribution: Interpreting Information Interactions Inside Transformer](https://mlsanity.com/paper/2004.11207/) (2020) - [TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space](https://mlsanity.com/paper/2402.17811/) (2024) - [Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP](https://mlsanity.com/paper/2308.14179/) (2023) - [Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?](https://mlsanity.com/paper/2312.03729/) (2023) - [Understanding Information Storage and Transfer in Multi-modal Large Language Models](https://mlsanity.com/paper/2406.04236/) (2024) - [InternalInspector I2: Robust Confidence Estimation in LLMs through Internal States](https://mlsanity.com/paper/2406.12053/) (2024) - [Probing Language Models for Pre-training Data Detection](https://mlsanity.com/paper/2406.01333/) (2024) - [Uncovering hidden geometry in Transformers via disentangling position and context](https://mlsanity.com/paper/2310.04861/) (2023) - [Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering](https://mlsanity.com/paper/2411.10950/) (2024) - [Hierarchical Deconstruction of LLM Reasoning: A Graph-Based Framework for Analyzing Knowledge Utilization](https://mlsanity.com/paper/2406.19502/) (2024) - [Curved Inference: Concern-Sensitive Geometry in Large Language Model Residual Streams](https://mlsanity.com/paper/2507.21107/) (2025) ## Built on by (most cited first, 5 of 5) - [LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals](https://mlsanity.com/paper/2604.05655/) (2026) - [Echoes within the Reasoning: Stealthy and Effective Watermarking via Chain of Thought](https://mlsanity.com/paper/2605.28890/) (2026) - [Decomposing how prompting steers behavior](https://mlsanity.com/paper/2606.03093/) (2026) - [The Geometry of Thought: How Scale Restructures Reasoning In Large Language Models](https://mlsanity.com/paper/2601.13358/) (2026) - [LoRi: Low-Rank Distillation for Implicit Reasoning](https://mlsanity.com/paper/2606.05315/) (2026) ## Abstract (arXiv) Understanding how Large Language Models (LLMs) perform complex reasoning and their failure mechanisms is a challenge in interpretability research. To provide a measurable geometric analysis perspective, we define the concept of the Reasoning Manifold, a latent low-dimensional geometric structure formed by the internal representations corresponding to all correctly reasoned generations. This structure can be conceptualized as the embodiment of the effective thinking paths that the model has learned to successfully solve a given task. Based on this concept, we build REMA, a framework that explains the origins of failures by quantitatively comparing the spatial relationships of internal model representations corresponding to both erroneous and correct reasoning samples. Specifically, REMA first quantifies the geometric deviation of each erroneous representation by calculating its k-nearest neighbors distance to the approximated manifold formed by correct representations, thereby providing a unified failure signal. It then localizes the divergence points where these deviations first become significant by tracking this deviation metric across the model's layers and comparing it against a baseline of internal fluctuations from correct representations, thus identifying where the reasoning chain begins to go off-track. Our extensive experiments on diverse language and multimodal models and tasks demonstrate the low-dimensional nature of the reasoning manifold and the high separability between erroneous and correct reasoning representations. The results also validate the effectiveness of the REMA framework in analyzing the origins of reasoning failures. This research connects abstract reasoning failures to measurable geometric deviations in representations, providing new avenues for in-depth understanding and diagnosis of the internal computational processes of black-box models.