# COMPASS: Finding Where Reasoning Lives in Language Models

arXiv: https://arxiv.org/abs/2610.07469 · HTML: https://mlsanity.com/paper/2610.07469/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 5 Oct 2026.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 18 of 18)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Gemma 4 Technical Report](https://mlsanity.com/paper/2607.02770/) (2026)
- [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://mlsanity.com/paper/2311.01460/) (2023)
- [Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification](https://mlsanity.com/paper/2504.05419/) (2025)
- [Learning without training: The implicit dynamics of in-context learning](https://mlsanity.com/paper/2507.16003/) (2025)
- [HARP: A challenging human-annotated math reasoning benchmark](https://mlsanity.com/paper/2412.08819/) (2024)
- [The First to Know: How Token Distributions Reveal Hidden Knowledge in Large Vision-Language Models?](https://mlsanity.com/paper/2403.09037/) (2024)
- [Fractional Reasoning via Latent Steering Vectors Improves Inference Time Compute](https://mlsanity.com/paper/2506.15882/) (2025)
- [Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models](https://mlsanity.com/paper/2601.08058/) (2026)
- [Code Correctness Is Linearly Decodable from LLM Hidden States Before Generation](https://mlsanity.com/paper/2606.14530/) (2026)
- [FineSteer: A Unified Framework for Fine-Grained Inference-Time Steering in Large Language Models](https://mlsanity.com/paper/2604.15488/) (2026)
- [Knowledge Knows, Verbalization Tells: Disentangling Latent Directions for Mathematical Solvability in LLMs](https://mlsanity.com/paper/2607.05013/) (2026)
- [Decomposing and Steering Functional Metacognition in Large Language Models](https://mlsanity.com/paper/2605.08942/) (2026)
- [Invariant Reasoning Directions in Latent Trajectories of Language Models](https://mlsanity.com/paper/2606.29164/) (2026)

## Abstract (arXiv)

Explicitly eliciting reasoning substantially improves LLM performance. Existing approaches require a predefined characterization of reasoning, whether through CoT prompt design, contrastive CoT directions, or via SAE derived reasoning features. For mathematical reasoning with verifiable answers, we show that a much simpler signal suffices, which is the correctness of the model's own direct answer attempts. This signal yields a latent direction that elicits reasoning. This direction is decodable within the activations of most attention heads, but only a small subset of them can be effectively intervened. We introduce COMPASS, an inference-time steering method that identifies these heads using a logit-space attribution score and steers their activations along the correctness direction, requiring only per-head activation statistics. Across three model families and multiple math benchmarks, COMPASS outperforms the activation-steering baselines we compare against, improves GSM8K accuracy by 16 percentage points on average, and approaches CoT accuracy with 20-70% fewer generated tokens. Interventions transfer without re-fitting to unseen benchmarks, and ablations show that both the correctness direction and the small set of heads carrying it are necessary, with the effect concentrated in remarkably few heads.