# Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering

arXiv: https://arxiv.org/abs/2503.11314 · HTML: https://mlsanity.com/paper/2503.11314/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 11 of its 59 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)
- Mechanistic interpretability / [Steering and interpreting LLM reasoning](https://mlsanity.com/mechinterp/steering-and-interpreting-llm-reasoning/)

## Builds on (most cited first, 11 of 11)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://mlsanity.com/paper/2501.05366/) (2025)
- [Demystifying Long Chain-of-Thought Reasoning in LLMs](https://mlsanity.com/paper/2502.03373/) (2025)
- [rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking](https://mlsanity.com/paper/2501.04519/) (2025)
- [Layer by Layer: Uncovering Hidden Representations in Language Models](https://mlsanity.com/paper/2502.02013/) (2025)
- [Virgo: A Preliminary Exploration on Reproducing o1-like MLLM](https://mlsanity.com/paper/2501.01904/) (2025)
- [RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?](https://mlsanity.com/paper/2501.11284/) (2025)
- [BOLT: Bootstrap Long Chain-of-Thought in Language Models without Distillation](https://mlsanity.com/paper/2502.03860/) (2025)
- [BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoning](https://mlsanity.com/paper/2501.03226/) (2025)

## Built on by (most cited first, 30 of 36)

- [WebThinker: Empowering Large Reasoning Models with Deep Research Capability](https://mlsanity.com/paper/2504.21776/) (2025)
- [AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraint](https://mlsanity.com/paper/2506.07022/) (2025)
- [Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models](https://mlsanity.com/paper/2508.02120/) (2025)
- [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026)
- [Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilities](https://mlsanity.com/paper/2503.17979/) (2025)
- [Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward](https://mlsanity.com/paper/2509.01321/) (2025)
- [Rethinking Sample Polarity in Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2512.21625/) (2025)
- [Explainable Chain-of-Thought Reasoning: An Empirical Analysis on State-Aware Reasoning Dynamics](https://mlsanity.com/paper/2509.00190/) (2025)
- [Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought](https://mlsanity.com/paper/2510.24941/) (2025)
- [Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2505.15634/) (2025)
- [Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Models](https://mlsanity.com/paper/2505.17697/) (2025)
- [On Reasoning Strength Planning in Large Reasoning Models](https://mlsanity.com/paper/2506.08390/) (2025)
- [SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization](https://mlsanity.com/paper/2505.11166/) (2025)
- [MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering](https://mlsanity.com/paper/2510.04217/) (2025)
- [Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering](https://mlsanity.com/paper/2511.23231/) (2025)
- [Spectral Attention Steering for Prompt Highlighting](https://mlsanity.com/paper/2603.01281/) (2026)
- [Enhancing Cross-task Transfer of Large Language Models via Activation Steering](https://mlsanity.com/paper/2507.13236/) (2025)
- [RCP-Merging: Merging Long Chain-of-Thought Models with Domain-Specific Models by Considering Reasoning Capability as Prior](https://mlsanity.com/paper/2508.03140/) (2025)
- [Path Drift in Large Reasoning Models:How First-Person Commitments Override Safety](https://mlsanity.com/paper/2510.10013/) (2025)
- [Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.11995/) (2025)
- [Logit Arithmetic Elicits Long Reasoning Capabilities Without Training](https://mlsanity.com/paper/2510.09354/) (2025)
- [S3-CoT: Self-Sampled Succinct Reasoning Enables Efficient Chain-of-Thought LLMs](https://mlsanity.com/paper/2602.01982/) (2026)
- [ThoughtProbe: Classifier-Guided LLM Thought Space Exploration via Probing Representations](https://mlsanity.com/paper/2510.27355/) (2025)
- [ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrieval](https://mlsanity.com/paper/2602.01639/) (2026)
- [Interpreting and Controlling LLM Reasoning through Integrated Policy Gradient](https://mlsanity.com/paper/2602.02313/) (2026)
- [Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.20825/) (2025)
- [Logit Arithmetic Elicits Long Reasoning Capabilities Without Training](https://mlsanity.com/paper/2507.12759/) (2025)
- [Eliciting Chain-of-Thought in Base LLMs via Gradient-Based Representation Optimization](https://mlsanity.com/paper/2511.19131/) (2025)
- [The Tell-Tale Norm: ℓ2 Magnitude as a Signal for Reasoning Dynamics in Large Language Models](https://mlsanity.com/paper/2606.06188/) (2026)
- [Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFN](https://mlsanity.com/paper/2505.17153/) (2025)

## Abstract (arXiv)

Recent advancements in long chain-of-thoughts(long CoTs) have significantly improved the reasoning capabilities of large language models(LLMs). Existing work finds that the capability of long CoT reasoning can be efficiently elicited by tuning on only a few examples and can easily transfer to other tasks. This motivates us to investigate whether long CoT reasoning is a general capability for LLMs. In this work, we conduct an empirical analysis for this question from the perspective of representation. We find that LLMs do encode long CoT reasoning as a general capability, with a clear distinction from vanilla CoTs. Furthermore, domain-specific representations are also required for the effective transfer of long CoT reasoning. Inspired by these findings, we propose GLoRE, a novel representation engineering method to unleash the general long CoT reasoning capabilities of LLMs. Extensive experiments demonstrate the effectiveness and efficiency of GLoRE in both in-domain and cross-domain scenarios.