mlsanityData to 14 Oct 2026

Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering

arXiv 2503.11314 ↗First posted 14 Mar 2025cs.CLcited by 36

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Recent advancements in long chain-of-thoughts(long CoTs) have significantly improved the reasoning capabilities of large language models(LLMs). Existing work finds that the capability of long CoT reasoning can be efficiently elicited by tuning on only a few examples and can easily transfer to other tasks. This motivates us to investigate whether long CoT reasoning is a general capability for LLMs. In this work, we conduct an empirical analysis for this question from the perspective of representation. We find that LLMs do encode long CoT reasoning as a general capability, with a clear distinction from vanilla CoTs. Furthermore, domain-specific representations are also required for the effective transfer of long CoT reasoning. Inspired by these findings, we propose GLoRE, a novel representation engineering method to unleash the general long CoT reasoning capabilities of LLMs. Extensive experiments demonstrate the effectiveness and efficiency of GLoRE in both in-domain and cross-domain scenarios.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 11 of its 59 references that are in the map. First posted on arXiv 14 Mar 2025.

Builds on

11 of 59 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025BOLT: Bootstrap Long Chain-of-Thought in Language Models without Distillation6 Feb
  2. 2025RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?20 Jan

Other references · 9

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  2. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  3. 2025LIMO: Less is More for Reasoningcited by 498
  4. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 433
  5. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
  6. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  7. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  8. 2025Virgo: A Preliminary Exploration on Reproducing o1-like MLLMcited by 68
  9. 2025BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoningcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 26; 2026: 10.

36 papers.

Sort:

By area:

  1. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
  2. 2025AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraintcited by 49
  3. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 37
  4. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  5. 2025Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilitiescited by 20
  6. 2025Towards High Data Efficiency in Reinforcement Learning with Verifiable Rewardcited by 19
  7. 2025Rethinking Sample Polarity in Reinforcement Learning with Verifiable Rewardscited by 19
  8. 2025Explainable Chain-of-Thought Reasoning: An Empirical Analysis on State-Aware Reasoning Dynamicscited by 16
  9. 2025Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thoughtcited by 15
  10. 2025Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Modelscited by 13
  11. 2025Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Modelscited by 13
  12. 2025On Reasoning Strength Planning in Large Reasoning Modelscited by 13
24 more
  1. 2025SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimizationcited by 10
  2. 2025MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steeringcited by 10
  3. 2025Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineeringcited by 8
  4. 2026Spectral Attention Steering for Prompt Highlightingcited by 8
  5. 2025Enhancing Cross-task Transfer of Large Language Models via Activation Steeringcited by 6
  6. 2025RCP-Merging: Merging Long Chain-of-Thought Models with Domain-Specific Models by Considering Reasoning Capability as Priorcited by 5
  7. 2025Path Drift in Large Reasoning Models:How First-Person Commitments Override Safetycited by 5
  8. 2025Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generationcited by 4
  9. 2025Logit Arithmetic Elicits Long Reasoning Capabilities Without Trainingcited by 4
  10. 2026S3-CoT: Self-Sampled Succinct Reasoning Enables Efficient Chain-of-Thought LLMscited by 4
  11. 2025ThoughtProbe: Classifier-Guided LLM Thought Space Exploration via Probing Representationscited by 3
  12. 2026ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrievalcited by 3
  13. 2026Interpreting and Controlling LLM Reasoning through Integrated Policy Gradientcited by 3
  14. 2025Reinforced Informativeness Optimization for Long-Form Retrieval-Augmented Generationcited by 2
  15. 2025Logit Arithmetic Elicits Long Reasoning Capabilities Without Trainingcited by 2
  16. 2025Eliciting Chain-of-Thought in Base LLMs via Gradient-Based Representation Optimizationcited by 2
  17. 2026The Tell-Tale Norm: ℓ2 Magnitude as a Signal for Reasoning Dynamics in Large Language Modelscited by 2
  18. 2025Amplify Adjacent Token Differences: Enhancing Long Chain-of-Thought Reasoning with Shift-FFNcited by 1
  19. 2025Bias-Restrained Prefix Representation Finetuning for Mathematical Reasoningcited by 1
  20. 2026Palette: A Modular, Controllable, and Efficient Framework for On-demand Authorized Safety Alignment Relaxation in LLMscited by 1
  21. 2025Enhancing Supervised Composed Image Retrieval via Reasoning-Augmented Representation Engineeringcited by 0
  22. 2026Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidancecited by 0
  23. 2026Large-scale factor analysis shows machine intelligence is only partially interpretablecited by 0
  24. 2026SelectTTRL: Representation-Guided Data Selection for Efficient Test-Time Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
  2. 2025Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation
  3. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
  4. 2025Long Chain-of-Thought Reasoning Across Languages
  5. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!
  6. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance
  7. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
  8. 2025Efficient Long CoT Reasoning in Small Language Models