mlsanityData to 7 Oct 2026

Thought Manipulation: External Thought Can Be Efficient for Large Reasoning Models

arXiv 2504.13626 ↗First posted 18 Apr 2025cs.CL cs.AIcited by 23

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent advancements in large reasoning models (LRMs) have demonstrated the effectiveness of scaling test-time computation to enhance reasoning capabilities on various tasks. However, LRMs often suffer from an ``overthinking'' problem, where the model generates excessively redundant reasoning steps with limited performance gains. In this work, we empirically reveal an important characteristic of LRM behaviors that placing external CoTs generated by smaller models between the thinking token (texttt<think> and texttt</think>) can effectively manipulate the model to generate fewer thoughts. Building on this finding, we propose a simple yet efficient pipeline, Method, to enable LRMs to bypass unnecessary intermediate steps, thereby significantly reducing computational costs. We conduct extensive experiments to evaluate the utility and efficiency of Method. For instance, when applied to QwQ-32B on the LiveBench/Code dataset, Method keeps the original performance while reducing output token counts by approximately 30%, with minimal overhead introduced by the CoT generator. Furthermore, we identify two suboptimal modes, blindly following flawed external thoughts and unnecessary rethinking, and show that simple mitigations, such as difficulty-aware fallbacks, can further improve performance. Overall, Method offers a practical, general, and efficient way to optimize LRM inference, making powerful reasoning models more accessible and scalable for real-world applications.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 30 of its 44 references that are in the map. First posted on arXiv 18 Apr 2025.

Builds on

30 of 44 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Reasoning Models Can Be Effective Without Thinking14 Apr
  2. 2025Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?9 Apr
  3. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
  4. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs17 Feb
  5. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuning13 Feb
  6. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  7. 2024Token-Budget-Aware LLM Reasoning24 Dec
  8. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representations17 Dec
  9. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug

Other references · 21

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  8. 2023A Survey of Large Language Modelscited by 2,537
  9. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  10. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  11. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  12. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
9 more
  1. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  2. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 225
  3. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  4. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 140
  5. 2024Can Language Models Learn to Skip Steps?cited by 59
  6. 2025All Roads Lead to Likelihood: The Value of Reinforcement Learning in Fine-Tuningcited by 58
  7. 2024Break the Chain: Large Language Models Can be Shortcut Reasonerscited by 32
  8. 2024Imitating Language via Scalable Inverse Reinforcement Learningcited by 21
  9. 2024Quantized Delta Weight Is Safety Keepercited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 15; 2026: 8.

23 papers.

Sort:

By area:

  1. 2025Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RLcited by 59
  2. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Modelscited by 52
  3. 2025Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Modelscited by 37
  4. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 36
  5. 2025Mitigating Overthinking in Large Reasoning Models via Manifold Steeringcited by 32
  6. 2025When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoningcited by 26
  7. 2025VeriThinker: Learning to Verify Makes Reasoning Model Efficientcited by 24
  8. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  9. 2025The Price of a Second Thought: On the Evaluation of Reasoning Efficiency in Large Language Modelscited by 17
  10. 2026Does Your Reasoning Model Implicitly Know When to Stop Thinking?cited by 17
  11. 2025Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoningcited by 16
  12. 2025Performative Thinking? The Brittle Correlation Between CoT Length and Problem Complexitycited by 9
11 more
  1. 2025Long or short CoT? Investigating Instance-level Switch of Large Reasoning Modelscited by 8
  2. 2026Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Modelscited by 7
  3. 2025Auditing Data Membership in Reinforcement Learning With Verifiable Rewardscited by 6
  4. 2026Probing the Trajectories of Reasoning Traces in Large Language Modelscited by 6
  5. 2026FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoningcited by 5
  6. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 5
  7. 2025Last Layer Logits to Logic: Empowering LLMs with Logic-Consistent Structured Knowledge Reasoningcited by 4
  8. 2025Distractor Injection Attacks on Large Reasoning Models: Characterization and Defensecited by 2
  9. 2026Rethinking Model Efficiency: Multi-Agent Inference with Large Modelscited by 0
  10. 2026Revisiting Anthropomorphic Reflection Markers in Large Language Model Reasoningcited by 0
  11. 2026ADaPT: Token-Level Decoupling for Efficient Large Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ThinkSwitcher: When to Think Hard, When to Think Fast
  2. 2025THINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models
  3. 2026Beyond Model Scaling: Test-Time Intervention for Efficient Deep Reasoning
  4. 2025Efficient Inference for Large Reasoning Models: A Survey
  5. 2026When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
  6. 2026CoTJudger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in LRMs
  7. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning
  8. 2026TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning