# Thought Manipulation: External Thought Can Be Efficient for Large Reasoning Models

arXiv: https://arxiv.org/abs/2504.13626 · HTML: https://mlsanity.com/paper/2504.13626/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 18 Apr 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 30)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs](https://mlsanity.com/paper/2412.21187/) (2024)
- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://mlsanity.com/paper/2503.16419/) (2025)
- [DeepInception: Hypnotize Large Language Model to Be Jailbreaker](https://mlsanity.com/paper/2311.03191/) (2023)
- [Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models](https://mlsanity.com/paper/2501.09686/) (2025)
- [TokenSkip: Controllable Chain-of-Thought Compression in LLMs](https://mlsanity.com/paper/2502.12067/) (2025)
- [Token-Budget-Aware LLM Reasoning](https://mlsanity.com/paper/2412.18547/) (2024)
- [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://mlsanity.com/paper/2405.14838/) (2024)
- [Jailbreak Attacks and Defenses Against Large Language Models: A Survey](https://mlsanity.com/paper/2407.04295/) (2024)
- [Compressed Chain of Thought: Efficient Reasoning Through Dense Representations](https://mlsanity.com/paper/2412.13171/) (2024)
- [Reasoning Models Can Be Effective Without Thinking](https://mlsanity.com/paper/2504.09858/) (2025)
- [CoT-Valve: Length-Compressible Chain-of-Thought Tuning](https://mlsanity.com/paper/2502.09601/) (2025)
- [Multi-Step Reasoning with Large Language Models, a Survey](https://mlsanity.com/paper/2407.11511/) (2024)
- [Can Language Models Learn to Skip Steps?](https://mlsanity.com/paper/2411.01855/) (2024)
- [All Roads Lead to Likelihood: The Value of Reinforcement Learning in Fine-Tuning](https://mlsanity.com/paper/2503.01067/) (2025)
- [Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?](https://mlsanity.com/paper/2504.06514/) (2025)
- [Break the Chain: Large Language Models Can be Shortcut Reasoners](https://mlsanity.com/paper/2406.06580/) (2024)
- [Imitating Language via Scalable Inverse Reinforcement Learning](https://mlsanity.com/paper/2409.01369/) (2024)
- [Quantized Delta Weight Is Safety Keeper](https://mlsanity.com/paper/2411.19530/) (2024)

## Built on by (most cited first, 23 of 23)

- [Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL](https://mlsanity.com/paper/2505.10832/) (2025)
- [R-KV: Redundancy-aware KV Cache Compression for Reasoning Models](https://mlsanity.com/paper/2505.24133/) (2025)
- [Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models](https://mlsanity.com/paper/2505.03469/) (2025)
- [Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models](https://mlsanity.com/paper/2508.02120/) (2025)
- [Mitigating Overthinking in Large Reasoning Models via Manifold Steering](https://mlsanity.com/paper/2505.22411/) (2025)
- [When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoning](https://mlsanity.com/paper/2505.15400/) (2025)
- [VeriThinker: Learning to Verify Makes Reasoning Model Efficient](https://mlsanity.com/paper/2505.17941/) (2025)
- [Does Your Reasoning Model Implicitly Know When to Stop Thinking?](https://mlsanity.com/paper/2602.08354/) (2026)
- [The Price of a Second Thought: On the Evaluation of Reasoning Efficiency in Large Language Models](https://mlsanity.com/paper/2505.22017/) (2025)
- [Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey](https://mlsanity.com/paper/2507.09662/) (2025)
- [Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning](https://mlsanity.com/paper/2506.09853/) (2025)
- [Performative Thinking? The Brittle Correlation Between CoT Length and Problem Complexity](https://mlsanity.com/paper/2509.07339/) (2025)
- [Long or short CoT? Investigating Instance-level Switch of Large Reasoning Models](https://mlsanity.com/paper/2506.04182/) (2025)
- [Auditing Data Membership in Reinforcement Learning With Verifiable Rewards](https://mlsanity.com/paper/2511.14045/) (2025)
- [Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Models](https://mlsanity.com/paper/2601.11340/) (2026)
- [Probing the Trajectories of Reasoning Traces in Large Language Models](https://mlsanity.com/paper/2601.23163/) (2026)
- [Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures](https://mlsanity.com/paper/2601.19928/) (2026)
- [FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoning](https://mlsanity.com/paper/2601.19001/) (2026)
- [Last Layer Logits to Logic: Empowering LLMs with Logic-Consistent Structured Knowledge Reasoning](https://mlsanity.com/paper/2511.07910/) (2025)
- [Distractor Injection Attacks on Large Reasoning Models: Characterization and Defense](https://mlsanity.com/paper/2510.16259/) (2025)
- [Rethinking Model Efficiency: Multi-Agent Inference with Large Models](https://mlsanity.com/paper/2604.04929/) (2026)
- [Revisiting Anthropomorphic Reflection Markers in Large Language Model Reasoning](https://mlsanity.com/paper/2605.28305/) (2026)
- [ADaPT: Token-Level Decoupling for Efficient Large Reasoning Models](https://mlsanity.com/paper/2606.19919/) (2026)

## Abstract (arXiv)

Recent advancements in large reasoning models (LRMs) have demonstrated the effectiveness of scaling test-time computation to enhance reasoning capabilities on various tasks. However, LRMs often suffer from an ``overthinking'' problem, where the model generates excessively redundant reasoning steps with limited performance gains. In this work, we empirically reveal an important characteristic of LRM behaviors that placing external CoTs generated by smaller models between the thinking token (texttt<think> and texttt</think>) can effectively manipulate the model to generate fewer thoughts. Building on this finding, we propose a simple yet efficient pipeline, Method, to enable LRMs to bypass unnecessary intermediate steps, thereby significantly reducing computational costs. We conduct extensive experiments to evaluate the utility and efficiency of Method. For instance, when applied to QwQ-32B on the LiveBench/Code dataset, Method keeps the original performance while reducing output token counts by approximately 30%, with minimal overhead introduced by the CoT generator. Furthermore, we identify two suboptimal modes, blindly following flawed external thoughts and unnecessary rethinking, and show that simple mitigations, such as difficulty-aware fallbacks, can further improve performance. Overall, Method offers a practical, general, and efficient way to optimize LRM inference, making powerful reasoning models more accessible and scalable for real-world applications.