# Effectively Controlling Reasoning Models through Thinking Intervention

arXiv: https://arxiv.org/abs/2503.24370 · HTML: https://mlsanity.com/paper/2503.24370/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 68 of its 82 references that are in the map.
- First posted on arXiv 31 Mar 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 68)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Instruction-Following Evaluation for Large Language Models](https://mlsanity.com/paper/2311.07911/) (2023)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)

## Built on by (most cited first, 30 of 65)

- [Reasoning Models Can Be Effective Without Thinking](https://mlsanity.com/paper/2504.09858/) (2025)
- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [Safety in Large Reasoning Models: A Survey](https://mlsanity.com/paper/2504.17704/) (2025)
- [Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL](https://mlsanity.com/paper/2505.10832/) (2025)
- [Defending Against Prompt Injection with DataFilter](https://mlsanity.com/paper/2510.19207/) (2025)
- [Defending Against Prompt Injection With a Few DefensiveTokens](https://mlsanity.com/paper/2507.07974/) (2025)
- [Qwen Look Again: Guiding Vision-Language Reasoning Models to Re-attention Visual Information](https://mlsanity.com/paper/2505.23558/) (2025)
- [Leaky Thoughts: Large Reasoning Models Are Not Private Thinkers](https://mlsanity.com/paper/2506.15674/) (2025)
- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [Measuring the Faithfulness of Thinking Drafts in Large Reasoning Models](https://mlsanity.com/paper/2505.13774/) (2025)
- [Overclocking LLM Reasoning: Monitoring and Controlling Thinking Path Lengths in LLMs](https://mlsanity.com/paper/2506.07240/) (2025)
- [Not All Tokens Are What You Need In Thinking](https://mlsanity.com/paper/2505.17827/) (2025)
- [MMATH: A Multilingual Benchmark for Mathematical Reasoning](https://mlsanity.com/paper/2505.19126/) (2025)
- [OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for Over-Reasoning Mitigation](https://mlsanity.com/paper/2506.02397/) (2025)
- [Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning](https://mlsanity.com/paper/2506.09853/) (2025)
- [Think Clearly: Improving Reasoning via Redundant Token Pruning](https://mlsanity.com/paper/2507.08806/) (2025)
- [Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training](https://mlsanity.com/paper/2505.14681/) (2025)
- [StreamingThinker: Large Language Models Can Think While Reading](https://mlsanity.com/paper/2510.17238/) (2025)
- [Controllable LLM Reasoning via Sparse Autoencoder-Based Steering](https://mlsanity.com/paper/2601.03595/) (2026)
- [When Models Reason in Your Language: Controlling Thinking Language Comes at the Cost of Accuracy](https://mlsanity.com/paper/2505.22888/) (2025)
- [Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skills](https://mlsanity.com/paper/2506.12963/) (2025)
- [MMPB: It's Time for Multi-Modal Personalization](https://mlsanity.com/paper/2509.22820/) (2025)
- [SABER: Switchable and Balanced Training for Efficient LLM Reasoning](https://mlsanity.com/paper/2508.10026/) (2025)
- [Retrieval-of-Thought: Efficient Reasoning via Reusing Thoughts](https://mlsanity.com/paper/2509.21743/) (2025)
- [Towards Safe Reasoning in Large Reasoning Models via Corrective Intervention](https://mlsanity.com/paper/2509.24393/) (2025)
- [Investigating CoT Monitorability in Large Reasoning Models](https://mlsanity.com/paper/2511.08525/) (2025)
- [Mitigating Indirect Prompt Injection via Instruction-Following Intent Analysis](https://mlsanity.com/paper/2512.00966/) (2025)
- [Thinking Sparks!: Emergent Attention Heads in Reasoning Models During Post Training](https://mlsanity.com/paper/2509.25758/) (2025)
- [Tree-of-Reasoning: Towards Complex Medical Diagnosis via Multi-Agent Reasoning with Evidence Tree](https://mlsanity.com/paper/2508.03038/) (2025)
- [Asynchronous Reasoning: Training-Free Interactive Thinking LLMs](https://mlsanity.com/paper/2512.10931/) (2025)

## Abstract (arXiv)

Reasoning-enhanced large language models (LLMs) explicitly generate intermediate reasoning steps prior to generating final answers, helping the model excel in complex problem-solving. In this paper, we demonstrate that this emerging generation framework offers a unique opportunity for more fine-grained control over model behavior. We propose Thinking Intervention, a novel paradigm designed to explicitly guide the internal reasoning processes of LLMs by strategically inserting or revising specific thinking tokens. We find that the Thinking Intervention paradigm enhances the capabilities of reasoning models across a wide range of tasks, including instruction following on IFEval and Overthinking, instruction hierarchy on SEP, and safety alignment on XSTest and SorryBench. Our results demonstrate that Thinking Intervention significantly outperforms baseline prompting approaches, achieving up to 6.7% accuracy gains in instruction-following scenarios, 15.4% improvements in reasoning about instruction hierarchies, and a 40.0% increase in refusal rates for unsafe prompts using open-source DeepSeek R1 models. Overall, our work opens a promising new research avenue for controlling reasoning LLMs.