mlsanityData to 14 Oct 2026

Effectively Controlling Reasoning Models through Thinking Intervention

arXiv 2503.24370 ↗First posted 31 Mar 2025cs.LG cs.AI cs.CLcited by 65

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Reasoning-enhanced large language models (LLMs) explicitly generate intermediate reasoning steps prior to generating final answers, helping the model excel in complex problem-solving. In this paper, we demonstrate that this emerging generation framework offers a unique opportunity for more fine-grained control over model behavior. We propose Thinking Intervention, a novel paradigm designed to explicitly guide the internal reasoning processes of LLMs by strategically inserting or revising specific thinking tokens. We find that the Thinking Intervention paradigm enhances the capabilities of reasoning models across a wide range of tasks, including instruction following on IFEval and Overthinking, instruction hierarchy on SEP, and safety alignment on XSTest and SorryBench. Our results demonstrate that Thinking Intervention significantly outperforms baseline prompting approaches, achieving up to 6.7% accuracy gains in instruction-following scenarios, 15.4% improvements in reasoning about instruction hierarchies, and a 40.0% increase in refusal rates for unsafe prompts using open-source DeepSeek R1 models. Overall, our work opens a promising new research avenue for controlling reasoning LLMs.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 68 of its 82 references that are in the map. First posted on arXiv 31 Mar 2025.

Builds on

68 of 82 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
  2. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  3. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach3 Mar
  4. 2025s1: Simple test-time scaling31 Jan
  5. 2024Token-Budget-Aware LLM Reasoning24 Dec
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug

Other references · 62

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  11. 2023Let's Verify Step by Stepcited by 3,789
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
38 more
  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  7. 2020Learning to summarize from human feedbackcited by 2,588
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  10. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  11. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  12. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  13. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  14. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  15. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  16. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  17. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  18. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  19. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  20. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  21. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  22. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  23. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  24. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  25. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 431
  26. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  27. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
  28. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  29. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 337
  30. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 308
  31. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 297
  32. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 282
  33. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 267
  34. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  35. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 181
  36. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 170
  37. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 170
  38. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 141

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 44; 2026: 21.

65 papers.

Sort:

By area:

  1. 2025Reasoning Models Can Be Effective Without Thinkingcited by 187
  2. 2025A Survey of Context Engineering for Large Language Modelscited by 126
  3. 2025Safety in Large Reasoning Models: A Surveycited by 64
  4. 2025Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RLcited by 62
  5. 2025Defending Against Prompt Injection with DataFiltercited by 35
  6. 2025Defending Against Prompt Injection With a Few DefensiveTokenscited by 33
  7. 2025Qwen Look Again: Guiding Vision-Language Reasoning Models to Re-attention Visual Informationcited by 28
  8. 2025Leaky Thoughts: Large Reasoning Models Are Not Private Thinkerscited by 28
  9. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  10. 2025Measuring the Faithfulness of Thinking Drafts in Large Reasoning Modelscited by 23
  11. 2025Overclocking LLM Reasoning: Monitoring and Controlling Thinking Path Lengths in LLMscited by 22
  12. 2025Not All Tokens Are What You Need In Thinkingcited by 20
38 more
  1. 2025MMATH: A Multilingual Benchmark for Mathematical Reasoningcited by 17
  2. 2025OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for Over-Reasoning Mitigationcited by 17
  3. 2025Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoningcited by 17
  4. 2025Think Clearly: Improving Reasoning via Redundant Token Pruningcited by 17
  5. 2025Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Trainingcited by 16
  6. 2025StreamingThinker: Large Language Models Can Think While Readingcited by 15
  7. 2026Controllable LLM Reasoning via Sparse Autoencoder-Based Steeringcited by 15
  8. 2025When Models Reason in Your Language: Controlling Thinking Language Comes at the Cost of Accuracycited by 14
  9. 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skillscited by 14
  10. 2025MMPB: It's Time for Multi-Modal Personalizationcited by 14
  11. 2025SABER: Switchable and Balanced Training for Efficient LLM Reasoningcited by 12
  12. 2025Retrieval-of-Thought: Efficient Reasoning via Reusing Thoughtscited by 12
  13. 2025Towards Safe Reasoning in Large Reasoning Models via Corrective Interventioncited by 11
  14. 2025Investigating CoT Monitorability in Large Reasoning Modelscited by 9
  15. 2025Mitigating Indirect Prompt Injection via Instruction-Following Intent Analysiscited by 9
  16. 2025Thinking Sparks!: Emergent Attention Heads in Reasoning Models During Post Trainingcited by 8
  17. 2025Tree-of-Reasoning: Towards Complex Medical Diagnosis via Multi-Agent Reasoning with Evidence Treecited by 7
  18. 2025Asynchronous Reasoning: Training-Free Interactive Thinking LLMscited by 7
  19. 2026What Matters For Safety Alignment?cited by 7
  20. 2026Evaluation Awareness in Language Models Has Limited Effect on Behaviourcited by 7
  21. 2026Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden Statescited by 7
  22. 2025Efficient Long CoT Reasoning in Small Language Modelscited by 6
  23. 2025CARE: Decoding Time Safety Alignment via Rollback and Introspection Interventioncited by 6
  24. 2026Breaking the Chain: A Causal Analysis of LLM Faithfulness to Intermediate Structurescited by 6
  25. 2025CyclicReflex: Improving Reasoning Models via Cyclical Reflection Token Schedulingcited by 5
  26. 2025Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Modelscited by 5
  27. 2025Off-Trajectory Reasoning: Can LLMs Collaborate on Reasoning Trajectory?cited by 5
  28. 2025ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Momentscited by 4
  29. 2025Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberationcited by 4
  30. 2026On-Policy Supervised Fine-Tuning for Efficient Reasoningcited by 4
  31. 2026Plans Don't Persist: Why Context Management Is Load Bearing for LLM Agentscited by 4
  32. 2025Injecting External Knowledge into the Reasoning Process Enhances Retrieval-Augmented Generationcited by 3
  33. 2025ThinkPilot: Steering Reasoning Models via Automated Think-prefixes Optimizationcited by 3
  34. 2025Learning to Reason Efficiently with Discounted Reinforcement Learningcited by 3
  35. 2025K2-V2: A 360-Open, Reasoning-Enhanced LLMcited by 3
  36. 2025Does More Inference-Time Compute Really Help Robustness?cited by 2
  37. 2025On the Robustness of Answer Formats in Medical Reasoning Modelscited by 2
  38. 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Abilitycited by 2

50 most cited of 65 All 65 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Model Scaling: Test-Time Intervention for Efficient Deep Reasoning
  2. 2025When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs
  3. 2025ReasonIF: Large Reasoning Models Fail to Follow Instructions During Reasoning
  4. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  5. 2025Understanding Reasoning in Thinking Language Models via Steering Vectors
  6. 2025Thinking About Thinking: Evaluating Reasoning in Post-Trained Language Models
  7. 2025Attentive Reasoning Queries: A Systematic Method for Optimizing Instruction-Following in Large Language Models
  8. 2025Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models