mlsanityData to 14 Oct 2026

Thinking Isn't an Illusion: Overcoming the Limitations of Reasoning Models via Tool Augmentations

arXiv 2507.17699 ↗First posted 23 Jul 2025cs.AIcited by 9

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Large Reasoning Models (LRMs) have become a central focus in today's large language model (LLM) research, where models are designed to output a step-by-step thinking process before arriving at a final answer to handle complex reasoning tasks. Despite their promise, recent empirical studies (e.g., [Shojaee et al., 2025] from Apple) suggest that this thinking process may not actually enhance reasoning ability, where LLMs without explicit reasoning actually outperform LRMs on tasks with low or high complexity. In this work, we revisit these findings and investigate whether the limitations of LRMs persist when tool augmentations are introduced. We incorporate two types of tools, Python interpreters and scratchpads, and evaluate three representative LLMs and their LRM counterparts on Apple's benchmark reasoning puzzles. Our results show that, with proper tool use, LRMs consistently outperform their non-reasoning counterparts across all levels of task complexity. These findings challenge the recent narrative that reasoning is an illusion and highlight the potential of tool-augmented LRMs for solving complex problems.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 14 of its 44 references that are in the map. First posted on arXiv 23 Jul 2025.

Builds on

14 of 44 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity7 Jun
  2. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
  3. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling10 Feb
  4. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec

Other references · 10

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  7. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  8. 2024DeepSeek-V3 Technical Reportcited by 811
  9. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-trainingcited by 232
  10. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 4.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fast, Slow, and Tool-augmented Thinking for LLMs: A Review
  2. 2025Exploring the System 1 Thinking Capability of Large Reasoning Models
  3. 2025From Proof to Program: Characterizing Tool-Induced Reasoning Hallucinations in Large Language Models
  4. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
  5. 2025Reasoning LLMs are Wandering Solution Explorers
  6. 2025THINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models
  7. 2022Towards Reasoning in Large Language Models: A Survey
  8. 2025Understanding Tool-Integrated Reasoning