mlsanityData to 14 Oct 2026

Reasoning Can Hurt the Inductive Abilities of Large Language Models

arXiv 2505.24225 ↗First posted 30 May 2025cs.CV cs.AI cs.CLcited by 4

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large Language Models (LLMs) have shown remarkable progress across domains, yet their ability to perform inductive reasoning - inferring latent rules from sparse examples - remains limited. It is often assumed that chain-of-thought (CoT) prompting, as used in Large Reasoning Models (LRMs), enhances such reasoning. We investigate this assumption with creating four controlled, diagnostic game-based tasks - chess, Texas Hold'em, dice games, and blackjack - with hidden human-defined rules. We find that CoT reasoning can degrade inductive performance, with LRMs often underperforming their non-reasoning counterparts. To explain this, we present a theoretical framework that reveals how reasoning steps can amplify error through three failure modes: incorrect sub-task decomposition, incorrect sub-task solving, and incorrect final answer summarization. Based on our theoretical and empirical analysis, we introduce structured interventions that adapt CoT generation according to our identified failure types. These interventions improve inductive accuracy without retraining. Our findings suggest that effective (CoT) reasoning depends not only on taking more steps but also on ensuring those steps are well-structured.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 44 references that are in the map. First posted on arXiv 30 May 2025.

Builds on

30 of 44 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025When More is Less: Understanding Chain-of-Thought Length in LLMs11 Feb
  2. 2024Understanding Chain-of-Thought in LLMs through Information Theory18 Nov
  3. 2024A Theoretical Understanding of Chain-of-Thought: Coherent Reasoning and Error-Aware Demonstration21 Oct
  4. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problems20 Feb
  5. 2024The Impact of Reasoning Step Length on Large Language Models10 Jan
  6. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May

Other references · 24

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2024OpenAI o1 System Cardcited by 1,244
  6. 2024DeepSeek-V3 Technical Reportcited by 811
  7. 2024Token-Budget-Aware LLM Reasoningcited by 247
  8. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contextscited by 245
  9. 2020The Next Decade in AI: Four Steps Towards Robust Artificial Intelligencecited by 195
  10. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinkingcited by 99
  11. 2023LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representationscited by 82
  12. 2024Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queriescited by 78
12 more
  1. 2024KOR-Bench: Benchmarking Language Models on Knowledge-Orthogonal Reasoning Taskscited by 70
  2. 2024Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Modelscited by 68
  3. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approachescited by 43
  4. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 36
  5. 2024What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspectivecited by 29
  6. 2024The Role of Deductive and Inductive Reasoning in Large Language Modelscited by 20
  7. 2024MIRAGE: Evaluating and Explaining Inductive Reasoning Process in Language Modelscited by 17
  8. 2024An Examination on the Effectiveness of Divide-and-Conquer Prompting in Large Language Modelscited by 12
  9. 2024Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers?cited by 10
  10. 2024WILT: A Multi-Turn, Memorization-Robust Inductive Logic Benchmark for LLMscited by 9
  11. 2025MIR-Bench: Can Your LLM Recognize Complicated Patterns via Many-Shot In-Context Reasoning?cited by 7
  12. 2024DCR: Divide-and-Conquer Reasoning for Multi-choice Question Answering with LLMscited by 5

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoningcited by 51
  2. 2025GuardVal: Dynamic Large Language Model Jailbreak Evaluation for Comprehensive Safety Testingcited by 2
  3. 2025InvThink: Premortem Reasoning for Safer Language Modelscited by 0
  4. 2025Lessons from Neuroscience for AI: How integrating Actions, Compositional Structure and Episodic Memory could enable Safe, Interpretable and Human-Like AIcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations
  2. 2025Can Large Reasoning Models Improve Accuracy on Mathematical Tasks Using Flawed Thinking?
  3. 2025When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs
  4. 2025The Challenge of Teaching Reasoning to LLMs Without RL or Distillation
  5. 2025Innate Reasoning is Not Enough: In-Context Learning Enhances Reasoning Large Language Models with Less Overthinking
  6. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
  7. 2025Reasoning Capabilities of Large Language Models on Dynamic Tasks
  8. 2023Stress Testing Chain-of-Thought Prompting for Large Language Models