mlsanityData to 7 Oct 2026

Break the Chain: Large Language Models Can be Shortcut Reasoners

arXiv 2406.06580 ↗First posted 4 Jun 2024cs.CL cs.AIcited by 32

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Recent advancements in Chain-of-Thought (CoT) reasoning utilize complex modules but are hampered by high token consumption, limited applicability, and challenges in reproducibility. This paper conducts a critical evaluation of CoT prompting, extending beyond arithmetic to include complex logical and commonsense reasoning tasks, areas where standard CoT methods fall short. We propose the integration of human-like heuristics and shortcuts into language models (LMs) through "break the chain" strategies. These strategies disrupt traditional CoT processes using controlled variables to assess their efficacy. Additionally, we develop innovative zero-shot prompting strategies that encourage the use of shortcuts, enabling LMs to quickly exploit reasoning clues and bypass detailed procedural steps. Our comprehensive experiments across various LMs, both commercial and open-source, reveal that LMs maintain effective performance with "break the chain" strategies. We also introduce ShortcutQA, a dataset specifically designed to evaluate reasoning through shortcuts, compiled from competitive tests optimized for heuristic reasoning tasks such as forward/backward reasoning and simplification. Our analysis confirms that ShortcutQA not only poses a robust challenge to LMs but also serves as an essential benchmark for enhancing reasoning efficiency in AI.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 57 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

26 of 57 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step23 May
  2. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Models24 Apr
  3. 2024The Impact of Reasoning Step Length on Large Language Models10 Jan
  4. 2023Implicit Chain of Thought Reasoning via Knowledge Distillation2 Nov
  5. 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul
  6. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models6 May
  7. 2023SCOTT: Self-Consistent Chain-of-Thought Distillation3 May
  8. 2023When do you need Chain-of-Thought Prompting for ChatGPT?6 Apr
  9. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters20 Dec
  10. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks22 Nov
  11. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar

Other references · 15

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  4. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  5. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  6. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  7. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  8. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  9. 2024Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffoldingcited by 115
  10. 2024Premise Order Matters in Reasoning with Large Language Modelscited by 71
  11. 2019Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QAcited by 63
  12. 2023Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Modelscited by 38
3 more
  1. 2023Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuningcited by 26
  2. 2023Exploring Self-supervised Logic-enhanced Training for Large Language Modelscited by 8
  3. 2023Chain-of-Symbol Prompting Elicits Planning in Large Langauge Modelscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 20; 2026: 11.

32 papers.

Sort:

By area:

  1. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMscited by 246
  2. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuningcited by 167
  3. 2025LightThinker: Thinking Step-by-Step Compressioncited by 104
  4. 2025Efficient Reasoning Models: A Surveycited by 93
  5. 2024Not All LLM Reasoners Are Created Equalcited by 29
  6. 2025Thought Manipulation: External Thought Can Be Efficient for Large Reasoning Modelscited by 23
  7. 2025Think Clearly: Improving Reasoning via Redundant Token Pruningcited by 17
  8. 2025Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoningcited by 16
  9. 2025LIMOPro: Reasoning Refinement for Efficient and Effective Test-time Scalingcited by 15
  10. 2025Fast on the Easy, Deep on the Hard: Efficient Reasoning via Powered Length Penaltycited by 15
  11. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  12. 2025Learning to Focus: Causal Attention Distillation via Gradient-Guided Token Pruningcited by 13
20 more
  1. 2025The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoningcited by 12
  2. 2025A*-Thought: Efficient Reasoning via Bidirectional Compression for Low-Resource Settingscited by 9
  3. 2025Assessing Coherency and Consistency of Code Execution Reasoning by Large Language Modelscited by 5
  4. 2025Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Modelscited by 4
  5. 2025KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integrationcited by 4
  6. 2026Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesiscited by 4
  7. 2026Accordion-Thinking: Self-Regulated Step Summaries for Efficient and Readable LLM Reasoningcited by 4
  8. 2026Efficient Reasoning with Balanced Thinkingcited by 4
  9. 2026LightThinker++: From Reasoning Compression to Memory Managementcited by 4
  10. 2025Unveiling the Mechanisms of Multi-Hop Reasoning in Transformers via Identity Bridgecited by 3
  11. 2025Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoningcited by 3
  12. 2025StreetMath: Study of LLMs' Approximation Behaviorscited by 2
  13. 2026Verifiable Reasoning for LLM-based Generative Recommendationcited by 2
  14. 2026Short Chains, Deep Thoughts: Balancing Reasoning Efficiency and Intra-Segment Capability via Split-Merge Optimizationcited by 1
  15. 2026Reasoning Models Know What's Important, and Encode It in Their Activationscited by 1
  16. 2025Merlin's Whisper: Enabling Efficient Reasoning in Large Language Models via Black-box Persuasive Promptingcited by 0
  17. 2026DyCon: Dynamic Reasoning Control via Evolving Difficulty Modelingcited by 0
  18. 2026When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillationcited by 0
  19. 2026Can We Trust LLM's Logic? Quantifying Uncertainty, Coherence, and Robustness via a Graph-Based Frameworkcited by 0
  20. 2026MetaCtrl: Your Large Language Models Can Reason Better and More Concisely with a Metacognitive Controllercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  2. 2024ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
  3. 2025Early Stopping Chain-of-thoughts in Large Language Models
  4. 2024Chain-of-Thought Reasoning Without Prompting
  5. 2025Innate Reasoning is Not Enough: In-Context Learning Enhances Reasoning Large Language Models with Less Overthinking
  6. 2025The Challenge of Teaching Reasoning to LLMs Without RL or Distillation
  7. 2025Short-Path Prompting in LLMs: Analyzing Reasoning Instability and Solutions for Robust Performance
  8. 2022Large Language Models are Zero-Shot Reasoners