# Break the Chain: Large Language Models Can be Shortcut Reasoners

arXiv: https://arxiv.org/abs/2406.06580 · HTML: https://mlsanity.com/paper/2406.06580/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 26 of its 57 references that are in the map.
- First posted on arXiv 4 Jun 2024.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 26 of 26)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023)
- [Decomposed Prompting: A Modular Approach for Solving Complex Tasks](https://mlsanity.com/paper/2210.02406/) (2022)
- [From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step](https://mlsanity.com/paper/2405.14838/) (2024)
- [Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters](https://mlsanity.com/paper/2212.10001/) (2022)
- [Let's Think Dot by Dot: Hidden Computation in Transformer Language Models](https://mlsanity.com/paper/2404.15758/) (2024)
- [Implicit Chain of Thought Reasoning via Knowledge Distillation](https://mlsanity.com/paper/2311.01460/) (2023)
- [The Impact of Reasoning Step Length on Large Language Models](https://mlsanity.com/paper/2401.04925/) (2024)
- [Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding](https://mlsanity.com/paper/2401.12954/) (2024)
- [SCOTT: Self-Consistent Chain-of-Thought Distillation](https://mlsanity.com/paper/2305.01879/) (2023)
- [Premise Order Matters in Reasoning with Large Language Models](https://mlsanity.com/paper/2402.08939/) (2024)
- [Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA](https://mlsanity.com/paper/1906.07132/) (2019)
- [Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Models](https://mlsanity.com/paper/2311.09278/) (2023)
- [When do you need Chain-of-Thought Prompting for ChatGPT?](https://mlsanity.com/paper/2304.03262/) (2023)
- [Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning](https://mlsanity.com/paper/2310.11716/) (2023)
- [Exploring Self-supervised Logic-enhanced Training for Large Language Models](https://mlsanity.com/paper/2305.13718/) (2023)
- [Chain-of-Symbol Prompting Elicits Planning in Large Langauge Models](https://mlsanity.com/paper/2305.10276/) (2023)

## Built on by (most cited first, 30 of 32)

- [TokenSkip: Controllable Chain-of-Thought Compression in LLMs](https://mlsanity.com/paper/2502.12067/) (2025)
- [CoT-Valve: Length-Compressible Chain-of-Thought Tuning](https://mlsanity.com/paper/2502.09601/) (2025)
- [LightThinker: Thinking Step-by-Step Compression](https://mlsanity.com/paper/2502.15589/) (2025)
- [Efficient Reasoning Models: A Survey](https://mlsanity.com/paper/2504.10903/) (2025)
- [Not All LLM Reasoners Are Created Equal](https://mlsanity.com/paper/2410.01748/) (2024)
- [Thought Manipulation: External Thought Can Be Efficient for Large Reasoning Models](https://mlsanity.com/paper/2504.13626/) (2025)
- [Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning](https://mlsanity.com/paper/2506.09853/) (2025)
- [Think Clearly: Improving Reasoning via Redundant Token Pruning](https://mlsanity.com/paper/2507.08806/) (2025)
- [Generative AI Act II: Test Time Scaling Drives Cognition Engineering](https://mlsanity.com/paper/2504.13828/) (2025)
- [LIMOPro: Reasoning Refinement for Efficient and Effective Test-time Scaling](https://mlsanity.com/paper/2505.19187/) (2025)
- [Fast on the Easy, Deep on the Hard: Efficient Reasoning via Powered Length Penalty](https://mlsanity.com/paper/2506.10446/) (2025)
- [Learning to Focus: Causal Attention Distillation via Gradient-Guided Token Pruning](https://mlsanity.com/paper/2506.07851/) (2025)
- [The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoning](https://mlsanity.com/paper/2510.06557/) (2025)
- [A*-Thought: Efficient Reasoning via Bidirectional Compression for Low-Resource Settings](https://mlsanity.com/paper/2505.24550/) (2025)
- [Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesis](https://mlsanity.com/paper/2601.17687/) (2026)
- [Accordion-Thinking: Self-Regulated Step Summaries for Efficient and Readable LLM Reasoning](https://mlsanity.com/paper/2602.03249/) (2026)
- [Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Models](https://mlsanity.com/paper/2507.02663/) (2025)
- [Assessing Coherency and Consistency of Code Execution Reasoning by Large Language Models](https://mlsanity.com/paper/2510.15079/) (2025)
- [KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration](https://mlsanity.com/paper/2508.15790/) (2025)
- [Unveiling the Mechanisms of Multi-Hop Reasoning in Transformers via Identity Bridge](https://mlsanity.com/paper/2509.24653/) (2025)
- [Efficient Reasoning with Balanced Thinking](https://mlsanity.com/paper/2603.12372/) (2026)
- [LightThinker++: From Reasoning Compression to Memory Management](https://mlsanity.com/paper/2604.03679/) (2026)
- [Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning](https://mlsanity.com/paper/2511.07483/) (2025)
- [StreetMath: Study of LLMs' Approximation Behaviors](https://mlsanity.com/paper/2510.25776/) (2025)
- [Verifiable Reasoning for LLM-based Generative Recommendation](https://mlsanity.com/paper/2603.07725/) (2026)
- [Short Chains, Deep Thoughts: Balancing Reasoning Efficiency and Intra-Segment Capability via Split-Merge Optimization](https://mlsanity.com/paper/2602.03141/) (2026)
- [Reasoning Models Know What's Important, and Encode It in Their Activations](https://mlsanity.com/paper/2604.18307/) (2026)
- [Merlin's Whisper: Enabling Efficient Reasoning in Large Language Models via Black-box Persuasive Prompting](https://mlsanity.com/paper/2510.10528/) (2025)
- [DyCon: Dynamic Reasoning Control via Evolving Difficulty Modeling](https://mlsanity.com/paper/2606.07108/) (2026)
- [When Compression Helps and When It Hurts: Condition-Aware Analysis of Chain-of-Thought Distillation](https://mlsanity.com/paper/2606.21704/) (2026)

## Abstract (arXiv)

Recent advancements in Chain-of-Thought (CoT) reasoning utilize complex modules but are hampered by high token consumption, limited applicability, and challenges in reproducibility. This paper conducts a critical evaluation of CoT prompting, extending beyond arithmetic to include complex logical and commonsense reasoning tasks, areas where standard CoT methods fall short. We propose the integration of human-like heuristics and shortcuts into language models (LMs) through "break the chain" strategies. These strategies disrupt traditional CoT processes using controlled variables to assess their efficacy. Additionally, we develop innovative zero-shot prompting strategies that encourage the use of shortcuts, enabling LMs to quickly exploit reasoning clues and bypass detailed procedural steps. Our comprehensive experiments across various LMs, both commercial and open-source, reveal that LMs maintain effective performance with "break the chain" strategies. We also introduce ShortcutQA, a dataset specifically designed to evaluate reasoning through shortcuts, compiled from competitive tests optimized for heuristic reasoning tasks such as forward/backward reasoning and simplification. Our analysis confirms that ShortcutQA not only poses a robust challenge to LMs but also serves as an essential benchmark for enhancing reasoning efficiency in AI.