# Revisiting Chain-of-Thought Prompting: Zero-shot Can Be Stronger than Few-shot

arXiv: https://arxiv.org/abs/2506.14641 · HTML: https://mlsanity.com/paper/2506.14641/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 14 of its 47 references that are in the map.
- First posted on arXiv 17 Jun 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 14 of 14)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://mlsanity.com/paper/2409.12122/) (2024)
- [Larger language models do in-context learning differently](https://mlsanity.com/paper/2303.03846/) (2023)
- [One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention](https://mlsanity.com/paper/2307.03576/) (2023)
- [LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning](https://mlsanity.com/paper/2007.08124/) (2020)
- [Self-Generated In-Context Learning: Leveraging Auto-regressive Language Models as a Demonstration Generator](https://mlsanity.com/paper/2206.08082/) (2022)

## Built on by (most cited first, 12 of 12)

- [DAGGER: Distractor-Aware Graph Generation for Executable Reasoning in Math Problems](https://mlsanity.com/paper/2601.06853/) (2026)
- [Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Models](https://mlsanity.com/paper/2601.11340/) (2026)
- [TabSieve: Explicit In-Table Evidence Selection for Tabular Prediction](https://mlsanity.com/paper/2602.11700/) (2026)
- [From Harm to Help: Turning Reasoning In-Context Demos into Assets for Reasoning LMs](https://mlsanity.com/paper/2509.23196/) (2025)
- [Process In-Context Learning: Enhancing Mathematical Reasoning via Dynamic Demonstration Insertion](https://mlsanity.com/paper/2601.11979/) (2026)
- [Hierarchical Chain-of-Thought: Enhancing LLM Reasoning Performance and Efficiency](https://mlsanity.com/paper/2604.00130/) (2026)
- [A Systematic Evaluation of Large Language Models for PTSD Severity Estimation: The Role of Contextual Knowledge and Modeling Strategies](https://mlsanity.com/paper/2602.06015/) (2026)
- [FewMMBench: A Benchmark for Multimodal Few-Shot Learning](https://mlsanity.com/paper/2602.21854/) (2026)
- [LLM as a Meta-Judge: Synthetic Data for NLP Evaluation Metric Validation](https://mlsanity.com/paper/2603.09403/) (2026)
- [UtilityMax Prompting: A Formal Framework for Multi-Objective Large Language Model Tasks](https://mlsanity.com/paper/2603.11583/) (2026)
- [Benchmarking Large Language Models for Safety Data Extraction](https://mlsanity.com/paper/2606.11204/) (2026)
- [Soft Guidance Starts to Outperform CoT Prompting as LLMs Improve](https://mlsanity.com/paper/2608.03550/) (2026)

## Abstract (arXiv)

In-Context Learning (ICL) is an essential emergent ability of Large Language Models (LLMs), and recent studies introduce Chain-of-Thought (CoT) to exemplars of ICL to enhance the reasoning capability, especially in mathematics tasks. However, given the continuous advancement of model capabilities, it remains unclear whether CoT exemplars still benefit recent, stronger models in such tasks. Through systematic experiments, we find that for recent strong models such as the Qwen2.5 series, adding traditional CoT exemplars does not improve reasoning performance compared to Zero-Shot CoT. Instead, their primary function is to align the output format with human expectations. We further investigate the effectiveness of enhanced CoT exemplars, constructed using answers from advanced models such as textttQwen2.5-Max and textttDeepSeek-R1. Experimental results indicate that these enhanced exemplars still fail to improve the model's reasoning performance. Further analysis reveals that models tend to ignore the exemplars and focus primarily on the instructions, leading to no observable gain in reasoning ability. Overall, our findings highlight the limitations of the current ICL+CoT framework in mathematical reasoning, calling for a re-examination of the ICL paradigm and the definition of exemplars.