mlsanityData to 7 Oct 2026

Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning

arXiv 2401.10480 ↗First posted 19 Jan 2024cs.CL cs.AIcited by 105

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Self-consistency (SC) has been a widely used decoding strategy for chain-of-thought reasoning. Despite bringing significant performance improvements across a variety of multi-step reasoning tasks, it is a high-cost method that requires multiple sampling with the preset size. In this paper, we propose a simple and scalable sampling process, Early-Stopping Self-Consistency (ESC), to greatly reduce the cost of SC without sacrificing performance. On this basis, one control scheme for ESC is further derivated to dynamically choose the performance-cost balance for different tasks and models. To demonstrate ESC's effectiveness, we conducted extensive experiments on three popular categories of reasoning tasks: arithmetic, commonsense and symbolic reasoning over language models with varying scales. The empirical results show that ESC reduces the average number of sampling of chain-of-thought reasoning by a significant margin on six benchmarks, including MATH (-33.8%), GSM8K (-80.1%), StrategyQA (-76.8%), CommonsenseQA (-78.5%), Coin Flip (-84.2%) and Last Letters (-67.4%), while attaining comparable performances.

Key facts. Cited by 105 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 20 of its 24 references that are in the map. First posted on arXiv 19 Jan 2024.

Builds on

20 of 24 references.

References · 20

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2021Program Synthesis with Large Language Modelscited by 3,509
  9. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  11. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  12. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
8 more
  1. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  2. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  3. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  4. 2016Solving General Arithmetic Word Problemscited by 353
  5. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  6. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
  7. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  8. 2023Turning Dust into Gold: Distilling Complex Reasoning Capabilities from LLMs by Leveraging Negative Datacited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 45; 2026: 48.

105 papers.

Sort:

By area:

  1. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
  2. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  3. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  4. 2025Deep Think with Confidencecited by 199
  5. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 154
  6. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  7. 2025Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMscited by 115
  8. 2025Confidence Improves Self-Consistency in LLMscited by 111
  9. 2025Efficient Reasoning Models: A Surveycited by 93
  10. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  11. 2025Efficient Test-Time Scaling via Self-Calibrationcited by 74
  12. 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Samplingcited by 63
38 more
  1. 2025Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decodingcited by 62
  2. 2024Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoningcited by 61
  3. 2024Efficiently Scaling LLM Reasoning with Certaindexcited by 56
  4. 2025Dynamic Parallel Tree Search for Efficient LLM Reasoningcited by 42
  5. 2025Budget-Aware Tool Use Enables Effective Agent Scalingcited by 42
  6. 2024MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoningcited by 40
  7. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  8. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  9. 2025Let LRMs Break Free from Overthinking via Self-Braking Tuningcited by 27
  10. 2025The LLM Already Knows: Estimating LLM-Perceived Question Difficulty via Hidden Representationscited by 26
  11. 2025KTAE: A Model-Free Algorithm to Key-Tokens Advantage Estimation in Mathematical Reasoningcited by 19
  12. 2025Early Stopping Chain-of-thoughts in Large Language Modelscited by 19
  13. 2024Path-Consistency with Prefix Enhancement for Efficient Inference in LLMscited by 17
  14. 2026Does Your Reasoning Model Implicitly Know When to Stop Thinking?cited by 17
  15. 2025Lost at the Beginning of Reasoningcited by 15
  16. 2025Stop When Enough: Adaptive Early-Stopping for Chain-of-Thought Reasoningcited by 15
  17. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  18. 2025Learning to Focus: Causal Attention Distillation via Gradient-Guided Token Pruningcited by 13
  19. 2026Parallel-Probe: Towards Efficient Parallel Thinking via 2D Probingcited by 13
  20. 2025Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMscited by 12
  21. 2025Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregationcited by 11
  22. 2025Exploring the System 1 Thinking Capability of Large Reasoning Modelscited by 10
  23. 2025Harnessing Consistency for Robust Test-Time LLM Ensemblecited by 10
  24. 2025iMAD: Intelligent Multi-Agent Debate for Efficient and Accurate LLM Inferencecited by 10
  25. 2026Reliability-Aware Adaptive Self-Consistency for Efficient Sampling in LLM Reasoningcited by 10
  26. 2024Derailer-Rerailer: Adaptive Verification for Efficient and Reliable Language Model Reasoningcited by 9
  27. 2024Focused Large Language Models are Stable Many-Shot Learnerscited by 8
  28. 2025Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic Confidencecited by 8
  29. 2025Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoningcited by 8
  30. 2025Optimal Self-Consistency for Efficient Reasoning with Large Language Modelscited by 8
  31. 2024Adapting While Learning: Grounding LLMs for Scientific Problems with Intelligent Tool Usage Adaptationcited by 7
  32. 2026Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Modelscited by 7
  33. 2026Optimal Bayesian Stopping for Efficient Inference of Consistent LLM Answerscited by 7
  34. 2026LLMs Improving LLMs: Agentic Discovery for Test-Time Scalingcited by 7
  35. 2025Speculative Decoding for Multi-Sample Inferencecited by 6
  36. 2025Optimal Stopping vs Best-of-N for Inference Time Optimizationcited by 6
  37. 2025Survey of GenAI for Automotive Software Development: From Requirements to Executable Codecited by 5
  38. 2024Let's Be Self-generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Modelscited by 4

50 most cited of 105 All 105 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
  2. 2023Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs
  3. 2024Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting
  4. 2025A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
  5. 2024Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation
  6. 2023Universal Self-Consistency for Large Language Model Generation
  7. 2026Pause and Reflect: Conformal Aggregation for Chain-of-Thought Reasoning
  8. 2025Slim-SC: Thought Pruning for Efficient Scaling with Self-Consistency