mlsanityData to 14 Oct 2026

Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs

arXiv 2305.11860 ↗First posted 19 May 2023cs.CLcited by 115

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

A popular approach for improving the correctness of output from large language models (LLMs) is Self-Consistency - poll the LLM multiple times and output the most frequent solution. Existing Self-Consistency techniques always generate a constant number of samples per question, where a better approach will be to non-uniformly distribute the available budget based on the amount of agreement in the samples generated so far. In response, we introduce Adaptive-Consistency, a cost-efficient, model-agnostic technique that dynamically adjusts the number of samples per question using a lightweight stopping criterion. Our experiments over 17 reasoning and code generation datasets and three LLMs demonstrate that Adaptive-Consistency reduces sample budget by up to 7.9 times with an average accuracy drop of less than 0.1%. Our code and data are available at https://www.sample-step-by-step.info

Key facts. Cited by 115 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 31 of its 54 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

31 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvement27 Apr

Other references · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2021Program Synthesis with Large Language Modelscited by 3,637
  8. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  9. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  10. 2021Measuring Coding Challenge Competence With APPScited by 902
  11. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  12. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
18 more
  1. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  2. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 439
  3. 2022CodeT: Code Generation with Generated Testscited by 384
  4. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
  5. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  6. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 273
  7. 2020DynaBERT: Dynamic BERT with Adaptive Width and Depthcited by 251
  8. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 226
  9. 2021Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networkscited by 117
  10. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  11. 2021Top-KAST: Top-K Always Sparse Trainingcited by 79
  12. 2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 74
  13. 2019Sampling Bias in Deep Active Classification: An Empirical Studycited by 36
  14. 2020Early Exit or Not: Resource-Efficient Blind Quality Enhancement for Compressed Imagescited by 28
  15. 2023Adaptive Computation with Elastic Input Sequencecited by 22
  16. 2021RomeBERT: Robust Training of Multi-Exit BERTcited by 19
  17. 2017An Adaptive Sampling Scheme to Efficiently Train Fully Convolutional Networks for Semantic Segmentationcited by 3
  18. 2022FLOWGEN: Fast and slow graph generationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 9; 2025: 41; 2026: 61.

115 papers.

Sort:

By area:

  1. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 384
  2. 2025Dynamic Early Exit in Reasoning Modelscited by 219
  3. 2025Deep Think with Confidencecited by 205
  4. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 137
  5. 2023AutoMix: Automatically Mixing Language Modelscited by 125
  6. 2025Confidence Improves Self-Consistency in LLMscited by 113
  7. 2025Efficient Reasoning Models: A Surveycited by 95
  8. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  9. 2023Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Modelscited by 77
  10. 2025Efficient Test-Time Scaling via Self-Calibrationcited by 75
  11. 2024Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoningcited by 63
  12. 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Samplingcited by 63
38 more
  1. 2025Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decodingcited by 63
  2. 2024Efficiently Scaling LLM Reasoning with Certaindexcited by 58
  3. 2025The Majority is not always right: RL training for solution aggregationcited by 47
  4. 2024MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoningcited by 41
  5. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  6. 2024AlphaVerus: Bootstrapping Formally Verified Code Generation through Self-Improving Translation and Treefinementcited by 35
  7. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  8. 2025Think Smarter not Harder: Adaptive Reasoning with Inference Aware Optimizationcited by 29
  9. 2026Does Your Reasoning Model Implicitly Know When to Stop Thinking?cited by 19
  10. 2026TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Modelscited by 18
  11. 2024Path-Consistency with Prefix Enhancement for Efficient Inference in LLMscited by 17
  12. 2023Semantic Consistency for Assuring Reliability of Large Language Modelscited by 15
  13. 2025Lost at the Beginning of Reasoningcited by 15
  14. 2026Parallel-Probe: Towards Efficient Parallel Thinking via 2D Probingcited by 14
  15. 2025Improving Consistency in Large Language Models through Chain of Guidancecited by 13
  16. 2025Learning to Focus: Causal Attention Distillation via Gradient-Guided Token Pruningcited by 13
  17. 2023Adaptive-Solver Framework for Dynamic Strategy Selection in Large Language Model Reasoningcited by 12
  18. 2025Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMscited by 12
  19. 2025From Efficiency to Adaptivity: A Deeper Look at Adaptive Reasoning in Large Language Modelscited by 12
  20. 2025Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregationcited by 11
  21. 2025Harnessing Consistency for Robust Test-Time LLM Ensemblecited by 10
  22. 2026LLMs Improving LLMs: Agentic Discovery for Test-Time Scalingcited by 10
  23. 2026When Agents Commit Too Soon: Diagnosing Premature Commitment in LLM Agentscited by 10
  24. 2025Optimal Self-Consistency for Efficient Reasoning with Large Language Modelscited by 9
  25. 2026Optimal Bayesian Stopping for Efficient Inference of Consistent LLM Answerscited by 8
  26. 2025DynScaling: Efficient Verifier-free Inference Scaling via Dynamic and Integrated Samplingcited by 7
  27. 2025Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accuratelycited by 6
  28. 2025Zero-Overhead Introspection for Adaptive Test-Time Computecited by 6
  29. 2025Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoningcited by 6
  30. 2025Speculative Decoding for Multi-Sample Inferencecited by 5
  31. 2025Survey of GenAI for Automotive Software Development: From Requirements to Executable Codecited by 5
  32. 2026How Much Can Repeated Running Buy? A Measured Variance Ceiling for LLM Code Evaluationcited by 5
  33. 2024Reranking Laws for Language Generation: A Communication-Theoretic Perspectivecited by 4
  34. 2024Let's Be Self-generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Modelscited by 4
  35. 2025Training-free LLM Verification via Recycling Few-shot Examplescited by 4
  36. 2026ATLAS: Agentic Test-time Learning-to-Allocate Scalingcited by 4
  37. 2026Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascadecited by 4
  38. 2025CDW-CoT: Clustered Distance-Weighted Chain-of-Thoughts Reasoningcited by 3

50 most cited of 115 All 115 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation
  2. 2025A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
  3. 2023Universal Self-Consistency for Large Language Model Generation
  4. 2025Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs
  5. 2023Self-Consistency of Large Language Models under Ambiguity
  6. 2026Reliability-Aware Adaptive Self-Consistency for Efficient Sampling in LLM Reasoning
  7. 2024Refining Answer Distributions for Improved Large Language Model Reasoning
  8. 2024Evaluating Consistency and Reasoning Capabilities of Large Language Models