mlsanityData to 7 Oct 2026

Self-Evaluation Guided Beam Search for Reasoning

arXiv 2305.00633 ↗First posted 1 May 2023cs.CL cs.AI cs.LGcited by 195

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Breaking down a problem into intermediate steps has demonstrated impressive performance in Large Language Model (LLM) reasoning. However, the growth of the reasoning chain introduces uncertainty and error accumulation, making it challenging to elicit accurate final results. To tackle this challenge of uncertainty in multi-step reasoning, we introduce a stepwise self-evaluation mechanism to guide and calibrate the reasoning process of LLMs. We propose a decoding algorithm integrating the self-evaluation guidance via stochastic beam search. The self-evaluation guidance serves as a better-calibrated automatic criterion, facilitating an efficient search in the reasoning space and resulting in superior prediction quality. Stochastic beam search balances exploitation and exploration of the search space with temperature-controlled randomness. Our approach surpasses the corresponding Codex-backboned baselines in few-shot accuracy by 6.34%, 9.56%, and 5.46% on the GSM8K, AQuA, and StrategyQA benchmarks, respectively. Experiment results with Llama-2 on arithmetic reasoning demonstrate the efficiency of our method in outperforming the baseline methods with comparable computational budgets. Further analysis in multi-step reasoning finds our self-evaluation guidance pinpoints logic failures and leads to higher consistency and robustness. Our code is publicly available at https://guideddecoding.github.io/.

Key facts. Cited by 195 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 39 of its 52 references that are in the map. First posted on arXiv 1 May 2023.

Builds on

39 of 52 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023REFINER: Reasoning Feedback on Intermediate Representations4 Apr
  2. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar
  3. 2022Large Language Models Can Self-Improve20 Oct
  4. 2022Language Models (Mostly) Know What They Know11 Jul

Other references · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2017On Calibration of Modern Neural Networkscited by 4,860
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  10. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  11. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  12. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
23 more
  1. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  2. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  3. 2018Hierarchical Neural Story Generationcited by 1,182
  4. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  5. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  6. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  7. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  8. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  9. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  10. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  11. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  12. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  13. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  14. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 205
  15. 2020Measuring Association Between Labels and Free-Text Rationalescited by 159
  16. 2021Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processingcited by 126
  17. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  18. 2017Long Text Generation via Adversarial Training with Leaked Informationcited by 100
  19. 2018Language GANs Falling Shortcited by 97
  20. 2022Coder Reviewer Reranking for Code Generationcited by 83
  21. 2020Best-First Beam Searchcited by 40
  22. 2022SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Modelscited by 38
  23. 2020Towards Interpretable Natural Language Understanding with Explanations as Latent Variablescited by 30

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 56; 2025: 88; 2026: 42.

195 papers.

Sort:

By area:

  1. 2025s1: Simple test-time scalingcited by 1,311
  2. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  3. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  4. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  5. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  6. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Modelscited by 282
  7. 2024TextGrad: Automatic "Differentiation" via Textcited by 250
  8. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  9. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  10. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  11. 2024Chain-of-Thought Reasoning Without Promptingcited by 208
  12. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
38 more
  1. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 140
  2. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 136
  3. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  4. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 123
  5. 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
  6. 2023ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Searchcited by 103
  7. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoningcited by 96
  8. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  9. 2023LLMs cannot find reasoning errors, but can correct them given the error locationcited by 85
  10. 2024A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learningcited by 74
  11. 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 72
  12. 2025Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?cited by 69
  13. 2025AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learningcited by 68
  14. 2024StateFlow: Enhancing LLM Task-Solving through State-Driven Workflowscited by 65
  15. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  16. 2024Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learningcited by 63
  17. 2025VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perceptioncited by 60
  18. 2025Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifierscited by 58
  19. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 57
  20. 2024Efficiently Scaling LLM Reasoning with Certaindexcited by 56
  21. 2024DeAL: Decoding-time Alignment for Large Language Modelscited by 55
  22. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
  23. 2025Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verificationcited by 49
  24. 2025MindJourney: Test-Time Scaling with World Models for Spatial Reasoningcited by 49
  25. 2024Towards Efficient LLM Grounding for Embodied Multi-Agent Collaborationcited by 45
  26. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 43
  27. 2025Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflectioncited by 42
  28. 2023A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoningcited by 41
  29. 2024Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflectioncited by 40
  30. 2025VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Datacited by 39
  31. 2024FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answeringcited by 38
  32. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  33. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  34. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  35. 2025Navigate the Unknown: Enhancing LLM Reasoning with Intrinsic Motivation Guided Explorationcited by 33
  36. 2024Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewardscited by 31
  37. 2024LiteSearch: Efficacious Tree Search for LLMcited by 30
  38. 2023PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuningcited by 29

50 most cited of 195 All 195 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Provable Test-Time Scaling for Beam Search in LLM Reasoning
  2. 2025SELT: Self-Evaluation Tree Search for LLMs with Task Decomposition
  3. 2025MITS: Enhanced Tree Search Reasoning for LLMs via Pointwise Mutual Information
  4. 2024Inference Scaling vs Reasoning: An Empirical Analysis of Compute-Optimal LLM Problem-Solving
  5. 2025MARGE: Improving Math Reasoning for LLMs with Guided Exploration
  6. 2025Bag of Tricks for Inference-time Computation of LLM Reasoning
  7. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models
  8. 2025REASONING COMPILER: LLM-Guided Optimizations for Efficient Model Serving