mlsanityData to 7 Oct 2026

GRACE: Discriminator-Guided Chain-of-Thought Reasoning

arXiv 2305.14934 ↗First posted 24 May 2023cs.CL cs.AIcited by 35

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

In the context of multi-step reasoning, e.g., with chain-of-thought, language models (LMs) can easily assign a high likelihood to incorrect steps. As a result, decoding strategies that optimize for solution likelihood often yield incorrect solutions. To address this issue, we propose Guiding chain-of-thought ReAsoning with a CorrectnEss Discriminator (GRACE), a stepwise decoding approach that steers the decoding process towards producing correct reasoning steps. GRACE employs a step-level verifier or discriminator trained with a contrastive loss over correct and incorrect steps, which is used during decoding to score next-step candidates based on their correctness. Importantly, GRACE only requires sampling from the LM, without the need for LM training or fine-tuning. Using models from FLAN-T5 and LLaMA families, we evaluate GRACE over four math and two symbolic reasoning tasks, where it exhibits substantial performance gains compared to greedy decoding, verifiers, and self-consistency in most settings. When further combined with self-consistency, GRACE outperforms all the baselines by sizeable margins. Human and LLM evaluations over GSM8K show that GRACE not only improves the final answer accuracy but also the correctness of the intermediate reasoning. Our implementation can be accessed at https://github.com/mukhal/grace.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 38 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

29 of 38 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Large Language Models Are Reasoning Teachers20 Dec
  2. 2022Complexity-Based Prompting for Multi-Step Reasoning3 Oct
  3. 2022Large Language Models are Zero-Shot Reasoners24 May
  4. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar

Other references · 24

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  7. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  8. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  9. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  10. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  11. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  12. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
12 more
  1. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  2. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  3. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  4. 2016Solving General Arithmetic Word Problemscited by 353
  5. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  6. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
  7. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 175
  8. 2018Learning to Write with Cooperative Discriminatorscited by 119
  9. 2020A Distributional Approach to Controlled Text Generationcited by 117
  10. 2022Generating Natural Language Proofs with Verifier-Guided Searchcited by 52
  11. 2022Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutionscited by 38
  12. 2023BOLT: Fast Energy-based Controlled Text Generation with Tunable Biasescited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 12; 2025: 18; 2026: 1.

35 papers.

Sort:

By area:

  1. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  3. 2025Process Reward Models That Thinkcited by 118
  4. 2024ARGS: Alignment as Reward-Guided Searchcited by 99
  5. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 85
  6. 2024Reversing the Forget-Retain Objectives: An Efficient LLM Unlearning Framework from Logit Differencecited by 85
  7. 2024MARIO: MAth Reasoning with code Interpreter Output -- A Reproducible Pipelinecited by 81
  8. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  9. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  10. 2024Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflectioncited by 40
  11. 2024MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoningcited by 40
  12. 2024LiteSearch: Efficacious Tree Search for LLMcited by 30
23 more
  1. 2025Thinking Machines: A Survey of LLM based Reasoning Strategiescited by 25
  2. 2023SCREWS: A Modular Framework for Reasoning with Revisionscited by 16
  3. 2024Enhancing Language Model Factuality via Activation-Based Confidence Calibration and Guided Decodingcited by 14
  4. 2025Full-Step-DPO: Self-Supervised Preference Optimization with Step-wise Rewards for Mathematical Reasoningcited by 14
  5. 2025Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewardscited by 14
  6. 2024A Critical Look At Tokenwise Reward-Guided Text Generationcited by 12
  7. 2024FactCheckmate: Preemptively Detecting and Mitigating Hallucinations in LMscited by 11
  8. 2025A Survey on Feedback-based Multi-step Reasoning for Large Language Models on Mathematicscited by 11
  9. 2025When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifierscited by 11
  10. 2025Rethinking Reward Models for Multi-Domain Test-Time Scalingcited by 10
  11. 2025Policy Guided Tree Search for Enhanced LLM Reasoningcited by 8
  12. 2024Controllable Text Generation in the Instruction-Tuning Eracited by 7
  13. 2025Enhancing Large Language Model Reasoning with Reward Models: An Analytical Surveycited by 7
  14. 2025Towards Cost-Effective Reward Guided Text Generationcited by 5
  15. 2025Step-Wise Formal Verification for LLM-Based Mathematical Problem Solvingcited by 5
  16. 2023Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learningcited by 4
  17. 2025SPARE: Single-Pass Annotation with Reference-Guided Evaluation for Automatic Process Supervision and Reward Modellingcited by 4
  18. 2025Can Memory-Augmented Language Models Generalize on Reasoning-in-a-Haystack Tasks?cited by 2
  19. 2025Logit Arithmetic Elicits Long Reasoning Capabilities Without Trainingcited by 2
  20. 2025Logit Arithmetic Elicits Long Reasoning Capabilities Without Trainingcited by 2
  21. 2025In-Token Rationality Optimization: Towards Accurate and Concise LLM Reasoning via Self-Feedbackcited by 2
  22. 2024Correct after Answer: Enhancing Multi-Span Question Answering with Post-Processing Methodcited by 0
  23. 2026Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026GRACE: Step-Level Benchmark for Faithful Reasoning over Context
  2. 2025Latent Veracity Inference for Identifying Errors in Stepwise Reasoning
  3. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
  4. 2025Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs
  5. 2025When Reasoning Beats Scale: A 1.5B Reasoning Model Outranks 13B LLMs as Discriminator
  6. 2023OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoning
  7. 2025Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoning
  8. 2023Faithful Chain-of-Thought Reasoning