mlsanityData to 14 Oct 2026

Forward-Backward Reasoning in Large Language Models for Mathematical Verification

arXiv 2308.07758 ↗First posted 15 Aug 2023cs.CL cs.AI cs.LGcited by 40

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Self-Consistency samples diverse reasoning chains with answers and chooses the final answer by majority voting. It is based on forward reasoning and cannot further improve performance by sampling more reasoning chains when saturated. To further boost performance, we introduce backward reasoning to verify candidate answers. Specifically, for mathematical tasks, we mask a number in the question and ask the LLM to answer a backward question created by a simple template, i.e., to predict the masked number when a candidate answer is provided. Instead of using forward or backward reasoning alone, we propose FOBAR to combine FOrward and BAckward Reasoning for verification. Extensive experiments on six standard mathematical data sets and three LLMs show that FOBAR achieves state-of-the-art performance. In particular, FOBAR outperforms Self-Consistency, which uses forward reasoning alone, demonstrating that combining forward and forward reasoning is better. In addition, FOBAR performs better than existing verification methods, showing the effectiveness of the simple template used in backward reasoning and the proposed combination. Extensions to non-mathematical problems are also discussed and validated empirically.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 44 of its 57 references that are in the map. First posted on arXiv 15 Aug 2023.

Builds on

44 of 57 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thought19 May
  2. 2022Peano: Learning Formal Mathematical Reasoning29 Nov
  3. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning29 Sep

Other references · 41

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  9. 2023Let's Verify Step by Stepcited by 3,789
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
29 more
  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  2. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  3. 2023Teaching Large Language Models to Self-Debugcited by 988
  4. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  5. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  6. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  7. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  8. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  9. 2021MetaICL: Learning to Learn In Contextcited by 454
  10. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 388
  11. 2023Language Models can Solve Computer Taskscited by 372
  12. 2016Solving General Arithmetic Word Problemscited by 360
  13. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 307
  14. 2022Generating Sequences by Learning to Self-Correctcited by 277
  15. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 205
  16. 2021Meta-learning via Language Model In-context Tuningcited by 164
  17. 2023Active Prompting with Chain-of-Thought for Large Language Modelscited by 154
  18. 2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 149
  19. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  20. 2023Compositional Exemplars for In-context Learningcited by 144
  21. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 124
  22. 2023Self-Edit: Fault-Aware Code Editor for Code Generationcited by 111
  23. 2022LAMBADA: Backward Chaining for Automated Reasoning in Natural Languagecited by 66
  24. 2020Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Modelscited by 65
  25. 2023Natural Language Reasoning, A Surveycited by 61
  26. 2023OpenICL: An Open-Source Framework for In-context Learningcited by 51
  27. 2023kNN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inferencecited by 47
  28. 2020Learning to Prove Theorems by Learning to Generate Theoremscited by 43
  29. 2023Boosted Prompt Ensembles for Large Language Modelscited by 36

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 11; 2025: 18; 2026: 9.

40 papers.

Sort:

By area:

  1. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  2. 2025Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoningcited by 91
  3. 2024RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Modelscited by 88
  4. 2025PhysReason: A Comprehensive Benchmark towards Physics-Based Reasoningcited by 59
  5. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  6. 2025Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verificationcited by 50
  7. 2024GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solvingcited by 45
  8. 2024GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoningcited by 33
  9. 2024Theorem-Validated Reverse Chain-of-Thought Problem Generation for Geometric Reasoningcited by 32
  10. 2025O2-Searcher: A Searching-based Agent Model for Open-Domain Open-Ended Question Answeringcited by 23
  11. 2024MuMath-Code: Combining Tool-Use Large Language Models with Multi-perspective Data Augmentation for Mathematical Reasoningcited by 22
  12. 2024Neuro-Symbolic Data Generation for Math Reasoningcited by 22
28 more
  1. 2024MindMerger: Efficient Boosting LLM Reasoning in non-English Languagescited by 21
  2. 2025ST-Think: How Multimodal Large Language Models Reason About 4D Worlds from Ego-Centric Videoscited by 21
  3. 2025LEMMA: Learning from Errors for MatheMatical Advancement in LLMscited by 21
  4. 2024Learn Beyond The Answer: Training Language Models with Reflection for Mathematical Reasoningcited by 15
  5. 2025When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifierscited by 15
  6. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMscited by 12
  7. 2025Cascaded Language Models for Cost-effective Human-AI Decision-Makingcited by 11
  8. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  9. 2023Towards A Unified View of Answer Calibration for Multi-Step Reasoningcited by 8
  10. 2025MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generationcited by 8
  11. 2025Towards Active Synthetic Data Generation for Finetuning Language Modelscited by 6
  12. 2025Understanding and Evaluating Hallucinations in 3D Visual Language Modelscited by 5
  13. 2026Guideline-Grounded Evidence Accumulation for High-Stakes Agent Verificationcited by 5
  14. 2025Thinking Machines: Mathematical Reasoning in the Age of LLMscited by 4
  15. 2026Can LLMs Compress (and Decompress)? Evaluating Code Understanding and Execution via Invertibilitycited by 4
  16. 2024VLLaVO: Mitigating Visual Gap through LLMscited by 2
  17. 2025From Deferral to Learning: Online In-Context Knowledge Distillation for LLM Cascadescited by 1
  18. 2025CryptoQA: A Large-scale Question-answering Dataset for AI-assisted Cryptographycited by 1
  19. 2024SAAS: Solving Ability Amplification Strategy for Enhanced Mathematical Reasoning in Large Language Modelscited by 0
  20. 2025Audited Reasoning Refinement: Fine-Tuning Language Models via LLM-Guided Step-Wise Evaluation and Correctioncited by 0
  21. 2025Generating Verifiable Chain of Thoughts from Exection-Tracescited by 0
  22. 2026DGPO: Beyond Pairwise Preferences with Directional Consistent Groupwise Optimizationcited by 0
  23. 2026MetaMoE: Diversity-Aware Proxy Selection for Privacy-Preserving Mixture-of-Experts Unificationcited by 0
  24. 2026SPARD: Defending Harmful Fine-Tuning Attack via Safety Projection with Relevance-Diversity Data Selectioncited by 0
  25. 2026When Tokenization is Secretly Output Supervisioncited by 0
  26. 2026Construting Reverse Thinking: Developing Large Language Models' Reverse Thingking Abilitycited by 0
  27. 2026ReSolve: Reusing Candidate Reasoning through Selective Generative Moderationcited by 0
  28. 2026Anchors Enhance Controllability and Reasoning of Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Fill in the Blank: Exploring and Enhancing LLM Capabilities for Backward Reasoning in Math Word Problems
  2. 2026FABSVer: Faster Training and Better Self-Verification for LLM Mathematical Reasoning
  3. 2024Not All Votes Count! Programs as Verifiers Improve Self-Consistency of Language Models for Math Reasoning
  4. 2025Catch Me If You Can: How Smaller Reasoning Models Pretend to Reason with Mathematical Fidelity
  5. 2025Putnam-AXIOM: A Functional and Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs
  6. 2025ReliableMath: Benchmark of Reliable Mathematical Reasoning on Large Language Models
  7. 2026LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches
  8. 2026EternalMath: A Living Benchmark of Frontier Mathematics that Evolves with Human Discovery