mlsanityData to 14 Oct 2026

LAMBADA: Backward Chaining for Automated Reasoning in Natural Language

arXiv 2212.13894 ↗First posted 20 Dec 2022cs.AI cs.LGcited by 66

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Remarkable progress has been made on automated reasoning with natural text, by using Language Models (LMs) and methods such as Chain-of-Thought and Selection-Inference. These techniques search for proofs in the forward direction from axioms to the conclusion, which suffers from a combinatorial explosion of the search space, and thus high failure rates for problems requiring longer chains of reasoning. The classical automated reasoning literature has shown that reasoning in the backward direction (i.e. from the intended conclusion to supporting axioms) is significantly more efficient at proof-finding. Importing this intuition into the LM setting, we develop a Backward Chaining algorithm, called LAMBADA, that decomposes reasoning into four sub-modules. These sub-modules are simply implemented by few-shot prompted LM inference. We show that LAMBADA achieves sizable accuracy boosts over state-of-the-art forward reasoning methods on challenging logical reasoning datasets, particularly when deep and accurate proof chains are required.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 35 of its 49 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

35 of 49 references.

Closest ancestors · 7

Same direction, newest first

  1. 2022Can Transformers Reason in Fragments of Natural Language?10 Nov
  2. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought3 Oct
  3. 2022FOLIO: Natural Language Reasoning with First-Order Logic2 Sep
  4. 2022Faithful Reasoning Using Large Language Models30 Aug
  5. 2022On the Paradox of Learning to Reason from Data23 May
  6. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning19 May
  7. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Models30 Nov

Other references · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  8. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  9. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  10. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  11. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  12. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
16 more
  1. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 324
  2. 2020Transformers as Soft Reasoners over Languagecited by 270
  3. 2022Transformers Learn Shortcuts to Automatacited by 268
  4. 2020Neurosymbolic AI: The 3rd Wavecited by 255
  5. 2022Exploring Length Generalization in Large Language Modelscited by 206
  6. 2020The Next Decade in AI: Four Steps Towards Robust Artificial Intelligencecited by 195
  7. 2021Explaining Answers with Entailment Treescited by 152
  8. 2022Teaching Algorithmic Reasoning via In-context Learningcited by 112
  9. 2022Successive Prompting for Decomposing Complex Questionscited by 94
  10. 2021Generate & Rank: A Multi-task Framework for Math Word Problemscited by 82
  11. 2022Iteratively Prompt Pre-trained Language Models for Chain of Thoughtcited by 75
  12. 2020Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answeringcited by 49
  13. 2020Measuring Systematic Generalization in Neural Proof Generation with Transformerscited by 43
  14. 2020Critical Thinking for Language Modelscited by 31
  15. 2021RuleBert: Teaching Soft Rules to Pre-trained Language Modelscited by 25
  16. 2022Natural Language Deduction with Incomplete Informationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 30; 2024: 18; 2025: 11; 2026: 7.

66 papers.

Sort:

By area:

  1. 2023Augmented Language Models: a Surveycited by 403
  2. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 302
  3. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 176
  4. 2023GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoningcited by 108
  5. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  6. 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 88
  7. 2023Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examplescited by 84
  8. 2023Evaluating Language Models for Mathematics through Interactionscited by 70
  9. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 70
  10. 2023Large Language Models can Learn Rulescited by 69
  11. 2024Understanding Transformer Reasoning Capabilities via Graph Algorithmscited by 67
  12. 2023Natural Language Reasoning, A Surveycited by 61
38 more
  1. 2024Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Samplingcited by 61
  2. 2023Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoningcited by 45
  3. 2023BoardgameQA: A Dataset for Natural Language Reasoning with Contradictory Informationcited by 44
  4. 2023Forward-Backward Reasoning in Large Language Models for Mathematical Verificationcited by 40
  5. 2023Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuningcited by 39
  6. 2023Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Modelscited by 37
  7. 2023Automatic Model Selection with Large Language Models for Reasoningcited by 35
  8. 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Modelscited by 35
  9. 2023DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacycited by 34
  10. 2023Mathematical Language Models: A Surveycited by 24
  11. 2024Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language Modelscited by 23
  12. 2025S2-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiencycited by 22
  13. 2024Transformers Struggle to Learn to Searchcited by 19
  14. 2024RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Modelscited by 18
  15. 2024Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMscited by 18
  16. 2023Faithful Question Answering with Monte-Carlo Planningcited by 16
  17. 2023Iterated Decomposition: Improving Science Q&A by Supervising Reasoning Processescited by 15
  18. 2023IAG: Induction-Augmented Generation Framework for Answering Reasoning Questionscited by 15
  19. 2024Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoningcited by 15
  20. 2024Structure Guided Prompt: Instructing Large Language Model in Multi-Step Reasoning by Exploring Graph Structure of the Textcited by 14
  21. 2023A Systematic Survey of Prompt Engineering on Vision-Language Foundation Modelscited by 12
  22. 2023TaskLAMA: Probing the Complex Task Understanding of Language Modelscited by 12
  23. 2025A Survey on Mathematical Reasoning and Optimization with Large Language Modelscited by 12
  24. 2023Large Language Model Programscited by 11
  25. 2025Metis: Training LLMs with FP4 Quantizationcited by 9
  26. 2023Certified Deductive Reasoning with Language Modelscited by 8
  27. 2025RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remappingcited by 7
  28. 2025Reliable Proof Generation with LLMs via Analogical Retrieval and Symbolic Verification: A Case Study in Euclidean Geometrycited by 7
  29. 2023Noisy Exemplars Make Large Language Models More Robust: A Domain-Agnostic Behavioral Analysiscited by 6
  30. 2023On Meta-Promptingcited by 6
  31. 2024Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic Approachcited by 6
  32. 2025LogicReward: Incentivizing LLM Reasoning via Step-Wise Logical Supervisioncited by 6
  33. 2024Curriculum Demonstration Selection for In-Context Learningcited by 5
  34. 2024Thought-Like-Pro: Enhancing Reasoning of Large Language Models through Self-Driven Prolog-based Chain-of-Thoughtcited by 4
  35. 2025Are Language Models Efficient Reasoners? A Perspective from Logic Programmingcited by 4
  36. 2026Spectra: Rethinking Optimizers for LLMs Under Spectral Anisotropycited by 4
  37. 2026Goal-Oriented Reasoning for RAG-based Memory in Conversational Agentic LLM Systemscited by 4
  38. 2023Concise and Organized Perception Facilitates Reasoning in Large Language Modelscited by 3

50 most cited of 66 All 66 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024SymBa: Symbolic Backward Chaining for Structured Natural Language Reasoning
  2. 2025From Hypothesis to Premises: LLM-based Backward Logical Reasoning with Selective Symbolic Translation
  3. 2024Bi-Chainer: Automated Large Language Models Reasoning with Bidirectional Chaining
  4. 2024Deductive Beam Search: Decoding Deducible Rationale for Chain-of-Thought Reasoning
  5. 2022Evaluating Step-by-Step Reasoning through Symbolic Verification
  6. 2024P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains
  7. 2024Exploring the Role of Reasoning Structures for Constructing Proofs in Multi-Step Natural Language Reasoning with Large Language Models
  8. 2023ReCEval: Evaluating Reasoning Chains via Correctness and Informativeness