mlsanityData to 14 Oct 2026

Logical Reasoning over Natural Language as Knowledge Representation: A Survey

arXiv 2303.12023 ↗First posted 21 Mar 2023cs.CL cs.AIcited by 32

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Logical reasoning is central to human cognition and intelligence. It includes deductive, inductive, and abductive reasoning. Past research of logical reasoning within AI uses formal language as knowledge representation and symbolic reasoners. However, reasoning with formal language has proved challenging (e.g., brittleness and knowledge-acquisition bottleneck). This paper provides a comprehensive overview on a new paradigm of logical reasoning, which uses natural language as knowledge representation and pretrained language models as reasoners, including philosophical definition and categorization of logical reasoning, advantages of the new paradigm, benchmarks and methods, challenges of the new paradigm, possible future directions, and relation to related NLP fields. This new paradigm is promising since it not only alleviates many challenges of formal representation but also has advantages over end-to-end neural methods. This survey focus on transformer-based LLMs explicitly working on deductive, inductive, and abductive reasoning over English representation.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 18 of its 97 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

18 of 97 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Language Models as Inductive Reasoners21 Dec
  2. 2022Towards Reasoning in Large Language Models: A Survey20 Dec
  3. 2022FOLIO: Natural Language Reasoning with First-Order Logic2 Sep
  4. 2022RobustLR: Evaluating Robustness to Logical Perturbation in Deductive Reasoning25 May
  5. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning19 May

Other references · 13

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Reasoning with Language Model Prompting: A Surveycited by 252
  4. 2019On the Measure of Intelligencecited by 144
  5. 2023Goal Driven Discovery of Distributional Differences via Language Descriptionscited by 62
  6. 2022Generating Natural Language Proofs with Verifier-Guided Searchcited by 52
  7. 2022Entailer: Answering Questions with Faithful and Truthful Chains of Reasoningcited by 38
  8. 2022Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvementcited by 37
  9. 2022Natural Language Deduction through Search over Statement Compositionscited by 35
  10. 2022Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computingcited by 35
  11. 2019How Additional Knowledge can Improve Natural Language Commonsense Question Answering?cited by 22
  12. 2022Natural Language Deduction with Incomplete Informationcited by 12
1 more
  1. 2022A Property Induction Framework for Neural Language Modelscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 13; 2024: 7; 2025: 10; 2026: 2.

32 papers.

Sort:

By area:

  1. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  2. 2023Large Language Models for Automated Open-domain Scientific Hypotheses Discoverycited by 100
  3. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Surveycited by 83
  4. 2024MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypothesescited by 72
  5. 2023Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyondcited by 65
  6. 2023Towards Better Chain-of-Thought Prompting Strategies: A Surveycited by 62
  7. 2023Natural Language Reasoning, A Surveycited by 61
  8. 2023A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoningcited by 41
  9. 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Modelscited by 35
  10. 2024IQA-EVAL: Automatic Evaluation of Human-Model Interactive Question Answeringcited by 19
  11. 2023RECKONING: Reasoning through Dynamic Knowledge Encodingcited by 18
  12. 2025ChronoSense: Exploring Temporal Understanding in Large Language Models with Time Intervals of Eventscited by 18
20 more
  1. 2024Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoningcited by 13
  2. 2023Knowledge Engineering using Large Language Modelscited by 9
  3. 2025From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systemscited by 8
  4. 2023Conditions for Length Generalization in Learning Reasoning Skillscited by 7
  5. 2025HardcoreLogic: Challenging Large Reasoning Models with Long-tail Logic Puzzle Gamescited by 7
  6. 2025GEAR: A General Evaluation Framework for Abductive Reasoningcited by 6
  7. 2025From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Modelscited by 5
  8. 2026LogicSkills: A Structured Benchmark for Formal Reasoning in Large Language Modelscited by 4
  9. 2024Towards Goal-oriented Prompt Engineering for Large Language Models: A Surveycited by 3
  10. 2025Rule2Text: Natural Language Explanation of Logical Rules in Knowledge Graphscited by 3
  11. 2025ARCHE: A Novel Task to Evaluate LLMs on Latent Reasoning Chain Extractioncited by 3
  12. 2023Are LLMs Rigorous Logical Reasoners? Empowering Natural Language Proof Generation by Stepwise Decoding with Contrastive Learningcited by 2
  13. 2024Pyramid-Driven Alignment: Pyramid Principle Guided Integration of Large Language Models and Knowledge Graphscited by 2
  14. 2025Rule2Text: A Framework for Generating and Evaluating Natural Language Explanations of Knowledge Graph Rulescited by 2
  15. 2025Do You Get the Hint? Benchmarking LLMs on the Board Game Conceptcited by 2
  16. 2025Inverse Knowledge Search over Verifiable Reasoning: Synthesizing a Scientific Encyclopedia from a Long Chains-of-Thought Knowledge Basecited by 2
  17. 2024AILS-NTUA at SemEval-2024 Task 9: Cracking Brain Teasers: Transformer Models for Lateral Thinking Puzzlescited by 1
  18. 2026Where Reasoning Breaks: Logic-Aware Path Selection by Controlling Logical Connectives in LLMs Reasoning Chainscited by 1
  19. 2023Reasoning about Ambiguous Definite Descriptionscited by 0
  20. 2023Transformers in the Service of Description Logic-based Contextscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Logical Reasoning in Large Language Models: A Survey
  2. 2024Proceedings of the First International Workshop on Next-Generation Language Models for Knowledge Representation and Reasoning (NeLaMKRR 2024)
  3. 2026Knowledge Vector of Logical Reasoning in Large Language Models
  4. 2023Assessing Logical Reasoning Capabilities of Encoder-Only Transformer Models
  5. 2025Proceedings of the Second International Workshop on Next-Generation Language Models for Knowledge Representation and Reasoning (NeLaMKRR 2025)
  6. 2023Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
  7. 2024Transformer-based Language Models for Reasoning in the Description Logic ALCQ
  8. 2024Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic Approach