# Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review

arXiv: https://arxiv.org/abs/2202.12205 · HTML: https://mlsanity.com/paper/2202.12205/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 62 of its 181 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 30 of 62)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Annotation Artifacts in Natural Language Inference Data](https://mlsanity.com/paper/1803.02324/) (2018)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Representation Learning on Graphs: Methods and Applications](https://mlsanity.com/paper/1709.05584/) (2017)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Neural Probabilistic Logic Programming in DeepProbLog](https://mlsanity.com/paper/1907.08194/) (2019)
- [The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision](https://mlsanity.com/paper/1904.12584/) (2019)
- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [Deep Learning for Symbolic Mathematics](https://mlsanity.com/paper/1912.01412/) (2019)
- [Neurosymbolic AI: The 3rd Wave](https://mlsanity.com/paper/2012.05876/) (2020)
- [SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization](https://mlsanity.com/paper/1911.03437/) (2019)
- [Deep Learning: A Critical Appraisal](https://mlsanity.com/paper/1801.00631/) (2018)
- [Learning Explanatory Rules from Noisy Data](https://mlsanity.com/paper/1711.04574/) (2017)
- [Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation](https://mlsanity.com/paper/1703.09902/) (2017)
- [Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems](https://mlsanity.com/paper/1903.12394/) (2019)
- [Neural-Symbolic Learning and Reasoning: A Survey and Interpretation](https://mlsanity.com/paper/1711.03902/) (2017)
- [Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning](https://mlsanity.com/paper/1905.06088/) (2019)
- [Neural Logic Machines](https://mlsanity.com/paper/1904.11694/) (2019)
- [Logical Neural Networks](https://mlsanity.com/paper/2006.13155/) (2020)
- [Case-based Reasoning for Natural Language Queries over Knowledge Bases](https://mlsanity.com/paper/2104.08762/) (2021)
- [The Cost of Training NLP Models: A Concise Overview](https://mlsanity.com/paper/2004.08900/) (2020)

## Built on by (most cited first, 30 of 38)

- [Reasoning with Language Model Prompting: A Survey](https://mlsanity.com/paper/2212.09597/) (2022)
- [Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions](https://mlsanity.com/paper/2310.19775/) (2023)
- [Large Language Models Can Learn Temporal Reasoning](https://mlsanity.com/paper/2401.06853/) (2024)
- [MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logic](https://mlsanity.com/paper/2305.03353/) (2023)
- [A Sentence is Worth a Thousand Pictures: Can Large Language Models Understand Hum4n L4ngu4ge and the W0rld behind W0rds?](https://mlsanity.com/paper/2308.00109/) (2023)
- [Interpretable Multimodal Misinformation Detection with Logic Reasoning](https://mlsanity.com/paper/2305.05964/) (2023)
- [SymbolicAI: A framework for logic-based approaches combining generative models and solvers](https://mlsanity.com/paper/2402.00854/) (2024)
- [Reliable Natural Language Understanding with Large Language Models and Answer Set Programming](https://mlsanity.com/paper/2302.03780/) (2023)
- [A Survey on Verification and Validation, Testing and Evaluations of Neurosymbolic Artificial Intelligence](https://mlsanity.com/paper/2401.03188/) (2024)
- [Natural Language Processing RELIES on Linguistics](https://mlsanity.com/paper/2405.05966/) (2024)
- [Neuro-Symbolic Learning: Principles and Applications in Ophthalmology](https://mlsanity.com/paper/2208.00374/) (2022)
- [Natural Language Commanding via Program Synthesis](https://mlsanity.com/paper/2306.03460/) (2023)
- [Conceptual Framework for Autonomous Cognitive Entities](https://mlsanity.com/paper/2310.06775/) (2023)
- [Neurosymbolic Graph Enrichment for Grounded World Models](https://mlsanity.com/paper/2411.12671/) (2024)
- [Dspy-based Neural-Symbolic Pipeline to Enhance Spatial Reasoning in LLMs](https://mlsanity.com/paper/2411.18564/) (2024)
- [Position: Foundation Models for Tabular Data within Systemic Contexts Need Grounding](https://mlsanity.com/paper/2505.19825/) (2025)
- [A Study on Neuro-Symbolic Artificial Intelligence: Healthcare Perspectives](https://mlsanity.com/paper/2503.18213/) (2025)
- [Large Process Models: A Vision for Business Process Management in the Age of Generative AI](https://mlsanity.com/paper/2309.00900/) (2023)
- [Take A Step Back: Rethinking the Two Stages in Visual Reasoning](https://mlsanity.com/paper/2407.19666/) (2024)
- [Toward Guarantees for Clinical Reasoning in Vision Language Models via Formal Verification](https://mlsanity.com/paper/2602.24111/) (2026)
- [Logical Tasks for Measuring Extrapolation and Rule Comprehension](https://mlsanity.com/paper/2211.07727/) (2022)
- [What comes after transformers? -- A selective survey connecting ideas in deep learning](https://mlsanity.com/paper/2408.00386/) (2024)
- [Trust in Language Grounding: a new AI challenge for human-robot teams](https://mlsanity.com/paper/2209.02066/) (2022)
- [Empirical Investigation of Neural Symbolic Reasoning Strategies](https://mlsanity.com/paper/2302.08148/) (2023)
- [Synergizing Machine Learning & Symbolic Methods: A Survey on Hybrid Approaches to Natural Language Processing](https://mlsanity.com/paper/2401.11972/) (2024)
- [First Heuristic Then Rational: Dynamic Use of Heuristics in Language Model Reasoning](https://mlsanity.com/paper/2406.16078/) (2024)
- [The American Sign Language Knowledge Graph: Infusing ASL Models with Linguistic Knowledge](https://mlsanity.com/paper/2411.03568/) (2024)
- [NSFlow: An End-to-End FPGA Framework with Scalable Dataflow Architecture for Neuro-Symbolic AI](https://mlsanity.com/paper/2504.19323/) (2025)
- [Recent Advances and Future Directions in Literature-Based Discovery](https://mlsanity.com/paper/2506.12385/) (2025)
- [In Transformer We Trust? A Perspective on Transformer Architecture Failure Modes](https://mlsanity.com/paper/2602.14318/) (2026)

## Abstract (arXiv)

Advocates for Neuro-Symbolic Artificial Intelligence (NeSy) assert that combining deep learning with symbolic reasoning will lead to stronger AI than either paradigm on its own. As successful as deep learning has been, it is generally accepted that even our best deep learning systems are not very good at abstract reasoning. And since reasoning is inextricably linked to language, it makes intuitive sense that Natural Language Processing (NLP), would be a particularly well-suited candidate for NeSy. We conduct a structured review of studies implementing NeSy for NLP, with the aim of answering the question of whether NeSy is indeed meeting its promises: reasoning, out-of-distribution generalization, interpretability, learning and reasoning from small data, and transferability to new domains. We examine the impact of knowledge representation, such as rules and semantic networks, language structure and relational structure, and whether implicit or explicit reasoning contributes to higher promise scores. We find that systems where logic is compiled into the neural network lead to the most NeSy goals being satisfied, while other factors such as knowledge representation, or type of neural architecture do not exhibit a clear correlation with goals being met. We find many discrepancies in how reasoning is defined, specifically in relation to human level reasoning, which impact decisions about model architectures and drive conclusions which are not always consistent across studies. Hence we advocate for a more methodical approach to the application of theories of human reasoning as well as the development of appropriate benchmarks, which we hope can lead to a better understanding of progress in the field. We make our data and code available on github for further analysis.