# A Neural-Symbolic Approach to Natural Language Understanding

arXiv: https://arxiv.org/abs/2203.10557 · HTML: https://mlsanity.com/paper/2203.10557/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 23 of its 31 references that are in the map.
- First posted on arXiv 20 Mar 2022.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [Bidirectional Attention Flow for Machine Comprehension](https://mlsanity.com/paper/1611.01603/) (2016)
- [Thinking Fast and Slow with Deep Learning and Tree Search](https://mlsanity.com/paper/1705.08439/) (2017)
- [QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension](https://mlsanity.com/paper/1804.09541/) (2018)
- [The Consciousness Prior](https://mlsanity.com/paper/1709.08568/) (2017)
- [Neural Module Networks for Reasoning over Text](https://mlsanity.com/paper/1912.04971/) (2019)
- [NumNet: Machine Reading Comprehension with Numerical Reasoning](https://mlsanity.com/paper/1910.06701/) (2019)
- [Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehension](https://mlsanity.com/paper/1909.00109/) (2019)
- [EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference](https://mlsanity.com/paper/1901.03735/) (2019)
- [Question Directed Graph Attention Network for Numerical Reasoning over Text](https://mlsanity.com/paper/2009.07448/) (2020)
- [ConjNLI: Natural Language Inference Over Conjunctive Sentences](https://mlsanity.com/paper/2010.10418/) (2020)
- [Deep Reasoning Networks: Thinking Fast and Slow](https://mlsanity.com/paper/1906.00855/) (2019)
- [Thinking, Fast and Slow: Combining Vector Spaces and Knowledge Graphs](https://mlsanity.com/paper/1708.03310/) (2017)
- [DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference](https://mlsanity.com/paper/1802.05577/) (2018)

## Built on by (most cited first, 8 of 8)

- [Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinking](https://mlsanity.com/paper/2502.12470/) (2025)
- [Unlocking the Potential of Generative AI through Neuro-Symbolic Architectures: Benefits and Limitations](https://mlsanity.com/paper/2502.11269/) (2025)
- [A Neural Divide-and-Conquer Reasoning Framework for Image Retrieval from Linguistically Complex Text](https://mlsanity.com/paper/2305.02265/) (2023)
- [Take A Step Back: Rethinking the Two Stages in Visual Reasoning](https://mlsanity.com/paper/2407.19666/) (2024)
- [Bridging Social Psychology and LLM Reasoning: Conflict-Aware Meta-Review Generation via Cognitive Alignment](https://mlsanity.com/paper/2503.13879/) (2025)
- [Neurocognitive Modeling for Text Generation: Deep Learning Architecture for EEG Data](https://mlsanity.com/paper/2509.07202/) (2025)
- [Bridging the Gap: Representation Spaces in Neuro-Symbolic AI](https://mlsanity.com/paper/2411.04393/) (2024)
- [Hybrid Learners Do Not Forget: A Brain-Inspired Neuro-Symbolic Approach to Continual Learning](https://mlsanity.com/paper/2503.12635/) (2025)

## Abstract (arXiv)

Deep neural networks, empowered by pre-trained language models, have achieved remarkable results in natural language understanding (NLU) tasks. However, their performances can drastically deteriorate when logical reasoning is needed. This is because NLU in principle depends on not only analogical reasoning, which deep neural networks are good at, but also logical reasoning. According to the dual-process theory, analogical reasoning and logical reasoning are respectively carried out by System 1 and System 2 in the human brain. Inspired by the theory, we present a novel framework for NLU called Neural-Symbolic Processor (NSP), which performs analogical reasoning based on neural processing and logical reasoning based on both neural and symbolic processing. As a case study, we conduct experiments on two NLU tasks, question answering (QA) and natural language inference (NLI), when numerical reasoning (a type of logical reasoning) is necessary. The experimental results show that our method significantly outperforms state-of-the-art methods in both tasks.