# Neuro-Symbolic AI: Explainability, Challenges, and Future Trends arXiv: https://arxiv.org/abs/2411.04383 ยท HTML: https://mlsanity.com/paper/2411.04383/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 114 of its 186 references that are in the map. - First posted on arXiv 7 Nov 2024. ## Builds on (most cited first, 30 of 114) - [Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning](https://mlsanity.com/paper/2305.12295/) (2023) - [The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision](https://mlsanity.com/paper/1904.12584/) (2019) - [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020) - [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019) - [Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding](https://mlsanity.com/paper/1810.02338/) (2018) - [A Semantic Loss Function for Deep Learning with Symbolic Knowledge](https://mlsanity.com/paper/1711.11157/) (2017) - [Deep Learning for Symbolic Mathematics](https://mlsanity.com/paper/1912.01412/) (2019) - [COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs](https://mlsanity.com/paper/2010.05953/) (2020) - [PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network](https://mlsanity.com/paper/1812.04426/) (2018) - [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021) - [Formal Security Analysis of Neural Networks using Symbolic Intervals](https://mlsanity.com/paper/1804.10829/) (2018) - [Learning to Plan Chemical Syntheses](https://mlsanity.com/paper/1708.04202/) (2017) - [SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver](https://mlsanity.com/paper/1905.12149/) (2019) - [Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference](https://mlsanity.com/paper/1506.02158/) (2015) - [MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning](https://mlsanity.com/paper/2205.00445/) (2022) - [Neural Logic Machines](https://mlsanity.com/paper/1904.11694/) (2019) - [ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory](https://mlsanity.com/paper/2306.03901/) (2023) - [Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment](https://mlsanity.com/paper/1611.03954/) (2016) - [Logical Neural Networks](https://mlsanity.com/paper/2006.13155/) (2020) - [Neural Variational Inference and Learning in Belief Networks](https://mlsanity.com/paper/1402.0030/) (2014) - [Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision](https://mlsanity.com/paper/1611.00020/) (2016) - [Semantic Probabilistic Layers for Neuro-Symbolic Learning](https://mlsanity.com/paper/2206.00426/) (2022) - [Symbolic Regression via Neural-Guided Genetic Programming Population Seeding](https://mlsanity.com/paper/2111.00053/) (2021) - [Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery](https://mlsanity.com/paper/1912.04825/) (2019) - [Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning](https://mlsanity.com/paper/2107.02794/) (2021) - [Compositional Generalization via Neural-Symbolic Stack Machines](https://mlsanity.com/paper/2008.06662/) (2020) - [SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning](https://mlsanity.com/paper/1811.00090/) (2018) - [Neural-Symbolic Models for Logical Queries on Knowledge Graphs](https://mlsanity.com/paper/2205.10128/) (2022) - [Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval](https://mlsanity.com/paper/2201.12431/) (2022) - [Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning](https://mlsanity.com/paper/2006.06649/) (2020) ## Built on by (most cited first, 10 of 10) - [AI Mother Tongue: Self-Emergent Communication in MARL via Endogenous Symbol Systems](https://mlsanity.com/paper/2507.10566/) (2025) - [A Unified Framework for Real-Time Failure Handling in Robotics Using Vision-Language Models, Reactive Planner and Behavior Trees](https://mlsanity.com/paper/2503.15202/) (2025) - [CoreThink: A Symbolic Reasoning Layer to reason over Long Horizon Tasks with LLMs](https://mlsanity.com/paper/2509.00971/) (2025) - [Vector Symbolic Algebras for the Abstraction and Reasoning Corpus](https://mlsanity.com/paper/2511.08747/) (2025) - [In Transformer We Trust? A Perspective on Transformer Architecture Failure Modes](https://mlsanity.com/paper/2602.14318/) (2026) - [Seeing Through Risk: A Symbolic Approximation of Prospect Theory](https://mlsanity.com/paper/2504.14448/) (2025) - [CLMN: Concept based Language Models via Neural Symbolic Reasoning](https://mlsanity.com/paper/2510.10063/) (2025) - [Multi-task GINN-LP for Multi-target Symbolic Regression](https://mlsanity.com/paper/2511.13463/) (2025) - [Neuro-Relational Programs: Unifying Queries and Neural Computation over Structured Data](https://mlsanity.com/paper/2606.11946/) (2026) - [Think-Verify-Revise: Neuro-Symbolic Visual Reasoning with Vision-Language Models and Dynamic Logic Tensor Networks](https://mlsanity.com/paper/2609.05388/) (2026) ## Abstract (arXiv) Explainability is an essential reason limiting the application of neural networks in many vital fields. Although neuro-symbolic AI hopes to enhance the overall explainability by leveraging the transparency of symbolic learning, the results are less evident than imagined. This article proposes a classification for explainability by considering both model design and behavior of 191 studies from 2013, focusing on neuro-symbolic AI, hoping to inspire scholars who want to understand the explainability of neuro-symbolic AI. Precisely, we classify them into five categories by considering whether the form of bridging the representation differences is readable as their design factor, if there are representation differences between neural networks and symbolic logic learning, and whether a model decision or prediction process is understandable as their behavior factor: implicit intermediate representations and implicit prediction, partially explicit intermediate representations and partially explicit prediction, explicit intermediate representations or explicit prediction, explicit intermediate representation and explicit prediction, unified representation and explicit prediction. We also analyzed the research trends and three significant challenges: unified representations, explainability and transparency, and sufficient cooperation from neural networks and symbolic learning. Finally, we put forward suggestions for future research in three aspects: unified representations, enhancing model explainability, ethical considerations, and social impact.