# The Role of Foundation Models in Neuro-Symbolic Learning and Reasoning arXiv: https://arxiv.org/abs/2402.01889 ยท HTML: https://mlsanity.com/paper/2402.01889/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. - Builds on 30 of its 48 references that are in the map. - First posted on arXiv 2 Feb 2024. ## Placed in - LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/) ## Builds on (most cited first, 30 of 30) - [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023) - [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022) - [ViperGPT: Visual Inference via Python Execution for Reasoning](https://mlsanity.com/paper/2303.08128/) (2023) - [Neural Probabilistic Logic Programming in DeepProbLog](https://mlsanity.com/paper/1907.08194/) (2019) - [Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly](https://mlsanity.com/paper/1911.03343/) (2019) - [Logic Tensor Networks](https://mlsanity.com/paper/2012.13635/) (2020) - [Neural-Symbolic Learning and Reasoning: A Survey and Interpretation](https://mlsanity.com/paper/1711.03902/) (2017) - [Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning](https://mlsanity.com/paper/1905.06088/) (2019) - [Hypothesis Search: Inductive Reasoning with Language Models](https://mlsanity.com/paper/2309.05660/) (2023) - [Logical Neural Networks](https://mlsanity.com/paper/2006.13155/) (2020) - [Block-Recurrent Transformers](https://mlsanity.com/paper/2203.07852/) (2022) - [Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations](https://mlsanity.com/paper/2011.12854/) (2020) - [NeurASP: Embracing Neural Networks into Answer Set Programming](https://mlsanity.com/paper/2307.07700/) (2023) - [Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning](https://mlsanity.com/paper/2107.02794/) (2021) - [PlantDoc: A Dataset for Visual Plant Disease Detection](https://mlsanity.com/paper/1911.10317/) (2019) - [Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts](https://mlsanity.com/paper/2305.19951/) (2023) - [Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Text](https://mlsanity.com/paper/2307.07696/) (2023) - [LAVIS: A Library for Language-Vision Intelligence](https://mlsanity.com/paper/2209.09019/) (2022) - [Abductive Knowledge Induction From Raw Data](https://mlsanity.com/paper/2010.03514/) (2020) - [NeuPSL: Neural Probabilistic Soft Logic](https://mlsanity.com/paper/2205.14268/) (2022) - [Leveraging Large Language Models to Generate Answer Set Programs](https://mlsanity.com/paper/2307.07699/) (2023) - [No Routing Needed Between Capsules](https://mlsanity.com/paper/2001.09136/) (2020) - [The ILASP system for Inductive Learning of Answer Set Programs](https://mlsanity.com/paper/2005.00904/) (2020) - [Deep Symbolic Learning: Discovering Symbols and Rules from Perceptions](https://mlsanity.com/paper/2208.11561/) (2022) - [FF-NSL: Feed-Forward Neural-Symbolic Learner](https://mlsanity.com/paper/2106.13103/) (2021) - [A Neuro-Symbolic ASP Pipeline for Visual Question Answering](https://mlsanity.com/paper/2205.07548/) (2022) - [Scalable Neural-Probabilistic Answer Set Programming](https://mlsanity.com/paper/2306.08397/) (2023) - [Neuro-Symbolic Learning of Answer Set Programs from Raw Data](https://mlsanity.com/paper/2205.12735/) (2022) - [Scalable Coupling of Deep Learning with Logical Reasoning](https://mlsanity.com/paper/2305.07617/) (2023) - [NeuralFastLAS: Fast Logic-Based Learning from Raw Data](https://mlsanity.com/paper/2310.05145/) (2023) ## Built on by (most cited first, 12 of 12) - [A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts](https://mlsanity.com/paper/2406.10368/) (2024) - [Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models](https://mlsanity.com/paper/2501.05382/) (2025) - [Can Large Language Models Act as Symbolic Reasoners?](https://mlsanity.com/paper/2410.21490/) (2024) - [Foundation Models for CPS-IoT: Opportunities and Challenges](https://mlsanity.com/paper/2501.16368/) (2025) - [The Road to Generalizable Neuro-Symbolic Learning Should be Paved with Foundation Models](https://mlsanity.com/paper/2505.24874/) (2025) - [Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunities](https://mlsanity.com/paper/2509.06921/) (2025) - [Predicate Renaming via Large Language Models](https://mlsanity.com/paper/2510.25517/) (2025) - [A Theoretical Survey on Foundation Models](https://mlsanity.com/paper/2410.11444/) (2024) - [Augmented Vision-Language Models: A Systematic Review](https://mlsanity.com/paper/2507.22933/) (2025) - [Visual Perceptual to Conceptual First-Order Rule Learning Networks](https://mlsanity.com/paper/2604.07897/) (2026) - [Accelerating NeurASP with vectorization and caching](https://mlsanity.com/paper/2606.10787/) (2026) - [Auto-Formalizing Neuro-Symbolic Predictors](https://mlsanity.com/paper/2610.01519/) (2026) ## Abstract (arXiv) Neuro-Symbolic AI (NeSy) holds promise to ensure the safe deployment of AI systems, as interpretable symbolic techniques provide formal behaviour guarantees. The challenge is how to effectively integrate neural and symbolic computation, to enable learning and reasoning from raw data. Existing pipelines that train the neural and symbolic components sequentially require extensive labelling, whereas end-to-end approaches are limited in terms of scalability, due to the combinatorial explosion in the symbol grounding problem. In this paper, we leverage the implicit knowledge within foundation models to enhance the performance in NeSy tasks, whilst reducing the amount of data labelling and manual engineering. We introduce a new architecture, called NeSyGPT, which fine-tunes a vision-language foundation model to extract symbolic features from raw data, before learning a highly expressive answer set program to solve a downstream task. Our comprehensive evaluation demonstrates that NeSyGPT has superior accuracy over various baselines, and can scale to complex NeSy tasks. Finally, we highlight the effective use of a large language model to generate the programmatic interface between the neural and symbolic components, significantly reducing the amount of manual engineering required.