# Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering

arXiv: https://arxiv.org/abs/1902.07864 · HTML: https://mlsanity.com/paper/1902.07864/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2019.

## Built on by (most cited first, 30 of 38)

- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [MultiBench: Multiscale Benchmarks for Multimodal Representation Learning](https://mlsanity.com/paper/2107.07502/) (2021)
- [Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions](https://mlsanity.com/paper/2209.03430/) (2022)
- [Neuro-Symbolic Artificial Intelligence: Current Trends](https://mlsanity.com/paper/2105.05330/) (2021)
- [MultiViz: Towards Visualizing and Understanding Multimodal Models](https://mlsanity.com/paper/2207.00056/) (2022)
- [Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computing](https://mlsanity.com/paper/2210.15889/) (2022)
- [DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations](https://mlsanity.com/paper/2203.02013/) (2022)
- [Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directions](https://mlsanity.com/paper/2306.05731/) (2023)
- [Found a Reason for me? Weakly-supervised Grounded Visual Question Answering using Capsules](https://mlsanity.com/paper/2105.04836/) (2021)
- [Iterated learning for emergent systematicity in VQA](https://mlsanity.com/paper/2105.01119/) (2021)
- [DALL-E for Detection: Language-driven Compositional Image Synthesis for Object Detection](https://mlsanity.com/paper/2206.09592/) (2022)
- [Grounded Object Centric Learning](https://mlsanity.com/paper/2307.09437/) (2023)
- [LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoning](https://mlsanity.com/paper/2309.13556/) (2023)
- [Softened Symbol Grounding for Neuro-symbolic Systems](https://mlsanity.com/paper/2403.00323/) (2024)
- [Visual Knowledge in the Big Model Era: Retrospect and Prospect](https://mlsanity.com/paper/2404.04308/) (2024)
- [Fact-R1: Towards Explainable Video Misinformation Detection with Deep Reasoning](https://mlsanity.com/paper/2505.16836/) (2025)
- [A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answering](https://mlsanity.com/paper/2201.05299/) (2022)
- [Domain-robust VQA with diverse datasets and methods but no target labels](https://mlsanity.com/paper/2103.15974/) (2021)
- [Beyond Generation: Harnessing Text to Image Models for Object Detection and Segmentation](https://mlsanity.com/paper/2309.05956/) (2023)
- [Neuro-Symbolic AI: Explainability, Challenges, and Future Trends](https://mlsanity.com/paper/2411.04383/) (2024)
- [Improving Visual Question Answering Models through Robustness Analysis and In-Context Learning with a Chain of Basic Questions](https://mlsanity.com/paper/2304.03147/) (2023)
- [V-LoL: A Diagnostic Dataset for Visual Logical Learning](https://mlsanity.com/paper/2306.07743/) (2023)
- [Neuro-Symbolic Learning: Principles and Applications in Ophthalmology](https://mlsanity.com/paper/2208.00374/) (2022)
- [Neural-Symbolic VideoQA: Learning Compositional Spatio-Temporal Reasoning for Real-world Video Question Answering](https://mlsanity.com/paper/2404.04007/) (2024)
- [An Interpretable Neuro-Symbolic Reasoning Framework for Task-Oriented Dialogue Generation](https://mlsanity.com/paper/2203.05843/) (2022)
- [A Study on Neuro-Symbolic Artificial Intelligence: Healthcare Perspectives](https://mlsanity.com/paper/2503.18213/) (2025)
- [Neuro-Symbolic VQA: A review from the perspective of AGI desiderata](https://mlsanity.com/paper/2104.06365/) (2021)
- [LiVLR: A Lightweight Visual-Linguistic Reasoning Framework for Video Question Answering](https://mlsanity.com/paper/2111.14547/) (2021)
- [Brainish: Formalizing A Multimodal Language for Intelligence and Consciousness](https://mlsanity.com/paper/2205.00001/) (2022)
- [Closed Loop Interactive Embodied Reasoning for Robot Manipulation](https://mlsanity.com/paper/2404.15194/) (2024)

## Abstract (arXiv)

We propose a new class of probabilistic neural-symbolic models, that have symbolic functional programs as a latent, stochastic variable. Instantiated in the context of visual question answering, our probabilistic formulation offers two key conceptual advantages over prior neural-symbolic models for VQA. Firstly, the programs generated by our model are more understandable while requiring lesser number of teaching examples. Secondly, we show that one can pose counterfactual scenarios to the model, to probe its beliefs on the programs that could lead to a specified answer given an image. Our results on the CLEVR and SHAPES datasets verify our hypotheses, showing that the model gets better program (and answer) prediction accuracy even in the low data regime, and allows one to probe the coherence and consistency of reasoning performed.