# Integrating Knowledge and Reasoning in Image Understanding

arXiv: https://arxiv.org/abs/1906.09954 · HTML: https://mlsanity.com/paper/1906.09954/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2019.

## Built on by (most cited first, 12 of 12)

- [Faster-LTN: a neuro-symbolic, end-to-end object detection architecture](https://mlsanity.com/paper/2107.01877/) (2021)
- [Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'](https://mlsanity.com/paper/2505.04950/) (2025)
- [A Survey on Visual Transfer Learning using Knowledge Graphs](https://mlsanity.com/paper/2201.11794/) (2022)
- [Automatic Modeling of Social Concepts Evoked by Art Images as Multimodal Frames](https://mlsanity.com/paper/2110.07420/) (2021)
- [Seeing the Intangible: Survey of Image Classification into High-Level and Abstract Categories](https://mlsanity.com/paper/2308.10562/) (2023)
- [Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding](https://mlsanity.com/paper/2508.20765/) (2025)
- [Deep Algorithmic Question Answering: Towards a Compositionally Hybrid AI for Algorithmic Reasoning](https://mlsanity.com/paper/2109.08006/) (2021)
- [Parallel Neurosymbolic Integration with Concordia](https://mlsanity.com/paper/2306.00480/) (2023)
- [Plants Don't Walk on the Street: Common-Sense Reasoning for Reliable Semantic Segmentation](https://mlsanity.com/paper/2104.09254/) (2021)
- [Stitching Gaps: Fusing Situated Perceptual Knowledge with Vision Transformers for High-Level Image Classification](https://mlsanity.com/paper/2402.19339/) (2024)
- [Augmented Vision-Language Models: A Systematic Review](https://mlsanity.com/paper/2507.22933/) (2025)
- [NLKI: A lightweight Natural Language Knowledge Integration Framework for Improving Small VLMs in Commonsense VQA Tasks](https://mlsanity.com/paper/2508.19724/) (2025)

## Abstract (arXiv)

Deep learning based data-driven approaches have been successfully applied in various image understanding applications ranging from object recognition, semantic segmentation to visual question answering. However, the lack of knowledge integration as well as higher-level reasoning capabilities with the methods still pose a hindrance. In this work, we present a brief survey of a few representative reasoning mechanisms, knowledge integration methods and their corresponding image understanding applications developed by various groups of researchers, approaching the problem from a variety of angles. Furthermore, we discuss upon key efforts on integrating external knowledge with neural networks. Taking cues from these efforts, we conclude by discussing potential pathways to improve reasoning capabilities.