# Interpretable Visual Reasoning via Induced Symbolic Space

arXiv: https://arxiv.org/abs/2011.11603 · HTML: https://mlsanity.com/paper/2011.11603/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2020.

## Built on by (most cited first, 10 of 10)

- [MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding](https://mlsanity.com/paper/2104.12763/) (2021)
- [Versatile Diffusion: Text, Images and Variations All in One Diffusion Model](https://mlsanity.com/paper/2211.08332/) (2022)
- [CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding](https://mlsanity.com/paper/2203.00680/) (2022)
- [Hadamard product in deep learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2504.13112/) (2025)
- [A Comprehensive Survey on Visual Question Answering Datasets and Algorithms](https://mlsanity.com/paper/2411.11150/) (2024)
- [Neuro-Symbolic AI for Military Applications](https://mlsanity.com/paper/2408.09224/) (2024)
- [Visual Superordinate Abstraction for Robust Concept Learning](https://mlsanity.com/paper/2205.14444/) (2022)
- [Enhancing Interpretability and Interactivity in Robot Manipulation: A Neurosymbolic Approach](https://mlsanity.com/paper/2210.00858/) (2022)
- [Towards Unsupervised Visual Reasoning: Do Off-The-Shelf Features Know How to Reason?](https://mlsanity.com/paper/2212.10292/) (2022)
- [Feudal Reinforcement Learning by Reading Manuals](https://mlsanity.com/paper/2110.06477/) (2021)

## Abstract (arXiv)

We study the problem of concept induction in visual reasoning, i.e., identifying concepts and their hierarchical relationships from question-answer pairs associated with images; and achieve an interpretable model via working on the induced symbolic concept space. To this end, we first design a new framework named object-centric compositional attention model (OCCAM) to perform the visual reasoning task with object-level visual features. Then, we come up with a method to induce concepts of objects and relations using clues from the attention patterns between objects' visual features and question words. Finally, we achieve a higher level of interpretability by imposing OCCAM on the objects represented in the induced symbolic concept space. Our model design makes this an easy adaption via first predicting the concepts of objects and relations and then projecting the predicted concepts back to the visual feature space so the compositional reasoning module can process normally. Experiments on the CLEVR and GQA datasets demonstrate: 1) our OCCAM achieves a new state of the art without human-annotated functional programs; 2) our induced concepts are both accurate and sufficient as OCCAM achieves an on-par performance on objects represented either in visual features or in the induced symbolic concept space.