# Show Why the Answer is Correct! Towards Explainable AI using Compositional Temporal Attention

arXiv: https://arxiv.org/abs/2105.07141 · HTML: https://mlsanity.com/paper/2105.07141/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 35 references that are in the map.
- First posted on arXiv 15 May 2021.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Unified Vision-Language Pre-Training for Image Captioning and VQA](https://mlsanity.com/paper/1909.11059/) (2019)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [Stacked Attention Networks for Image Question Answering](https://mlsanity.com/paper/1511.02274/) (2015)
- [Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey](https://mlsanity.com/paper/2006.11371/) (2020)
- [Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding](https://mlsanity.com/paper/1606.01847/) (2016)
- [RSVQA: Visual Question Answering for Remote Sensing Data](https://mlsanity.com/paper/2003.07333/) (2020)
- [Counterfactual Samples Synthesizing for Robust Visual Question Answering](https://mlsanity.com/paper/2003.06576/) (2020)
- [Learning to Reason: End-to-End Module Networks for Visual Question Answering](https://mlsanity.com/paper/1704.05526/) (2017)
- [A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input](https://mlsanity.com/paper/1410.0210/) (2014)
- [Modeling Relationships in Referential Expressions with Compositional Modular Networks](https://mlsanity.com/paper/1611.09978/) (2016)
- [An Analysis of Visual Question Answering Algorithms](https://mlsanity.com/paper/1703.09684/) (2017)
- [Learning to Compose Neural Networks for Question Answering](https://mlsanity.com/paper/1601.01705/) (2016)
- [Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challenge](https://mlsanity.com/paper/1708.02711/) (2017)
- [VQA-LOL: Visual Question Answering under the Lens of Logic](https://mlsanity.com/paper/2002.08325/) (2020)
- [Show, Ask, Attend, and Answer: A Strong Baseline For Visual Question Answering](https://mlsanity.com/paper/1704.03162/) (2017)
- [Semantic Equivalent Adversarial Data Augmentation for Visual Question Answering](https://mlsanity.com/paper/2007.09592/) (2020)
- [Simple Baseline for Visual Question Answering](https://mlsanity.com/paper/1512.02167/) (2015)
- [Learning from Few Samples: A Survey](https://mlsanity.com/paper/2007.15484/) (2020)
- [Visual Madlibs: Fill in the blank Image Generation and Question Answering](https://mlsanity.com/paper/1506.00278/) (2015)
- [Robust Explanations for Visual Question Answering](https://mlsanity.com/paper/2001.08730/) (2020)
- [Human Action Performance using Deep Neuro-Fuzzy Recurrent Attention Model](https://mlsanity.com/paper/2001.10953/) (2020)
- [Adaptive Clustering of Robust Semantic Representations for Adversarial Image Purification](https://mlsanity.com/paper/2104.02155/) (2021)

## Built on by (most cited first, 2 of 2)

- [Visual Robustness Benchmark for Visual Question Answering (VQA)](https://mlsanity.com/paper/2407.03386/) (2024)
- [Seeing the roads through the trees: A benchmark for modeling spatial dependencies with aerial imagery](https://mlsanity.com/paper/2401.06762/) (2024)

## Abstract (arXiv)

Visual Question Answering (VQA) models have achieved significant success in recent times. Despite the success of VQA models, they are mostly black-box models providing no reasoning about the predicted answer, thus raising questions for their applicability in safety-critical such as autonomous systems and cyber-security. Current state of the art fail to better complex questions and thus are unable to exploit compositionality. To minimize the black-box effect of these models and also to make them better exploit compositionality, we propose a Dynamic Neural Network (DMN), which can understand a particular question and then dynamically assemble various relatively shallow deep learning modules from a pool of modules to form a network. We incorporate compositional temporal attention to these deep learning based modules to increase compositionality exploitation. This results in achieving better understanding of complex questions and also provides reasoning as to why the module predicts a particular answer. Experimental analysis on the two benchmark datasets, VQA2.0 and CLEVR, depicts that our model outperforms the previous approaches for Visual Question Answering task as well as provides better reasoning, thus making it reliable for mission critical applications like safety and security.