# A Review on Explainability in Multimodal Deep Neural Nets

arXiv: https://arxiv.org/abs/2105.07878 · HTML: https://mlsanity.com/paper/2105.07878/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 144 of its 268 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 30 of 144)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)

## Built on by (most cited first, 30 of 46)

- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [Large Language Models in Law: A Survey](https://mlsanity.com/paper/2312.03718/) (2023)
- [Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities](https://mlsanity.com/paper/2111.06420/) (2021)
- [Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations](https://mlsanity.com/paper/2210.11584/) (2022)
- [Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP](https://mlsanity.com/paper/2308.14179/) (2023)
- [Diagnosing and Rectifying Vision Models using Language](https://mlsanity.com/paper/2302.04269/) (2023)
- [Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions](https://mlsanity.com/paper/2107.13782/) (2021)
- [Review of multimodal machine learning approaches in healthcare](https://mlsanity.com/paper/2402.02460/) (2024)
- [Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directions](https://mlsanity.com/paper/2306.05731/) (2023)
- [A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical Applications](https://mlsanity.com/paper/2408.02686/) (2024)
- [Fairness and Bias in Multimodal AI: A Survey](https://mlsanity.com/paper/2406.19097/) (2024)
- [Multimodal Explainability via Latent Shift applied to COVID-19 stratification](https://mlsanity.com/paper/2212.14084/) (2022)
- [A Survey on Safe Multi-Modal Learning System](https://mlsanity.com/paper/2402.05355/) (2024)
- [MultiModN- Multimodal, Multi-Task, Interpretable Modular Networks](https://mlsanity.com/paper/2309.14118/) (2023)
- [NERIF: GPT-4V for Automatic Scoring of Drawn Models](https://mlsanity.com/paper/2311.12990/) (2023)
- [Multi-objective optimization determines when, which and how to fuse deep networks: an application to predict COVID-19 outcomes](https://mlsanity.com/paper/2204.03772/) (2022)
- [Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources](https://mlsanity.com/paper/2305.06217/) (2023)
- [HEROHE Challenge: assessing HER2 status in breast cancer without immunohistochemistry or in situ hybridization](https://mlsanity.com/paper/2111.04738/) (2021)
- [CrisisKAN: Knowledge-infused and Explainable Multimodal Attention Network for Crisis Event Classification](https://mlsanity.com/paper/2401.06194/) (2024)
- [Explainable Misinformation Detection Across Multiple Social Media Platforms](https://mlsanity.com/paper/2203.11724/) (2022)
- [On the Robustness of Explanations of Deep Neural Network Models: A Survey](https://mlsanity.com/paper/2211.04780/) (2022)
- [Employing Multimodal Machine Learning for Stress Detection](https://mlsanity.com/paper/2306.09385/) (2023)
- [Manifestations of Xenophobia in AI Systems](https://mlsanity.com/paper/2212.07877/) (2022)
- [IMAD: IMage-Augmented multi-modal Dialogue](https://mlsanity.com/paper/2305.10512/) (2023)
- [Edge-Based Multimodal Sensor Data Fusion with Vision Language Models (VLMs) for Real-time Autonomous Vehicle Accident Avoidance](https://mlsanity.com/paper/2508.01057/) (2025)
- [Is Multi-Modal Necessarily Better? Robustness Evaluation of Multi-modal Fake News Detection](https://mlsanity.com/paper/2206.08788/) (2022)
- [Interpretable multimodal sentiment analysis based on textual modality descriptions by using large-scale language models](https://mlsanity.com/paper/2305.06162/) (2023)
- [Causal Intersectionality and Dual Form of Gradient Descent for Multimodal Analysis: a Case Study on Hateful Memes](https://mlsanity.com/paper/2308.11585/) (2023)
- [MicroXercise: A Micro-Level Comparative and Explainable System for Remote Physical Therapy](https://mlsanity.com/paper/2408.11837/) (2024)
- [Emulating Quantum Dynamics with Neural Networks via Knowledge Distillation](https://mlsanity.com/paper/2203.10200/) (2022)

## Abstract (arXiv)

Artificial Intelligence techniques powered by deep neural nets have achieved much success in several application domains, most significantly and notably in the Computer Vision applications and Natural Language Processing tasks. Surpassing human-level performance propelled the research in the applications where different modalities amongst language, vision, sensory, text play an important role in accurate predictions and identification. Several multimodal fusion methods employing deep learning models are proposed in the literature. Despite their outstanding performance, the complex, opaque and black-box nature of the deep neural nets limits their social acceptance and usability. This has given rise to the quest for model interpretability and explainability, more so in the complex tasks involving multimodal AI methods. This paper extensively reviews the present literature to present a comprehensive survey and commentary on the explainability in multimodal deep neural nets, especially for the vision and language tasks. Several topics on multimodal AI and its applications for generic domains have been covered in this paper, including the significance, datasets, fundamental building blocks of the methods and techniques, challenges, applications, and future trends in this domain