# Do Explanations make VQA Models more Predictable to a Human?

arXiv: https://arxiv.org/abs/1810.12366 · HTML: https://mlsanity.com/paper/1810.12366/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2018.

## Built on by (most cited first, 30 of 52)

- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions](https://mlsanity.com/paper/2209.03430/) (2022)
- [What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods](https://mlsanity.com/paper/2112.04417/) (2021)
- [Interpreting Language Models with Contrastive Explanations](https://mlsanity.com/paper/2202.10419/) (2022)
- [The effectiveness of feature attribution methods and its correlation with automatic evaluation scores](https://mlsanity.com/paper/2105.14944/) (2021)
- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- [Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanations](https://mlsanity.com/paper/2210.11584/) (2022)
- [Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations](https://mlsanity.com/paper/2307.08678/) (2023)
- [Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations](https://mlsanity.com/paper/2302.02187/) (2023)
- [In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Making](https://mlsanity.com/paper/2305.07722/) (2023)
- [A Review on Explainability in Multimodal Deep Neural Nets](https://mlsanity.com/paper/2105.07878/) (2021)
- [MultiViz: Towards Visualizing and Understanding Multimodal Models](https://mlsanity.com/paper/2207.00056/) (2022)
- [DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations](https://mlsanity.com/paper/2203.02013/) (2022)
- [Teaching Humans When To Defer to a Classifier via Exemplars](https://mlsanity.com/paper/2111.11297/) (2021)
- [A Meta-Analysis of the Utility of Explainable Artificial Intelligence in Human-AI Decision-Making](https://mlsanity.com/paper/2205.05126/) (2022)
- [Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2204.06916/) (2022)
- [Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations](https://mlsanity.com/paper/2112.09669/) (2021)
- [Connecting Attributions and QA Model Behavior on Realistic Counterfactuals](https://mlsanity.com/paper/2104.04515/) (2021)
- [Can Explanations Be Useful for Calibrating Black Box Models?](https://mlsanity.com/paper/2110.07586/) (2021)
- [Machine Explanations and Human Understanding](https://mlsanity.com/paper/2202.04092/) (2022)
- [Learning to Scaffold: Optimizing Model Explanations for Teaching](https://mlsanity.com/paper/2204.10810/) (2022)
- [Effective Human-AI Teams via Learned Natural Language Rules and Onboarding](https://mlsanity.com/paper/2311.01007/) (2023)
- [VisQA: X-raying Vision and Language Reasoning in Transformers](https://mlsanity.com/paper/2104.00926/) (2021)
- [Use-Case-Grounded Simulations for Explanation Evaluation](https://mlsanity.com/paper/2206.02256/) (2022)
- [Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalization](https://mlsanity.com/paper/2306.09299/) (2023)
- [Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods](https://mlsanity.com/paper/2210.07222/) (2022)
- [Are Human Explanations Always Helpful? Towards Objective Evaluation of Human Natural Language Explanations](https://mlsanity.com/paper/2305.03117/) (2023)
- [Improving Human Sequential Decision-Making with Reinforcement Learning](https://mlsanity.com/paper/2108.08454/) (2021)
- [Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources](https://mlsanity.com/paper/2305.06217/) (2023)
- [Utilizing Human Behavior Modeling to Manipulate Explanations in AI-Assisted Decision Making: The Good, the Bad, and the Scary](https://mlsanity.com/paper/2411.10461/) (2024)

## Abstract (arXiv)

A rich line of research attempts to make deep neural networks more transparent by generating human-interpretable 'explanations' of their decision process, especially for interactive tasks like Visual Question Answering (VQA). In this work, we analyze if existing explanations indeed make a VQA model -- its responses as well as failures -- more predictable to a human. Surprisingly, we find that they do not. On the other hand, we find that human-in-the-loop approaches that treat the model as a black-box do.