mlsanityData to 7 Oct 2026

Did the Model Understand the Question?

arXiv 1805.05492 ↗First posted 14 May 2018cs.CL cs.AIcited by 60

Not on the map. Why?

Abstract (arXiv)

We analyze state-of-the-art deep learning models for three tasks: question answering on (1) images, (2) tables, and (3) passages of text. Using the notion of attribution (word importance), we find that these deep networks often ignore important question terms. Leveraging such behavior, we perturb questions to craft a variety of adversarial examples. Our strongest attacks drop the accuracy of a visual question answering model from 61.1% to 19%, and that of a tabular question answering model from 33.5% to 3.3%. Additionally, we show how attributions can strengthen attacks proposed by Jia and Liang (2017) on paragraph comprehension models. Our results demonstrate that attributions can augment standard measures of accuracy and empower investigation of model performance. When a model is accurate but for the wrong reasons, attributions can surface erroneous logic in the model that indicates inadequacies in the test data.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 25; 2022: 14; 2023: 9; 2024: 10; 2025: 2.

60 papers.

Sort:

By area:

  1. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
  2. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  3. 2021Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Modelscited by 78
  4. 2021"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classificationcited by 66
  5. 2021Attacking Open-domain Question Answering by Injecting Misinformationcited by 41
  6. 2021Why Machine Reading Comprehension Models Learn Shortcuts?cited by 40
  7. 2021Evaluating Saliency Methods for Neural Language Modelscited by 37
  8. 2021Local Interpretations for Explainable Natural Language Processing: A Surveycited by 31
  9. 2021Evaluating the Faithfulness of Importance Measures in NLP by Recursively Masking Allegedly Important Tokens and Retrainingcited by 31
  10. 2023ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writingcited by 31
  11. 2021Challenges in Generalization in Open Domain Question Answeringcited by 30
  12. 2023Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directionscited by 25
38 more
  1. 2021Does BERT Learn as Humans Perceive? Understanding Linguistic Styles through Lexicacited by 23
  2. 2021Disfl-QA: A Benchmark Dataset for Understanding Disfluencies in Question Answeringcited by 20
  3. 2024Revealing the Barriers of Language Agents in Planningcited by 20
  4. 2021GraghVQA: Language-Guided Graph Neural Networks for Graph-based Visual Question Answeringcited by 16
  5. 2021Text Compression-aided Transformer Encodingcited by 13
  6. 2022Learning Robust Representations for Continual Relation Extraction via Adversarial Class Augmentationcited by 13
  7. 2021Robustness Challenges in Model Distillation and Pruning for Natural Language Understandingcited by 12
  8. 2024Navigating the Shortcut Maze: A Comprehensive Analysis of Shortcut Learning in Text Classification by Language Modelscited by 10
  9. 2022Interpretation of Black Box NLP Models: A Surveycited by 9
  10. 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Futurecited by 9
  11. 2021Making Attention Mechanisms More Robust and Interpretable with Virtual Adversarial Trainingcited by 8
  12. 2021Speaker-Conditioned Hierarchical Modeling for Automated Speech Scoringcited by 8
  13. 2022Can Question Rewriting Help Conversational Question Answering?cited by 8
  14. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithmscited by 8
  15. 2024Desiderata for the Context Use of Question Answering Systemscited by 7
  16. 2021AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defensescited by 5
  17. 2022Explaining Translationese: why are Neural Classifiers Better and what do they Learn?cited by 5
  18. 2022On the Robustness of Explanations of Deep Neural Network Models: A Surveycited by 5
  19. 2022Debiasing Stance Detection Models with Counterfactual Reasoning and Adversarial Bias Learningcited by 5
  20. 2023Adapting Pretrained Language Models for Solving Tabular Prediction Problems in the Electronic Health Recordcited by 5
  21. 2023Faithfulness Measurable Masked Language Modelscited by 5
  22. 2022Robustness of Fusion-based Multimodal Classifiers to Cross-Modal Content Dilutionscited by 4
  23. 2021Explaining the Road Not Takencited by 3
  24. 2022Semantic-based Pre-training for Dialogue Understandingcited by 3
  25. 2022StyLEx: Explaining Style Using Human Lexical Annotationscited by 3
  26. 2023Consistent Multi-Granular Rationale Extraction for Explainable Multi-hop Fact Verificationcited by 3
  27. 2023Did the Models Understand Documents? Benchmarking Models for Language Understanding in Document-Level Relation Extractioncited by 3
  28. 2023Automated Testing and Improvement of Named Entity Recognition Systemscited by 3
  29. 2023A New Baseline Assumption of Integated Gradients Based on Shaply valuecited by 3
  30. 2024Counterfactuals As a Means for Evaluating Faithfulness of Attribution Methods in Autoregressive Language Modelscited by 3
  31. 2021Model Agnostic Answer Reranking System for Adversarial Question Answeringcited by 2
  32. 2021Interactive Machine Comprehension with Dynamic Knowledge Graphscited by 2
  33. 2023Mitigating Shortcuts in Language Models with Soft Label Encodingcited by 2
  34. 2024Machine Translation Testing via Syntactic Tree Pruningcited by 2
  35. 2024Towards Data-Centric Automatic R&Dcited by 2
  36. 2025Fact in Fragments: Deconstructing Complex Claims via LLM-based Atomic Fact Extraction and Verificationcited by 2
  37. 2022Towards Understanding and Harnessing the Effect of Image Transformation in Adversarial Detectioncited by 1
  38. 2025Tell Me Without Telling Me: Two-Way Prediction of Visualization Literacy and Visual Attentioncited by 1

50 most cited of 60 All 60 (JSON)