mlsanityData to 7 Oct 2026

Explanation-Based Human Debugging of NLP Models: A Survey

arXiv 2104.15135 ↗First posted 30 Apr 2021cs.CL cs.AI cs.HCcited by 53

Not on the map. Why?

Abstract (arXiv)

Debugging a machine learning model is hard since the bug usually involves the training data and the learning process. This becomes even harder for an opaque deep learning model if we have no clue about how the model actually works. In this survey, we review papers that exploit explanations to enable humans to give feedback and debug NLP models. We call this problem explanation-based human debugging (EBHD). In particular, we categorize and discuss existing work along three dimensions of EBHD (the bug context, the workflow, and the experimental setting), compile findings on how EBHD components affect the feedback providers, and highlight open problems that could be future research directions.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 54 of its 115 references that are in the map. First posted on arXiv 30 Apr 2021.

Builds on

54 of 115 references.

References · 54

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  7. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  8. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  9. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  10. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  11. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  12. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
38 more
  1. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  2. 2016The Mythos of Model Interpretabilitycited by 716
  3. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  4. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  5. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  6. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  7. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  8. 2015Visual7W: Grounded Question Answering in Imagescited by 315
  9. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 282
  10. 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
  11. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  12. 2019Explainable Machine Learning in Deploymentcited by 210
  13. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  14. 2019Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systemscited by 203
  15. 2020Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AIcited by 166
  16. 2019One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniquescited by 164
  17. 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
  18. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  19. 2020Debugging Tests for Model Explanationscited by 146
  20. 2019Interpretations are useful: penalizing explanations to align neural networks with prior knowledgecited by 144
  21. 2020Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functionscited by 144
  22. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 141
  23. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  24. 2020FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debuggingcited by 120
  25. 2020"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humanscited by 86
  26. 2019The relationship between trust in AI and trustworthy machine learning technologiescited by 78
  27. 2019Interpretable and Steerable Sequence Learning via Prototypescited by 72
  28. 2018How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanationcited by 71
  29. 2018Interpreting Black Box Predictions using Fisher Kernelscited by 48
  30. 2020FIND: Human-in-the-Loop Debugging Deep Text Classifierscited by 40
  31. 2021Putting Humans in the Natural Language Processing Loop: A Surveycited by 40
  32. 2016Explaining Predictions of Non-Linear Classifiers in NLPcited by 37
  33. 2019exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Modelscited by 33
  34. 2019Human-grounded Evaluations of Explanation Methods for Text Classificationcited by 28
  35. 2021Refining Language Models with Compositional Explanationscited by 27
  36. 2020Soliciting Human-in-the-Loop User Feedback for Interactive Machine Learning Reduces User Trust and Impressions of Model Accuracycited by 19
  37. 2020Machine Guides, Human Supervises: Interactive Learning with Global Explanationscited by 16
  38. 2017Developing Bug-Free Machine Learning Systems With Formal Mathematicscited by 13

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 19; 2023: 15; 2024: 10; 2025: 3.

53 papers.

Sort:

By area:

  1. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  2. 2022Can language models learn from explanations in context?cited by 248
  3. 2021Towards a Science of Human-AI Decision Making: A Survey of Empirical Studiescited by 85
  4. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  5. 2022Concept-level Debugging of Part-Prototype Networkscited by 52
  6. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 52
  7. 2023Disentanglement via Latent Quantizationcited by 40
  8. 2022On the Explainability of Natural Language Processing Deep Modelscited by 37
  9. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  10. 2022Going Beyond XAI: A Systematic Survey for Explanation-Guided Learningcited by 35
  11. 2023ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writingcited by 31
  12. 2022A Rationale-Centric Framework for Human-in-the-loop Machine Learningcited by 27
38 more
  1. 2021What to Learn, and How: Toward Effective Learning from Rationalescited by 25
  2. 2024Properties and Challenges of LLM-Generated Explanationscited by 24
  3. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 24
  4. 2022A Survey of Adversarial Defences and Robustness in NLPcited by 22
  5. 2022Challenges in Applying Explainability Methods to Improve the Fairness of NLP Modelscited by 20
  6. 2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
  7. 2021Jointly Attacking Graph Neural Network and its Explanationscited by 18
  8. 2022Towards Explainable Evaluation Metrics for Natural Language Generationcited by 15
  9. 2022A survey on improving NLP models with human explanationscited by 15
  10. 2023Interpretation of Time-Series Deep Models: A Surveycited by 15
  11. 2023Towards Explainable Evaluation Metrics for Machine Translationcited by 15
  12. 2024Helpful or Harmful Data? Fine-tuning-free Shapley Attribution for Explaining Language Model Predictionscited by 13
  13. 2024Contestable AI needs Computational Argumentationcited by 12
  14. 2023QualEval: Qualitative Evaluation for Model Improvementcited by 11
  15. 2023ALMANACS: A Simulatability Benchmark for Language Model Explainabilitycited by 11
  16. 2023Are Your Explanations Reliable? Investigating the Stability of LIME in Explaining Text Classifiers by Marrying XAI and Adversarial Attackcited by 10
  17. 2022Human-Centric Research for NLP: Towards a Definition and Guiding Questionscited by 9
  18. 2023Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AIcited by 9
  19. 2022XMD: An End-to-End Framework for Interactive Explanation-Based Debugging of NLP Modelscited by 8
  20. 2023IFAN: An Explainability-Focused Interaction Framework for Humans and NLP Modelscited by 8
  21. 2024To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systemscited by 7
  22. 2022Causal Discovery and Knowledge Injection for Contestable Neural Networks (with Appendices)cited by 6
  23. 2024CoXQL: A Dataset for Parsing Explanation Requests in Conversational XAI Systemscited by 5
  24. 2022Cascading Biases: Investigating the Effect of Heuristic Annotation Strategies on Data and Modelscited by 4
  25. 2022Understanding Text Classification Data and Models Using Aggregated Input Saliencecited by 2
  26. 2023What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perceptioncited by 2
  27. 2024Facilitating Opinion Diversity through Hybrid NLP Approachescited by 2
  28. 2024Joint Universal Adversarial Perturbations with Interpretationscited by 2
  29. 2025Attribution Explanations for Deep Neural Networks: A Theoretical Perspectivecited by 2
  30. 2021Combining Transformers with Natural Language Explanationscited by 1
  31. 2022SparCAssist: A Model Risk Assessment Assistant Based on Sparse Generated Counterfactualscited by 1
  32. 2022Argumentative Explanations for Pattern-Based Text Classifierscited by 1
  33. 2023Wrapper Boxes: Faithful Attribution of Model Predictions to Training Datacited by 1
  34. 2024A look under the hood of the Interactive Deep Learning Enterprise (No-IDLE)cited by 1
  35. 2022Explainability of Text Processing and Retrieval Methods: A Surveycited by 0
  36. 2023Cross-lingual German Biomedical Information Extraction: from Zero-shot to Human-in-the-Loopcited by 0
  37. 2023Multi-resolution Interpretation and Diagnostics Tool for Natural Language Classifierscited by 0
  38. 2023Explanation-based Training with Differentiable Insertion/Deletion Metric-aware Regularizerscited by 0

50 most cited of 53 All 53 (JSON)