mlsanityData to 7 Oct 2026

Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing

arXiv 2102.12060 ↗First posted 24 Feb 2021cs.CL cs.AI cs.LGcited by 126

Not on the map. Why?

Abstract (arXiv)

Explainable NLP (ExNLP) has increasingly focused on collecting human-annotated textual explanations. These explanations are used downstream in three ways: as data augmentation to improve performance on a predictive task, as supervision to train models to produce explanations for their predictions, and as a ground-truth to evaluate model-generated explanations. In this review, we identify 65 datasets with three predominant classes of textual explanations (highlights, free-text, and structured), organize the literature on annotating each type, identify strengths and shortcomings of existing collection methodologies, and give recommendations for collecting ExNLP datasets in the future.

Key facts. Cited by 126 papers in the map (data through 7 Oct 2026). Builds on 100 of its 172 references that are in the map. First posted on arXiv 24 Feb 2021.

Builds on

100 of 172 references.

References · 100

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  4. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  5. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  7. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
  8. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  9. 2018Datasheets for Datasetscited by 1,371
  10. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  11. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  12. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
38 more
  1. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  2. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  3. 2016The Mythos of Model Interpretabilitycited by 716
  4. 2018CoQA: A Conversational Question Answering Challengecited by 674
  5. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  6. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  7. 2019Attention is not not Explanationcited by 629
  8. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  9. 2019ELI5: Long Form Question Answeringcited by 567
  10. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  11. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  12. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  13. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  14. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  15. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  16. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  17. 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 403
  18. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  19. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  20. 2019Interpretable machine learning: definitions, methods, and applicationscited by 303
  21. 2019Abductive Commonsense Reasoningcited by 300
  22. 2018Hypothesis Only Baselines in Natural Language Inferencecited by 291
  23. 2016Rationalizing Neural Predictionscited by 281
  24. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 281
  25. 2020ToTTo: A Controlled Table-To-Text Generation Datasetcited by 267
  26. 2018Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negativescited by 261
  27. 2020Data and its (dis)contents: A survey of dataset development and use in machine learning researchcited by 259
  28. 2018Metrics for Explainable AI: Challenges and Prospectscited by 257
  29. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
  30. 2019Visual Entailment: A Novel Task for Fine-Grained Image Understandingcited by 250
  31. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 248
  32. 2018Textual Explanations for Self-Driving Vehiclescited by 226
  33. 2020A Survey on the Explainability of Supervised Machine Learningcited by 225
  34. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 219
  35. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 209
  36. 2019Racial Bias in Hate Speech and Abusive Language Detection Datasetscited by 208
  37. 2018Multimodal Explanations: Justifying Decisions and Pointing to the Evidencecited by 179
  38. 2019Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasetscited by 173

50 most cited of 100 All 100 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 26; 2023: 30; 2024: 29; 2025: 20; 2026: 7.

126 papers.

Sort:

By area:

  1. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  2. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  3. 2021Trustworthy AI: From Principles to Practicescited by 206
  4. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 195
  5. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  6. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  7. 2022Explanations from Large Language Models Make Small Reasoners Bettercited by 123
  8. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  9. 2024Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Modelscited by 104
  10. 2024Are self-explanations from Large Language Models faithful?cited by 101
  11. 2024Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?cited by 68
  12. 2022ProGen: Progressive Zero-shot Dataset Generation via In-context Feedbackcited by 67
38 more
  1. 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
  2. 2025Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Abovecited by 51
  3. 2023Rigorously Assessing Natural Language Explanations of Neuronscited by 44
  4. 2022Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvementcited by 37
  5. 2023Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-Text Rationalescited by 37
  6. 2024Verification and Refinement of Natural Language Explanations through LLM-Symbolic Theorem Provingcited by 35
  7. 2021Local Interpretations for Explainable Natural Language Processing: A Surveycited by 31
  8. 2021On the Diversity and Limits of Human Explanationscited by 30
  9. 2022Reward Gaming in Conditional Text Generationcited by 29
  10. 2024The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Modelscited by 29
  11. 2022E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoningcited by 28
  12. 2022ferret: a Framework for Benchmarking Explainers on Transformerscited by 26
  13. 2024Mechanistic?cited by 25
  14. 2024Properties and Challenges of LLM-Generated Explanationscited by 24
  15. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 24
  16. 2024A Comparative Study on Annotation Quality of Crowdsourcing and LLM via Label Aggregationcited by 23
  17. 2022REV: Information-Theoretic Evaluation of Free-Text Rationalescited by 22
  18. 2023MDACE: MIMIC Documents Annotated with Code Evidencecited by 19
  19. 2022Generic Temporal Reasoning with Differential Analysis and Explanationcited by 18
  20. 2022METGEN: A Module-Based Entailment Tree Generation Framework for Answer Explanationcited by 17
  21. 2021Inherently Explainable Reinforcement Learning in Natural Languagecited by 16
  22. 2023Faithful Question Answering with Monte-Carlo Planningcited by 16
  23. 2023Atomic Inference for NLI with Generated Facts as Atomscited by 16
  24. 2025LLMs Don't Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanationscited by 16
  25. 2022Towards Explainable Evaluation Metrics for Natural Language Generationcited by 15
  26. 2022A survey on improving NLP models with human explanationscited by 15
  27. 2022Exploring Faithful Rationale for Multi-hop Fact Verification via Salience-Aware Graph Learningcited by 15
  28. 2023Language Models with Rationalitycited by 15
  29. 2023Towards Explainable Evaluation Metrics for Machine Translationcited by 15
  30. 2021NAREOR: The Narrative Reordering Problemcited by 14
  31. 2021Explainable Machine Learning with Prior Knowledge: An Overviewcited by 14
  32. 2025Position-aware Automatic Circuit Discoverycited by 14
  33. 2022Investigating the Benefits of Free-Form Rationalescited by 13
  34. 2023Are Human Explanations Always Helpful? Towards Objective Evaluation of Human Natural Language Explanationscited by 13
  35. 2024Enhancing Systematic Decompositional Natural Language Inference Using Informal Logiccited by 12
  36. 2023Using Natural Language Explanations to Rescale Human Judgmentscited by 11
  37. 2023ALMANACS: A Simulatability Benchmark for Language Model Explainabilitycited by 11
  38. 2021Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehensioncited by 10

50 most cited of 126 All 126 (JSON)