mlsanityData to 14 Oct 2026

Combining Transformers with Natural Language Explanations

arXiv 2110.00125 ↗First posted 2 Sep 2021cs.CL cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Many NLP applications require models to be interpretable. However, many successful neural architectures, including transformers, still lack effective interpretation methods. A possible solution could rely on building explanations from domain knowledge, which is often available as plain, natural language text. We thus propose an extension to transformer models that makes use of external memories to store natural language explanations and use them to explain classification outputs. We conduct an experimental evaluation on two domains, legal text analysis and argument mining, to show that our approach can produce relevant explanations while retaining or even improving classification performance.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 46 of its 74 references that are in the map. First posted on arXiv 2 Sep 2021.

Builds on

46 of 74 references.

References · 46

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  4. 2020Longformer: The Long-Document Transformercited by 3,413
  5. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  6. 2020Reformer: The Efficient Transformercited by 1,668
  7. 2015Prioritized Experience Replaycited by 1,099
  8. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
  9. 2019Attention is not Explanationcited by 929
  10. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  11. 2014Neural Turing Machinescited by 644
  12. 2019Attention is not not Explanationcited by 636
34 more
  1. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  2. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 530
  3. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  4. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  5. 2014Memory Networkscited by 363
  6. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  7. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 267
  8. 2021Internet-Augmented Dialogue Generationcited by 217
  9. 2019Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systemscited by 205
  10. 2016Harnessing Deep Neural Networks with Logic Rulescited by 135
  11. 2019Attention in Natural Language Processingcited by 101
  12. 2021An Interpretability Illusion for BERTcited by 97
  13. 2019Augmenting Self-attention with Persistent Memorycited by 93
  14. 2020Learning from Task Descriptionscited by 78
  15. 2020Memory Transformercited by 76
  16. 2018CLAUDETTE: an Automated Detector of Potentially Unfair Clauses in Online Terms of Servicecited by 73
  17. 2018Linguistically-Informed Self-Attention for Semantic Role Labelingcited by 71
  18. 2018Deriving Machine Attention from Human Rationalescited by 54
  19. 2021Explanation-Based Human Debugging of NLP Models: A Surveycited by 53
  20. 2019Metalearned Neural Memorycited by 44
  21. 2020WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challengecited by 43
  22. 2020Meta-learning for Few-shot Natural Language Processing: A Surveycited by 42
  23. 2018Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Insteadcited by 40
  24. 2019Human-Centered Artificial Intelligence and Machine Learningcited by 37
  25. 2020Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledgecited by 37
  26. 2017Exploring Human-like Attention Supervision in Visual Question Answeringcited by 36
  27. 2021Order in the Court: Explainable AI Methods Prone to Disagreementcited by 36
  28. 2020Semantics of the Black-Box: Can knowledge graphs help make deep learning systems more interpretable and explainable?cited by 31
  29. 2019Human-grounded Evaluations of Explanation Methods for Text Classificationcited by 28
  30. 2016Hierarchical Memory Networkscited by 15
  31. 2021Transformers: "The End of History" for NLP?cited by 12
  32. 2019A Gated Self-attention Memory Network for Answer Selectioncited by 11
  33. 2020Contextualize Knowledge Bases with Transformer for End-to-end Task-Oriented Dialogue Systemscited by 8
  34. 2020Taking Notes on the Fly Helps BERT Pre-trainingcited by 5

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Let Guidelines Guide You: A Prescriptive Guideline-Centered Data Annotation Methodologycited by 3