mlsanityData to 7 Oct 2026

Explainability of Text Processing and Retrieval Methods: A Survey

arXiv 2212.07126 ↗First posted 14 Dec 2022cs.IR cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep Learning and Machine Learning based models have become extremely popular in text processing and information retrieval. However, the non-linear structures present inside the networks make these models largely inscrutable. A significant body of research has focused on increasing the transparency of these models. This article provides a broad overview of research on the explainability and interpretability of natural language processing and information retrieval methods. More specifically, we survey approaches that have been applied to explain word embeddings, sequence modeling, attention modules, transformers, BERT, and document ranking. The concluding section suggests some possible directions for future research on this topic.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 122 of its 221 references that are in the map. First posted on arXiv 14 Dec 2022.

Builds on

122 of 221 references.

References · 122

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  9. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  10. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  11. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  12. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
38 more
  1. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  8. 2018Deep contextualized word representationscited by 2,285
  9. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  10. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  11. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  12. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  13. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  14. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  15. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  16. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  17. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  18. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  19. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  20. 2019Attention is not Explanationcited by 916
  21. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  22. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  23. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  24. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  25. 2020Quantifying Attention Flow in Transformerscited by 759
  26. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  27. 2016The Mythos of Model Interpretabilitycited by 716
  28. 2019Passage Re-ranking with BERTcited by 629
  29. 2019Attention is not not Explanationcited by 629
  30. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  31. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  32. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  33. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  34. 2019Is Attention Interpretable?cited by 387
  35. 2019The many Shapley values for model explanationcited by 341
  36. 2016From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classificationcited by 337
  37. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  38. 2019Revealing the Dark Secrets of BERTcited by 300

50 most cited of 122 All 122 (JSON)

Built on by

Not cited yet.