mlsanityData to 14 Oct 2026

Interpretable Text Embeddings and Text Similarity Explanation: A Survey

arXiv 2502.14862 ↗First posted 20 Feb 2025cs.CL cs.AI cs.IRcited by 12

Not on the map. Why?

Abstract (arXiv)

Text embeddings are a fundamental component in many NLP tasks, including classification, regression, clustering, and semantic search. However, despite their ubiquitous application, challenges persist in interpreting embeddings and explaining similarities between them. In this work, we provide a structured overview of methods specializing in inherently interpretable text embeddings and text similarity explanation, an underexplored research area. We characterize the main ideas, approaches, and trade-offs. We compare means of evaluation, discuss overarching lessons learned and finally identify opportunities and open challenges for future research.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 63 of its 103 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

63 of 103 references.

References · 63

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  4. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  6. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  7. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  8. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 1,001
  9. 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
  10. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  11. 2019Attention is not not Explanationcited by 636
  12. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
38 more
  1. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
  2. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  3. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 432
  4. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
  5. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
  6. 2019Interpretable machine learning: definitions, methods, and applicationscited by 305
  7. 2023Effective Long-Context Scaling of Foundation Modelscited by 303
  8. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  9. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 286
  10. 2020Evaluation of Text Generation: A Surveycited by 277
  11. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 236
  12. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  13. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  14. 2020A Survey of Evaluation Metrics Used for NLG Systemscited by 164
  15. 2019The Shapley Taylor Interaction Indexcited by 143
  16. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 120
  17. 2020Explaining Explanations: Axiomatic Feature Interactions for Deep Networkscited by 103
  18. 2019A Deep Look into Neural Ranking Models for Information Retrievalcited by 79
  19. 2020SBERT-WK: A Sentence Embedding Method by Dissecting BERT-based Word Modelscited by 56
  20. 2021GooAQ: Open Question Answering with Diverse Answer Typescited by 53
  21. 2020SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrievalcited by 50
  22. 2023Information Retrieval: Recent Advances and Beyondcited by 49
  23. 2020SparTerm: Learning Term-based Sparse Representation for Fast Text Retrievalcited by 45
  24. 2015Sparse Overcomplete Word Vector Representationscited by 44
  25. 2022Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approachescited by 39
  26. 2020Improving Local Identifiability in Probabilistic Box Embeddingscited by 35
  27. 2023Jina Embeddings: A Novel Set of High-Performance Sentence Embedding Modelscited by 35
  28. 2020Building and Interpreting Deep Similarity Modelscited by 34
  29. 2022SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Featurescited by 33
  30. 2023NevIR: Negation in Neural Information Retrievalcited by 29
  31. 2021Disentangling Semantics and Syntax in Sentence Embeddings with Pre-trained Language Modelscited by 22
  32. 2023SMATCH++: Standardized and Extended Evaluation of Semantic Graphscited by 21
  33. 2023Natural Language Decompositions of Implicit Content Enable Better Text Representationscited by 19
  34. 2023Meaning Representations from Trajectories in Autoregressive Modelscited by 18
  35. 2022Specialized Document Embeddings for Aspect-based Similarity of Research Paperscited by 12
  36. 2023Investigating semantic subspaces of Transformer sentence embeddings through linear structural probingcited by 12
  37. 2024Crafting Interpretable Embeddings by Asking LLMs Questionscited by 11
  38. 2024A General Framework for Producing Interpretable Semantic Text Embeddingscited by 10

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 10.

12 papers.

  1. 2026Information Representation Fairness in Long-Document Embeddings: The Peculiar Interaction of Positional and Language Biascited by 2
  2. 2025Lumos: Let there be Language Model System Certificationcited by 1
  3. 2026Xetrieval: Mechanistically Explaining Dense Retrievalcited by 1
  4. 2026ALEE: Any-Language Evaluation of Embeddings via English-Centric Minimal Pairscited by 1
  5. 2025Mechanistic Decomposition of Sentence Representationscited by 0
  6. 2026Are Emotions Arranged in a Circle? Geometric Analysis of Emotion Representations via Hyperspherical Contrastive Learningcited by 0
  7. 2026Asking the Right Questions: Ontology-Grounded Interpretable Embeddings for Biomedical Textcited by 0
  8. 2026Do LLM Embedding Spaces Recover Expert Structure?cited by 0
  9. 2026Explaining Image Similarity with Automatically Extracted Concept Activation Vectorscited by 0
  10. 2026xMICD: Explainable Representation of Multiple ICD Codescited by 0
  11. 2026FastE: Readout-Triggered Token Compression for LLM Embedding Inferencecited by 0
  12. 2026Embedding Models Measure in Peculiar Wayscited by 0