mlsanityData to 7 Oct 2026

Interpreting Embedding Spaces by Conceptualization

arXiv 2209.00445 ↗First posted 22 Aug 2022cs.CL cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

One of the main methods for computational interpretation of a text is mapping it into a vector in some embedding space. Such vectors can then be used for a variety of textual processing tasks. Recently, most embedding spaces are a product of training large language models (LLMs). One major drawback of this type of representation is their incomprehensibility to humans. Understanding the embedding space is crucial for several important needs, including the need to debug the embedding method and compare it to alternatives, and the need to detect biases hidden in the model. In this paper, we present a novel method of understanding embeddings by transforming a latent embedding space into a comprehensible conceptual space. We present an algorithm for deriving a conceptual space with dynamic on-demand granularity. We devise a new evaluation method, using either human rater or LLM-based raters, to show that the conceptualized vectors indeed represent the semantics of the original latent ones. We show the use of our method for various tasks, including comparing the semantics of alternative models and tracing the layers of the LLM. The code is available online https://github.com/adiSimhi/Interpreting-Embedding-Spaces-by-Conceptualization.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 28 of its 57 references that are in the map. First posted on arXiv 22 Aug 2022.

Builds on

28 of 57 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  6. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  9. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  10. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  11. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  12. 2020A Survey on the Explainability of Supervised Machine Learningcited by 225
16 more
  1. 2016Model-Agnostic Interpretability of Machine Learningcited by 219
  2. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 219
  3. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  4. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  5. 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
  6. 2018Identifying and Controlling Important Neurons in Neural Machine Translationcited by 110
  7. 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 106
  8. 2020Explaining NLP Models via Minimal Contrastive Editing (MiCE)cited by 105
  9. 2017SPINE: SParse Interpretable Neural Embeddingscited by 55
  10. 2020Explaining and Improving Model Behavior with k Nearest Neighbor Representationscited by 27
  11. 2017Semantic Structure and Interpretability of Word Embeddingscited by 21
  12. 2020The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddingscited by 15
  13. 2019Analytical Methods for Interpretable Ultradense Word Embeddingscited by 13
  14. 2018Imparting Interpretability to Word Embeddings while Preserving Semantic Structurecited by 6
  15. 2018Firearms and Tigers are Dangerous, Kitchen Knives and Zebras are Not: Testing whether Word Embeddings Can Tellcited by 5
  16. 2019Parallax: Visualizing and Understanding the Semantics of Embedding Spaces via Algebraic Formulaecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 2.

6 papers.

  1. 2025Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaningcited by 10
  2. 2025Towards an Explainable Comparison and Alignment of Feature Embeddingscited by 9
  3. 2024SWEA: Updating Factual Knowledge in Large Language Models via Subject Word Embedding Alteringcited by 7
  4. 2026EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectorscited by 2
  5. 2024Wave Network: An Ultra-Small Language Modelcited by 1
  6. 2026Persistent Convolution: A Topological Framework for AI Alignment Testing and Semantic Space Characterizationcited by 1