mlsanityData to 14 Oct 2026

Analytical Methods for Interpretable Ultradense Word Embeddings

arXiv 1904.08654 ↗First posted 18 Apr 2019cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Word embeddings are useful for a wide variety of tasks, but they lack interpretability. By rotating word spaces, interpretable dimensions can be identified while preserving the information contained in the embeddings without any loss. In this work, we investigate three methods for making word spaces interpretable by rotation: Densifier (Rothe et al., 2016), linear SVMs and DensRay, a new method we propose. In contrast to Densifier, DensRay can be computed in closed form, is hyperparameter-free and thus more robust than Densifier. We evaluate the three methods on lexicon induction and set-based word analogy. In addition we provide qualitative insights as to how interpretable word spaces can be used for removing gender bias from embeddings.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 3; 2023: 4; 2024: 2; 2025: 1.

13 papers.

Sort:

  1. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  2. 2022On the Explainability of Natural Language Processing Deep Modelscited by 38
  3. 2024Cross-Entropy Is All You Need To Invert the Data Generating Processcited by 31
  4. 2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
  5. 2023Fairness in Language Models Beyond English: Gaps and Challengescited by 16
  6. 2023Investigating Bias in Multilingual Language Models: Cross-Lingual Transfer of Debiasing Techniquescited by 14
  7. 2023Verb Conjugation in Transformers Is Determined by Linear Encodings of Subject Numbercited by 12
  8. 2021Locating Language-Specific Information in Contextualized Embeddingscited by 6
  9. 2022Interpreting Embedding Spaces by Conceptualizationcited by 6
  10. 2025One Swallow Does Not Make a Summer: Understanding Semantic Structures in Embedding Spacescited by 4
  11. 2022Explainability of Text Processing and Retrieval Methods: A Surveycited by 0
  12. 2023Dialectograms: Machine Learning Differences between Discursive Communitiescited by 0
1 more
  1. 2024Optimal synthesis embeddingscited by 0