mlsanityData to 14 Oct 2026

Adjusting Interpretable Dimensions in Embedding Space with Human Judgments

arXiv 2404.02619 ↗First posted 3 Apr 2024cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Embedding spaces contain interpretable dimensions indicating gender, formality in style, or even object properties. This has been observed multiple times. Such interpretable dimensions are becoming valuable tools in different areas of study, from social science to neuroscience. The standard way to compute these dimensions uses contrasting seed words and computes difference vectors over them. This is simple but does not always work well. We combine seed-based vectors with guidance from human ratings of where words fall along a specific dimension, and evaluate on predicting both object properties like size and danger, and the stylistic properties of formality and complexity. We obtain interpretable dimensions with markedly better performance especially in cases where seed-based dimensions do not work well.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 25 of its 47 references that are in the map. First posted on arXiv 3 Apr 2024.

Builds on

25 of 47 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  4. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  5. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  6. 2019Designing and Interpreting Probes with Control Taskscited by 640
  7. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  8. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 438
  9. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 253
  10. 2020Probing Pretrained Language Models for Lexical Semanticscited by 156
  11. 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 151
  12. 2022Towards Faithful Model Explanation in NLP: A Surveycited by 130
13 more
  1. 2018The Geometry of Culture: Analyzing Meaning through Word Embeddingscited by 87
  2. 2019Inducing Relational Knowledge from BERTcited by 75
  3. 2019Attenuating Bias in Word Vectorscited by 64
  4. 2021Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Sensescited by 39
  5. 2018SemAxis: A Lightweight Framework to Characterize Domain-Specific Word Semantics Beyond Sentimentcited by 26
  6. 2020FrameAxis: Characterizing Microframe Bias and Intensity with Word Embeddingcited by 21
  7. 2020BERT Knows Punta Cana is not just beautiful, it's gorgeous: Ranking Scalar Adjectives with Contextualised Representationscited by 15
  8. 2022Discovering Differences in the Representation of People using Contextualized Semantic Axescited by 12
  9. 2023Representation Of Lexical Stylistic Features In Language Models' Embedding Spacecited by 8
  10. 2020Cultural Cartography with Word Embeddingscited by 7
  11. 2020A Unified Feature Representation for Lexical Connotationscited by 6
  12. 2021Scalar Adjective Identification and Multilingual Rankingcited by 6
  13. 2016Entity Embeddings with Conceptual Subspaces as a Basis for Plausible Reasoningcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2024Latent Space Interpretation for Stylistic Analysis and Explainable Authorship Attributioncited by 7
  2. 2025Rethinking Word Similarity: Semantic Similarity through Classification Confusioncited by 2
  3. 2025Extracting Conceptual Spaces from LLMs Using Prototype Embeddingscited by 0
  4. 2025Interpretable dimensions support an effect of agentivity and telicity on split intransitivitycited by 0