mlsanityData to 7 Oct 2026

Discovering Differences in the Representation of People using Contextualized Semantic Axes

arXiv 2210.12170 ↗First posted 21 Oct 2022cs.CL cs.SIcited by 12

Not on the map. Why?

Abstract (arXiv)

A common paradigm for identifying semantic differences across social and temporal contexts is the use of static word embeddings and their distances. In particular, past work has compared embeddings against "semantic axes" that represent two opposing concepts. We extend this paradigm to BERT embeddings, and construct contextualized axes that mitigate the pitfall where antonyms have neighboring representations. We validate and demonstrate these axes on two people-centric datasets: occupations from Wikipedia, and multi-platform discussions in extremist, men's communities over fourteen years. In both studies, contextualized semantic axes can characterize differences among instances of the same word type. In the latter study, we show that references to women and the contexts around them have become more detestable over time.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 37 of its 94 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

37 of 94 references.

References · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  3. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  4. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  5. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 370
  6. 2020The Pushshift Reddit Datasetcited by 289
  7. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 251
  8. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 150
  9. 2020Neural Unsupervised Domain Adaptation in NLP---A Surveycited by 142
  10. 2016Counter-fitting Word Vectors to Linguistic Constraintscited by 138
  11. 2020Analysing Lexical Semantic Change with Contextualised Word Representationscited by 92
  12. 2018The Geometry of Culture: Analyzing Meaning through Word Embeddingscited by 87
25 more
  1. 2016Cultural Shift or Linguistic Drift? Comparing Two Computational Measures of Semantic Changecited by 63
  2. 2016Deep Reinforcement Learning for Mention-Ranking Coreference Modelscited by 63
  3. 2019Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddingscited by 55
  4. 2021Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Modelscited by 39
  5. 2019Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shiftcited by 37
  6. 2021Uncovering Implicit Gender Bias in Narratives through Commonsense Inferencecited by 32
  7. 2020A Framework for the Computational Linguistic Analysis of Dehumanizationcited by 31
  8. 2021Detecting Cross-Geographic Biases in Toxicity Modeling on Social Mediacited by 31
  9. 2018A Quantitative Approach to Understanding Online Antisemitismcited by 26
  10. 2018SemAxis: A Lightweight Framework to Characterize Domain-Specific Word Semantics Beyond Sentimentcited by 25
  11. 2021Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Modelcited by 23
  12. 2020FrameAxis: Characterizing Microframe Bias and Intensity with Word Embeddingcited by 21
  13. 2021Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corporacited by 21
  14. 2021Characterizing English Variation across Social Media Communities with BERTcited by 20
  15. 2019Unsupervised Discovery of Gendered Language through Latent-Variable Modelingcited by 19
  16. 2021Frequency-based Distortions in Contextualized Word Embeddingscited by 19
  17. 2020On the Robustness of Language Encoders against Grammatical Errorscited by 18
  18. 2020The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddingscited by 15
  19. 2020Do Platform Migrations Compromise Content Moderation? Evidence from r/The_Donald and r/Incelscited by 15
  20. 2020Living Machines: A study of atypical animacycited by 13
  21. 2019Entity-Centric Contextual Affective Analysiscited by 11
  22. 2022Richer Countries and Richer Representationscited by 7
  23. 2021The incel lexicon: Deciphering the emergent cryptolect of a global misogynistic communitycited by 4
  24. 2021Using Sociolinguistic Variables to Reveal Changing Attitudes Towards Sexuality and Gendercited by 4
  25. 2021"I'm a Professor, which isn't usually a dangerous job": Internet-Facilitated Harassment and its Impact on Researcherscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 2; 2025: 4; 2026: 2.

12 papers.

  1. 2023CoMPosT: Characterizing and Evaluating Caricature in LLM Simulationscited by 94
  2. 2023Unsupervised Contrast-Consistent Ranking with Language Modelscited by 13
  3. 2025From tools to thieves: Measuring and understanding public perceptions of AI through crowdsourced metaphorscited by 9
  4. 2023Wikibio: a Semantic Resource for the Intersectional Analysis of Biographical Eventscited by 8
  5. 2024Adjusting Interpretable Dimensions in Embedding Space with Human Judgmentscited by 4
  6. 2025Rethinking Word Similarity: Semantic Similarity through Classification Confusioncited by 2
  7. 2023CIDER: Context sensitive sentiment analysis for short-form textcited by 0
  8. 2024Capturing Differences in Character Representations Between Communities: An Initial Study with Fandomcited by 0
  9. 2025LogiPart: Local Large Language Models for Data Exploration at Scale with Logical Partitioningcited by 0
  10. 2025Interpretable dimensions support an effect of agentivity and telicity on split intransitivitycited by 0
  11. 2026Language Models as Measurement Apparatus for Culturecited by 0
  12. 2026STEREODISCO: Discovering Stereotypicality in LLMscited by 0