mlsanityData to 7 Oct 2026

Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology

arXiv 2212.07547 ↗First posted 14 Dec 2022cs.CL cs.AI cs.SIcited by 1

Not on the map. Why?

Abstract (arXiv)

We propose a fully unsupervised method to detect bias in contextualized embeddings. The method leverages the assortative information latently encoded by social networks and combines orthogonality regularization, structured sparsity learning, and graph neural networks to find the embedding subspace capturing this information. As a concrete example, we focus on the phenomenon of ideological bias: we introduce the concept of an ideological subspace, show how it can be found by applying our method to online discussion forums, and present techniques to probe it. Our experiments suggest that the ideological subspace encodes abstract evaluative semantics and reflects changes in the political left-right spectrum during the presidency of Donald Trump.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 52 of its 108 references that are in the map. First posted on arXiv 14 Dec 2022.

Builds on

52 of 108 references.

References · 52

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2016Variational Graph Auto-Encoderscited by 1,373
  5. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  6. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  7. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  8. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  9. 2016Domain Separation Networkscited by 317
  10. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  11. 2019Visualizing and Measuring the Geometry of BERTcited by 215
  12. 2019Gender Bias in Contextualized Word Embeddingscited by 203
38 more
  1. 2019Assessing BERT's Syntactic Abilitiescited by 191
  2. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  3. 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 134
  4. 2016Neural Photo Editing with Introspective Adversarial Networkscited by 109
  5. 2019Evaluating the Underlying Gender Bias in Contextualized Word Embeddingscited by 94
  6. 2020We Can Detect Your Bias: Predicting the Political Ideology of News Articlescited by 92
  7. 2017On orthogonality and learning recurrent networks with long term dependenciescited by 82
  8. 2015Fast ConvNets Using Group-wise Brain Damagecited by 81
  9. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  10. 2018Community Interaction and Conflict on the Webcited by 72
  11. 2019In Plain Sight: Media Bias Through the Lens of Factual Reportingcited by 68
  12. 2019Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootingscited by 67
  13. 2019Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddingscited by 55
  14. 2021Modeling Framing in Immigration Discourse on Social Mediacited by 37
  15. 2020A Framework for the Computational Linguistic Analysis of Dehumanizationcited by 31
  16. 2020Quantifying social organization and political polarization in online platformscited by 30
  17. 2020Moral Framing and Ideological Bias of Newscited by 29
  18. 2015Quantifying Controversy in Social Mediacited by 28
  19. 2018Multi-view Models for Political Ideology Detection of News Articlescited by 27
  20. 2020Weakly Supervised Learning of Nuanced Frames for Analyzing Polarization in News Mediacited by 27
  21. 2018SemAxis: A Lightweight Framework to Characterize Domain-Specific Word Semantics Beyond Sentimentcited by 25
  22. 2020Exploring the Linear Subspace Hypothesis in Gender Bias Mitigationcited by 25
  23. 2017Distinguishing Antonyms and Synonyms in a Pattern-based Neural Networkcited by 19
  24. 2021Structured Sparsity Inducing Adaptive Optimizers for Deep Learningcited by 17
  25. 2016Integrating Distributional Lexical Contrast into Word Embeddings for Antonym-Synonym Distinctioncited by 13
  26. 2016Hypernyms under Siege: Linguistically-motivated Artillery for Hypernymy Detectioncited by 13
  27. 2021Diachronic Analysis of German Parliamentary Proceedings: Ideological Shifts through the Lens of Political Biasescited by 13
  28. 2019Entity-Centric Contextual Affective Analysiscited by 11
  29. 2020Text-based inference of moral sentiment changecited by 11
  30. 2019Tracing Antisemitic Language Through Diachronic Embedding Projections: France 1789-1914cited by 9
  31. 2021Modeling Ideological Salience and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsitycited by 9
  32. 2014Controversy and Sentiment in Online Newscited by 6
  33. 2018Extracting Inter-community Conflicts in Redditcited by 6
  34. 2020Roots of Trumpism: Homophily and Social Feedback in Donald Trump Support on Redditcited by 6
  35. 2021SWEAT: Scoring Polarization of Topics across Different Corporacited by 5
  36. 2015Ideological and Temporal Components of Network Polarization in Online Political Participatory Mediacited by 4
  37. 2020A Computational Analysis of Polarization on Indian and Pakistani Social Mediacited by 4
  38. 2019Political Discussions in Homogeneous and Cross-Cutting Communication Spacescited by 2

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Measuring Scalar Constructs in Social Science with LLMscited by 8