# Compositional Demographic Word Embeddings

arXiv: https://arxiv.org/abs/2010.02986 · HTML: https://mlsanity.com/paper/2010.02986/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2020.

## Built on by (most cited first, 20 of 20)

- [CAPE: Context-Aware Private Embeddings for Private Language Learning](https://mlsanity.com/paper/2108.12318/) (2021)
- [Unifying Data Perspectivism and Personalization: An Application to Social Norms](https://mlsanity.com/paper/2210.14531/) (2022)
- [Personalized Text Generation with Fine-Grained Linguistic Control](https://mlsanity.com/paper/2402.04914/) (2024)
- [Large Human Language Models: A Need and the Challenges](https://mlsanity.com/paper/2312.07751/) (2023)
- [Can Demographic Factors Improve Text Classification? Revisiting Demographic Adaptation in the Age of Transformers](https://mlsanity.com/paper/2210.07362/) (2022)
- [Voices in a Crowd: Searching for Clusters of Unique Perspectives](https://mlsanity.com/paper/2407.14259/) (2024)
- [LMSOC: An Approach for Socially Sensitive Pretraining](https://mlsanity.com/paper/2110.10319/) (2021)
- [Corpus Considerations for Annotator Modeling and Scaling](https://mlsanity.com/paper/2404.02340/) (2024)
- [Towards Dog Bark Decoding: Leveraging Human Speech Processing for Automated Bark Classification](https://mlsanity.com/paper/2404.18739/) (2024)
- [Human Language Modeling](https://mlsanity.com/paper/2205.05128/) (2022)
- [Benchmarking and Improving LLM Robustness for Personalized Generation](https://mlsanity.com/paper/2509.19358/) (2025)
- [Reference-less Analysis of Context Specificity in Translation with Personalised Language Models](https://mlsanity.com/paper/2303.16618/) (2023)
- [PerPLM: Personalized Fine-tuning of Pretrained Language Models via Writer-specific Intermediate Learning and Prompts](https://mlsanity.com/paper/2309.07727/) (2023)
- [Evaluating Short-Term Temporal Fluctuations of Social Biases in Social Media Data and Masked Language Models](https://mlsanity.com/paper/2406.13556/) (2024)
- [Learning Dynamic Contextualised Word Embeddings via Template-based Temporal Adaptation](https://mlsanity.com/paper/2208.10734/) (2022)
- [IDIOLEX: Unified and Continuous Representations for Idiolectal and Stylistic Variation](https://mlsanity.com/paper/2604.04704/) (2026)
- [Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models](https://mlsanity.com/paper/2410.19812/) (2024)
- [How Well Do You Know Your Audience? Toward Socially-aware Question Generation](https://mlsanity.com/paper/2110.08445/) (2021)
- [On the Limitations of Sociodemographic Adaptation with Transformers](https://mlsanity.com/paper/2208.01029/) (2022)
- [Examining the Utility of Self-disclosure Types for Modeling Annotators of Social Norms](https://mlsanity.com/paper/2512.16034/) (2025)

## Abstract (arXiv)

Word embeddings are usually derived from corpora containing text from many individuals, thus leading to general purpose representations rather than individually personalized representations. While personalized embeddings can be useful to improve language model performance and other language processing tasks, they can only be computed for people with a large amount of longitudinal data, which is not the case for new users. We propose a new form of personalized word embeddings that use demographic-specific word representations derived compositionally from full or partial demographic information for a user (i.e., gender, age, location, religion). We show that the resulting demographic-aware word representations outperform generic word representations on two tasks for English: language modeling and word associations. We further explore the trade-off between the number of available attributes and their relative effectiveness and discuss the ethical implications of using them.