# Implicit Personalization in Language Models: A Systematic Study

arXiv: https://arxiv.org/abs/2405.14808 · HTML: https://mlsanity.com/paper/2405.14808/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 51 references that are in the map.
- First posted on arXiv 23 May 2024.

## Builds on (most cited first, 9 of 9)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes](https://mlsanity.com/paper/1711.08412/) (2017)
- [AI Deception: A Survey of Examples, Risks, and Potential Solutions](https://mlsanity.com/paper/2308.14752/) (2023)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [Context Steering: Controllable Personalization at Inference Time](https://mlsanity.com/paper/2405.01768/) (2024)
- [Learning Personalized End-to-End Goal-Oriented Dialog](https://mlsanity.com/paper/1811.04604/) (2018)
- [Where in the World are You? Geolocation and Language Identification in Twitter](https://mlsanity.com/paper/1308.0683/) (2013)
- [Joint Effects of Context and User History for Predicting Online Conversation Re-entries](https://mlsanity.com/paper/1906.01185/) (2019)

## Built on by (most cited first, 18 of 18)

- [Reading Between the Prompts: How Stereotypes Shape LLM's Implicit Personalization](https://mlsanity.com/paper/2505.16467/) (2025)
- [Presumed Cultural Identity: How Names Shape LLM Responses](https://mlsanity.com/paper/2502.11995/) (2025)
- [Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personas](https://mlsanity.com/paper/2501.11549/) (2025)
- [Synthetic Socratic Debates: Examining Persona Effects on Moral Decision and Persuasion Dynamics](https://mlsanity.com/paper/2506.12657/) (2025)
- [Drift: Decoding-time Personalized Alignments with Implicit User Preferences](https://mlsanity.com/paper/2502.14289/) (2025)
- [Agent-to-Agent Theory of Mind: Testing Interlocutor Awareness among Large Language Models](https://mlsanity.com/paper/2506.22957/) (2025)
- [Verbalizing LLMs' assumptions to explain and control sycophancy](https://mlsanity.com/paper/2604.03058/) (2026)
- [Linear socio-demographic representations emerge in Large Language Models from indirect cues](https://mlsanity.com/paper/2512.10065/) (2025)
- [A Platform for Interactive AI Character Experiences](https://mlsanity.com/paper/2601.01027/) (2026)
- [Locating and Controlling Implicit Personalization in Large Language Models](https://mlsanity.com/paper/2608.11735/) (2026)
- [Finding A Voice: Exploring the Potential of African American Dialect and Voice Generation for Chatbots](https://mlsanity.com/paper/2501.03441/) (2025)
- [From Guessing to Asking: An Approach to Resolving the Persona Knowledge Gap in LLMs during Multi-Turn Conversations](https://mlsanity.com/paper/2503.12556/) (2025)
- [Steerable Chatbots: Exploring Personalization Control Interfaces via LLM Activation Steering](https://mlsanity.com/paper/2505.04260/) (2025)
- [IP-Dialog: Evaluating Implicit Personalization in Dialogue Systems with Synthetic Data](https://mlsanity.com/paper/2506.02449/) (2025)
- [Prioritize Economy or Climate Action? Investigating ChatGPT Response Differences Based on Inferred Political Orientation](https://mlsanity.com/paper/2511.04706/) (2025)
- [The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents](https://mlsanity.com/paper/2601.12164/) (2026)
- [Unintended Effects of Geographic Conditioning in Large Language Models](https://mlsanity.com/paper/2606.18124/) (2026)
- [Inferring Hidden User Models from the Behavior of Personalized LLM Agents](https://mlsanity.com/paper/2609.03815/) (2026)

## Abstract (arXiv)

Implicit Personalization (IP) is a phenomenon of language models inferring a user's background from the implicit cues in the input prompts and tailoring the response based on this inference. While previous work has touched upon various instances of this problem, there lacks a unified framework to study this behavior. This work systematically studies IP through a rigorous mathematical formulation, a multi-perspective moral reasoning framework, and a set of case studies. Our theoretical foundation for IP relies on a structural causal model and introduces a novel method, indirect intervention, to estimate the causal effect of a mediator variable that cannot be directly intervened upon. Beyond the technical approach, we also introduce a set of moral reasoning principles based on three schools of moral philosophy to study when IP may or may not be ethically appropriate. Equipped with both mathematical and ethical insights, we present three diverse case studies illustrating the varied nature of the IP problem and offer recommendations for future research. Our code is at https://github.com/jiarui-liu/IP, and our data is at https://huggingface.co/datasets/Jerry999/ImplicitPersonalizationData.