# Personalize Your LLM: Fake it then Align it

arXiv: https://arxiv.org/abs/2503.01048 · HTML: https://mlsanity.com/paper/2503.01048/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 44 of its 52 references that are in the map.
- First posted on arXiv 2 Mar 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 30 of 44)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [Executable Code Actions Elicit Better LLM Agents](https://mlsanity.com/paper/2402.01030/) (2024)
- [A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.06211/) (2024)
- [LaMP: When Large Language Models Meet Personalization](https://mlsanity.com/paper/2304.11406/) (2023)
- [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023)
- [Is ChatGPT a Good Recommender? A Preliminary Study](https://mlsanity.com/paper/2304.10149/) (2023)
- [Uncovering ChatGPT's Capabilities in Recommender Systems](https://mlsanity.com/paper/2305.02182/) (2023)
- [Debiasing Vision-Language Models via Biased Prompts](https://mlsanity.com/paper/2302.00070/) (2023)
- [Personalized Language Modeling from Personalized Human Feedback](https://mlsanity.com/paper/2402.05133/) (2024)
- [A Neural Space-Time Representation for Text-to-Image Personalization](https://mlsanity.com/paper/2305.15391/) (2023)
- [Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction](https://mlsanity.com/paper/2305.06474/) (2023)
- [Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning](https://mlsanity.com/paper/2402.04401/) (2024)
- [Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback](https://mlsanity.com/paper/2303.05453/) (2023)
- [LLM-Rec: Personalized Recommendation via Prompting Large Language Models](https://mlsanity.com/paper/2307.15780/) (2023)
- [Word Embeddings Are Steers for Language Models](https://mlsanity.com/paper/2305.12798/) (2023)
- [Aligning Large Language Models with Representation Editing: A Control Perspective](https://mlsanity.com/paper/2406.05954/) (2024)
- [Aligning LLMs with Individual Preferences via Interaction](https://mlsanity.com/paper/2410.03642/) (2024)
- [HYDRA: Model Factorization Framework for Black-Box LLM Personalization](https://mlsanity.com/paper/2406.02888/) (2024)
- [PersonalLLM: Tailoring LLMs to Individual Preferences](https://mlsanity.com/paper/2409.20296/) (2024)
- [Controllable Text Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2408.12599/) (2024)
- [ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Models](https://mlsanity.com/paper/2305.06566/) (2023)
- [Integrating Summarization and Retrieval for Enhanced Personalization via Large Language Models](https://mlsanity.com/paper/2310.20081/) (2023)

## Built on by (most cited first, 15 of 15)

- [A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence](https://mlsanity.com/paper/2507.21046/) (2025)
- [Embedding-to-Prefix: Parameter-Efficient Personalization for Pre-Trained Large Language Models](https://mlsanity.com/paper/2505.17051/) (2025)
- [A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challenges](https://mlsanity.com/paper/2507.21117/) (2025)
- [LLM-Generated Ads: From Personalization Parity to Persuasion Superiority](https://mlsanity.com/paper/2512.03373/) (2025)
- [AI and My Values: User Perceptions of LLMs' Ability to Extract, Embody, and Explain Human Values from Casual Conversations](https://mlsanity.com/paper/2601.22440/) (2026)
- [LikeBench: Evaluating Subjective Likability in LLMs for Personalization](https://mlsanity.com/paper/2512.13077/) (2025)
- [Towards Effective Model Editing for LLM Personalization](https://mlsanity.com/paper/2512.13676/) (2025)
- [ChildEval: When large language models meet children's personalities](https://mlsanity.com/paper/2605.27805/) (2026)
- [Self-supervised User Profile Generation for Personalization](https://mlsanity.com/paper/2606.05336/) (2026)
- [Memory Has Geometry: Non-Uniform Geometric Memory for Long-Horizon Personalized AI](https://mlsanity.com/paper/2609.17969/) (2026)
- [Psi-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues](https://mlsanity.com/paper/2606.02754/) (2026)
- [Weightless Fine-Tuning: Personalizing LLMs via Logit-Space Transport](https://mlsanity.com/paper/2608.11342/) (2026)
- [PersonaEdit: Representative Sample Selection for Personalized Model Editing](https://mlsanity.com/paper/2608.27816/) (2026)
- [Evaluating the Hidden Costs of Personalization in Large Language Models](https://mlsanity.com/paper/2608.28833/) (2026)
- [From Static Personal Values to Contextualized Personalization: Bayesian Personalized Value Alignment for LLMs](https://mlsanity.com/paper/2609.28942/) (2026)

## Abstract (arXiv)

Personalizing large language models (LLMs) is essential for delivering tailored interactions that improve user experience. Many existing personalization methods require fine-tuning LLMs for each user, rendering them prohibitively expensive for widespread adoption. Although retrieval-based approaches offer a more compute-efficient alternative, they still depend on large, high-quality datasets that are not consistently available for all users. To address this challenge, we propose CHAMELEON, a scalable and efficient personalization approach that uses (1) self-generated personal preference data and (2) representation editing to enable quick and cost-effective personalization. Our experiments on various tasks, including those from the LaMP personalization benchmark, show that CHAMELEON efficiently adapts models to personal preferences, improving instruction-tuned models and outperforms two personalization baselines by an average of 40% across two model architectures.