mlsanityData to 7 Oct 2026

Personalize Your LLM: Fake it then Align it

arXiv 2503.01048 ↗First posted 2 Mar 2025cs.LGcited by 15

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

Personalizing large language models (LLMs) is essential for delivering tailored interactions that improve user experience. Many existing personalization methods require fine-tuning LLMs for each user, rendering them prohibitively expensive for widespread adoption. Although retrieval-based approaches offer a more compute-efficient alternative, they still depend on large, high-quality datasets that are not consistently available for all users. To address this challenge, we propose CHAMELEON, a scalable and efficient personalization approach that uses (1) self-generated personal preference data and (2) representation editing to enable quick and cost-effective personalization. Our experiments on various tasks, including those from the LaMP personalization benchmark, show that CHAMELEON efficiently adapts models to personal preferences, improving instruction-tuned models and outperforms two personalization baselines by an average of 40% across two model architectures.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 44 of its 52 references that are in the map. First posted on arXiv 2 Mar 2025.

Builds on

44 of 52 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Aligning LLMs with Individual Preferences via Interaction4 Oct
  2. 2024PersonalLLM: Tailoring LLMs to Individual Preferences30 Sep
  3. 2024AI Alignment with Changing and Influenceable Reward Functions28 May
  4. 2024Personalized Language Modeling from Personalized Human Feedback6 Feb
  5. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback9 Mar

Other references · 39

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  6. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  7. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  8. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  9. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  10. 2024Executable Code Actions Elicit Better LLM Agentscited by 537
  11. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  12. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
27 more
  1. 2023Improving Text Embeddings with Large Language Modelscited by 295
  2. 2023Is ChatGPT a Good Recommender? A Preliminary Studycited by 192
  3. 2023Uncovering ChatGPT's Capabilities in Recommender Systemscited by 160
  4. 2023Debiasing Vision-Language Models via Biased Promptscited by 147
  5. 2023A Neural Space-Time Representation for Text-to-Image Personalizationcited by 123
  6. 2023Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Predictioncited by 113
  7. 2024Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuningcited by 87
  8. 2023LLM-Rec: Personalized Recommendation via Prompting Large Language Modelscited by 77
  9. 2023Word Embeddings Are Steers for Language Modelscited by 70
  10. 2024Aligning Large Language Models with Representation Editing: A Control Perspectivecited by 62
  11. 2024HYDRA: Model Factorization Framework for Black-Box LLM Personalizationcited by 59
  12. 2024Controllable Text Generation for Large Language Models: A Surveycited by 55
  13. 2023ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Modelscited by 54
  14. 2024Personalized Large Language Modelscited by 44
  15. 2023Integrating Summarization and Retrieval for Enhanced Personalization via Large Language Modelscited by 40
  16. 2023Highly Personalized Text Embedding for Image Manipulation by Stable Diffusioncited by 39
  17. 2024PALP: Prompt Aligned Personalization of Text-to-Image Modelscited by 35
  18. 2024MedAdapter: Efficient Test-Time Adaptation of Large Language Models towards Medical Reasoningcited by 35
  19. 2023Zero-Shot Robustification of Zero-Shot Modelscited by 24
  20. 2023BookGPT: A General Framework for Book Recommendation Empowered by Large Language Modelcited by 22
  21. 2024RAM-EHR: Retrieval Augmentation Meets Clinical Predictions on Electronic Health Recordscited by 21
  22. 2023Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Modelscited by 20
  23. 2022Recognising the importance of preference change: A call for a coordinated multidisciplinary research effort in the age of AIcited by 12
  24. 2024MPCODER: Multi-user Personalized Code Generator with Explicit and Implicit Style Representation Learningcited by 12
  25. 2024PEFT-U: Parameter-Efficient Fine-Tuning for User Personalizationcited by 11
  26. 2024Is Free Self-Alignment Possible?cited by 4
  27. 2024Prompt-Softbox-Prompt: A Free-Text Embedding Control for Image Editingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 9.

15 papers.

Sort:

By area:

  1. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  2. 2025Embedding-to-Prefix: Parameter-Efficient Personalization for Pre-Trained Large Language Modelscited by 10
  3. 2025A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challengescited by 3
  4. 2025LLM-Generated Ads: From Personalization Parity to Persuasion Superioritycited by 2
  5. 2026AI and My Values: User Perceptions of LLMs' Ability to Extract, Embody, and Explain Human Values from Casual Conversationscited by 2
  6. 2025LikeBench: Evaluating Subjective Likability in LLMs for Personalizationcited by 1
  7. 2025Towards Effective Model Editing for LLM Personalizationcited by 1
  8. 2026Self-supervised User Profile Generation for Personalizationcited by 1
  9. 2026Memory Has Geometry: Non-Uniform Geometric Memory for Long-Horizon Personalized AIcited by 1
  10. 2026ChildEval: When large language models meet children's personalitiescited by 0
  11. 2026Psi-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialoguescited by 0
  12. 2026Weightless Fine-Tuning: Personalizing LLMs via Logit-Space Transportcited by 0
3 more
  1. 2026PersonaEdit: Representative Sample Selection for Personalized Model Editingcited by 0
  2. 2026Evaluating the Hidden Costs of Personalization in Large Language Modelscited by 0
  3. 2026From Static Personal Values to Contextualized Personalization: Bayesian Personalized Value Alignment for LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Many Preferences, Few Policies: Towards Scalable Language Model Personalization
  2. 2026Re-Centering Humans in LLM Personalization
  3. 2025POPI: Personalizing LLMs via Optimized Natural Language Preference Inference
  4. 2026Personalizing LLMs with Binary Feedback: A Preference-Corrected Optimization Framework
  5. 2024On the Way to LLM Personalization: Learning to Remember User Conversations
  6. 2024Few-shot Personalization of LLMs with Mis-aligned Responses
  7. 2025Evaluating Personalized Tool-Augmented LLMs from the Perspectives of Personalization and Proactivity
  8. 2025A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models