mlsanityData to 14 Oct 2026

Personalized LLM Decoding via Contrasting Personal Preference

arXiv 2506.12109 ↗First posted 13 Jun 2025cs.CL cs.AIcited by 12

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

As large language models (LLMs) are progressively deployed in various real-world applications, personalization of LLMs has become increasingly important. While various approaches to LLM personalization such as prompt-based and training-based methods have been actively explored, the development of effective decoding-time algorithms remains largely overlooked, despite their demonstrated potential. In this paper, we propose CoPe (Contrasting Personal Preference), a novel decoding-time approach applied after performing parameter-efficient fine-tuning (PEFT) on user-specific data. Our core idea is to leverage reward-guided decoding specifically for personalization by maximizing each user's implicit reward signal. We evaluate CoPe across five open-ended personalized text generation tasks. Our empirical results demonstrate that CoPe achieves strong performance, improving personalization by an average of 10.57% in ROUGE-L, without relying on external reward models or additional training procedures.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 33 of its 45 references that are in the map. First posted on arXiv 13 Jun 2025.

Builds on

33 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Aligning Language Models to User Opinions24 May

Other references · 32

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  9. 2024Qwen2.5 Technical Reportcited by 4,942
  10. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  11. 2023Let's Verify Step by Stepcited by 3,789
  12. 2025Gemma 3 Technical Reportcited by 1,824
20 more
  1. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  2. 2023Whose Opinions Do Language Models Reflect?cited by 773
  3. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  4. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  5. 2025Process Reinforcement through Implicit Rewardscited by 504
  6. 2023LaMP: When Large Language Models Meet Personalizationcited by 437
  7. 2019Episodic Memory in Lifelong Language Learningcited by 188
  8. 2024BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Samplingcited by 121
  9. 2024LongLaMP: A Benchmark for Personalized Long-form Text Generationcited by 97
  10. 2023Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrieverscited by 92
  11. 2024Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuningcited by 90
  12. 2020Continual Learning in Task-Oriented Dialogue Systemscited by 83
  13. 2023Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Modelcited by 79
  14. 2024HYDRA: Model Factorization Framework for Black-Box LLM Personalizationcited by 60
  15. 2024Bootstrapping Language Models with DPO Implicit Rewardscited by 53
  16. 2024Personalized LLM Response Generation with Parameterized Memory Injectioncited by 33
  17. 2024Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignmentcited by 27
  18. 2024Review-LLM: Harnessing Large Language Models for Personalized Review Generationcited by 14
  19. 2025Personalized Language Models via Privacy-Preserving Evolutionary Model Mergingcited by 8
  20. 2024SPAR: Personalized Content-Based Recommendation via Long Engagement Attentioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 10.

12 papers.

  1. 2025PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memorycited by 88
  2. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 14
  3. 2026SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generationcited by 6
  4. 2026SPRInG: Continual LLM Personalization via Selective Parametric Adaptation and Retrieval-Interpolated Generationcited by 4
  5. 2026Synthetic Interaction Data for Scalable Personalization in Large Language Modelscited by 4
  6. 2026CARD: Cluster-level Adaptation with Reward-guided Decoding for Personalized Text Generationcited by 3
  7. 2025TiTok: Transfer Token-level Knowledge via Contrastive Excess to Transplant LoRAcited by 2
  8. 2026Personalized Group Relative Policy Optimization for Heterogenous Preference Alignmentcited by 1
  9. 2026Personalizing LLMs with Binary Feedback: A Preference-Corrected Optimization Frameworkcited by 1
  10. 2026Learning to Adapt Cross-Domain Preferences via Meta-LoRA for LLM Personalizationcited by 1
  11. 2026PLUME: Parameter-Efficient Personalization of Large Language Models via Low-Rank User Modulation in Shared Subspacescited by 0
  12. 2026Gradient-Aligned Pair Selection for Personalized Preference Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024PAD: Personalized Alignment of LLMs at Decoding-Time
  2. 2025T-POP: Test-Time Personalization with Online Preference Feedback
  3. 2026COPE: Continual Personalization of LLMs under Sparse User Feedback via User Embeddings and Self-Evaluation
  4. 2025Personalize Your LLM: Fake it then Align it
  5. 2026FleetPilot: Fine-Grained Steering Empowers Real-World LLM Personalization
  6. 2026Many Preferences, Few Policies: Towards Scalable Language Model Personalization
  7. 2023Parameter Efficient Tuning Allows Scalable Personalization of LLMs for Text Entry: A Case Study on Abbreviation Expansion
  8. 2025Drift: Decoding-time Personalized Alignments with Implicit User Preferences