# From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignment

arXiv: https://arxiv.org/abs/2505.16610 · HTML: https://mlsanity.com/paper/2505.16610/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 29 of its 55 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Medical LLM alignment](https://mlsanity.com/alignment/medical-llm-alignment/)

## Builds on (most cited first, 29 of 29)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Evaluating Large Language Models at Evaluating Instruction Following](https://mlsanity.com/paper/2310.07641/) (2023)
- [Towards Emotional Support Dialog Systems](https://mlsanity.com/paper/2106.01144/) (2021)
- [Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study](https://mlsanity.com/paper/2304.00723/) (2023)
- [AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversation](https://mlsanity.com/paper/2202.13047/) (2022)
- [Aligning Large Language Models with Human Preferences through Representation Engineering](https://mlsanity.com/paper/2312.15997/) (2023)
- [Rewarding Chatbots for Real-World Engagement with Millions of Users](https://mlsanity.com/paper/2303.06135/) (2023)
- [InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions](https://mlsanity.com/paper/2403.11435/) (2024)
- [Control Globally, Understand Locally: A Global-to-Local Hierarchical Graph Network for Emotional Support Conversation](https://mlsanity.com/paper/2204.12749/) (2022)
- [ESCoT: Towards Interpretable Emotional Support Dialogue Systems](https://mlsanity.com/paper/2406.10960/) (2024)
- [Self-Boosting Large Language Models with Synthetic Preference Data](https://mlsanity.com/paper/2410.06961/) (2024)
- [PAL: Persona-Augmented Emotional Support Conversation Generation](https://mlsanity.com/paper/2212.09235/) (2022)
- [Predictive Engagement: An Efficient Metric For Automatic Evaluation of Open-Domain Dialogue Systems](https://mlsanity.com/paper/1911.01456/) (2019)
- [Beyond Imitation: Leveraging Fine-grained Quality Signals for Alignment](https://mlsanity.com/paper/2311.04072/) (2023)
- [Re-ReST: Reflection-Reinforced Self-Training for Language Agents](https://mlsanity.com/paper/2406.01495/) (2024)
- [SweetieChat: A Strategy-Enhanced Role-playing Framework for Diverse Scenarios Handling Emotional Support Agent](https://mlsanity.com/paper/2412.08389/) (2024)
- [Other Roles Matter! Enhancing Role-Oriented Dialogue Summarization via Role Interactions](https://mlsanity.com/paper/2205.13190/) (2022)
- [Mitigating Unhelpfulness in Emotional Support Conversations with Multifaceted AI Feedback](https://mlsanity.com/paper/2401.05928/) (2024)

## Built on by (most cited first, 9 of 9)

- [Teaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions](https://mlsanity.com/paper/2507.13773/) (2025)
- [iReasoner: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Models](https://mlsanity.com/paper/2601.05877/) (2026)
- [EmoHarbor: Evaluating Personalized Emotional Support by Simulating the User's Internal World](https://mlsanity.com/paper/2601.01530/) (2026)
- [Listening to the Echo: User-Reaction Aware Policy Optimization via Scalar-Verbal Hybrid Reinforcement Learning](https://mlsanity.com/paper/2603.15434/) (2026)
- [Discourse Diversity in Multi-Turn Empathic Dialogue](https://mlsanity.com/paper/2604.11742/) (2026)
- [PERCEIVE: A Benchmark for Personalized Emotion and Communication Behavior Understanding on Social Media](https://mlsanity.com/paper/2605.12525/) (2026)
- [MindTailor: Personalized Emotional Support via Post History-Grounded Case Formulation and Collaborative Refinement](https://mlsanity.com/paper/2606.21930/) (2026)
- [Towards High-Level Semantic Intelligence](https://mlsanity.com/paper/2607.24082/) (2026)
- [Dual-Loop Self-Evolution via Verifiable Emotion Feedback for Multi-Turn Empathetic Dialogue](https://mlsanity.com/paper/2608.10626/) (2026)

## Abstract (arXiv)

Effective emotional support hinges on understanding users' emotions and needs to provide meaningful comfort during multi-turn interactions. Large Language Models (LLMs) show great potential for expressing empathy; however, they often deliver generic and one-size-fits-all responses that fail to address users' specific needs. To tackle this issue, we propose a self-evolution framework designed to help LLMs improve their responses to better align with users' implicit preferences concerning user profiles (personalities), emotional states, and specific situations. Our framework consists of two distinct phases: textit(1) textitEmotional Support Experience Acquisition, where LLMs are fine-tuned on limited emotional support conversation data to provide basic support, and textit(2) textitSelf-Improvement for Personalized Emotional Support, where LLMs leverage self-reflection and self-refinement to generate personalized responses. Through iterative direct preference optimization between the pre- and post-refined responses, our model generates responses that reflect a better understanding of the user's implicit preferences. Extensive experiments and evaluations demonstrate that our method significantly enhances the model's performance in emotional support, reducing unhelpful responses and minimizing discrepancies between user preferences and model outputs.