mlsanityData to 14 Oct 2026

PERM: Psychology-grounded Empathetic Reward Modeling for Large Language Models

arXiv 2601.10532 ↗First posted 15 Jan 2026cs.CLcited by 5

On the map: LLM alignment / RLHF / preference optimisation → Medical LLM alignment

Abstract (arXiv)

Large Language Models (LLMs) are increasingly deployed in human-centric applications, yet they often fail to provide substantive emotional support. While Reinforcement Learning (RL) has been utilized to enhance empathy of LLMs, existing reward models typically evaluate empathy from a single perspective, overlooking the inherently bidirectional interaction nature of empathy between the supporter and seeker as defined by Empathy Cycle theory. To address this limitation, we propose Psychology-grounded Empathetic Reward Modeling (PERM). PERM operationalizes empathy evaluation through a bidirectional decomposition: 1) Supporter perspective, assessing internal resonation and communicative expression; 2) Seeker perspective, evaluating emotional reception. Additionally, it incorporates a bystander perspective to monitor overall interaction quality. Extensive experiments on a widely-used emotional intelligence benchmark and an industrial daily conversation dataset demonstrate that PERM outperforms state-of-the-art baselines by over 10%. Furthermore, a blinded user study reveals a 70% preference for our approach, highlighting its efficacy in generating more empathetic responses. Our code, dataset, and models are available at https://github.com/ZhengWwwq/PERM.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Medical LLM alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 46 of its 97 references that are in the map. First posted on arXiv 15 Jan 2026.

Builds on

46 of 97 references.

References · 46

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2020Learning to summarize from human feedbackcited by 2,588
  7. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  8. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  9. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
  10. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 406
  11. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 214
  12. 2025RewardBench 2: Advancing Reward Model Evaluationcited by 150
34 more
  1. 2025CollabLLM: From Passive Responders to Active Collaboratorscited by 124
  2. 2023SoulChat: Improving LLMs' Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy Conversationscited by 110
  3. 2021PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Supportcited by 92
  4. 2022MISC: A MIxed Strategy-Aware Model Integrating COMET for Emotional Support Conversationcited by 85
  5. 2022AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversationcited by 79
  6. 2021Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approachcited by 75
  7. 2025Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalizationcited by 45
  8. 2025Semantic-Aware Logical Reasoning via a Semiotic Frameworkcited by 32
  9. 2025NextQuill: Causal Preference Modeling for Enhancing LLM Personalizationcited by 30
  10. 2025Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generationcited by 30
  11. 2025RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agentscited by 25
  12. 2026Logical Phase Transitions: Understanding Collapse in LLM Logical Reasoningcited by 25
  13. 2025From Personas to Talks: Revisiting the Impact of Personas on LLM-Synthesized Emotional Support Conversationscited by 23
  14. 2025Kardia-R1: Unleashing LLMs to Reason toward Understanding and Empathy for Emotional Support via Rubric-as-Judge Reinforcement Learningcited by 22
  15. 2025Latent Inter-User Difference Modeling for LLM Personalizationcited by 21
  16. 2025Training language models to be warm and empathetic makes them less reliable and more sycophanticcited by 21
  17. 2025"My Boyfriend is AI": A Computational Analysis of Human-AI Companionship in Reddit's AI Communitycited by 20
  18. 2026ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selectioncited by 18
  19. 2025Explicit v.s. Implicit Memory: Exploring Multi-hop Complex Reasoning Over Personalized Informationcited by 17
  20. 2025Semantic-Space Exploration and Exploitation in RLVR for LLM Reasoningcited by 17
  21. 2026Double: Breaking the Acceleration Limit via Double Retrieval Speculative Parallelismcited by 16
  22. 2024The Illusion of Empathy: How AI Chatbots Shape Conversation Perceptioncited by 12
  23. 2025InSPO: Unlocking Intrinsic Self-Reflection for LLM Preference Optimizationcited by 11
  24. 2025Learning to Substitute Words with Model-based Score Rankingcited by 10
  25. 2025Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applicationscited by 10
  26. 2025CausalAbstain: Enhancing Multilingual LLMs with Causal Reasoning for Trustworthy Abstentioncited by 10
  27. 2025Are LLMs Empathetic to All? Investigating the Influence of Multi-Demographic Personas on a Model's Empathycited by 10
  28. 2025SENSE-7: Taxonomy and Dataset for Measuring User Perceptions of Empathy in Sustained Human-AI Conversationscited by 9
  29. 2024Empathy Level Alignment via Reinforcement Learning for Empathetic Response Generationcited by 8
  30. 2024Leveraging Memory Retrieval to Enhance LLM-based Generative Recommendationcited by 6
  31. 2025The Pursuit of Empathy: Evaluating Small Language Models for PTSD Dialogue Supportcited by 6
  32. 2024My Words Imply Your Opinion: Reader Agent-based Propagation Enhancement for Personalized Implicit Emotion Analysiscited by 4
  33. 2025LongEmotion: Measuring Emotional Intelligence of Large Language Models in Long-Context Interactioncited by 3
  34. 2025RECAP: Transparent Inference-Time Emotion Alignment for Medical Dialogue Systemscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Reason in Chains, Learn in Trees: Self-Rectification and Grafting for Multi-turn Agent Policy Optimizationcited by 8
  2. 2026EmoLLM: Appraisal-Grounded Cognitive-Emotional Co-Reasoning in Large Language Modelscited by 0
  3. 2026LLMs Should Incorporate Explicit Mechanisms for Human Empathycited by 0
  4. 2026Towards High-Level Semantic Intelligencecited by 0
  5. 2026Evolving Support Priorities in Empathetic Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026From Empathy to Personalized Empathy: Adapting Empathetic Strategies to Individual Users
  2. 2025PEER: Unified Process-Outcome Reinforcement Learning for Structured Empathetic Reasoning
  3. 2024EmPO: Emotion Grounding for Empathetic Response Generation through Preference Optimization
  4. 2025From Measurement to Expertise: Empathetic Expert Adapters for Context-Based Empathy in Conversational AI Agents
  5. 2024ReflectDiffu:Reflect between Emotion-intent Contagion and Mimicry for Empathetic Response Generation via a RL-Diffusion Framework
  6. 2025From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignment
  7. 2025Empathy-R1: A Chain-of-Empathy and Reinforcement Learning Framework for Long-Form Mental Health Support
  8. 2025Psyche-R1: Towards Reliable Psychological LLMs through Unified Empathy, Expertise, and Reasoning