mlsanityData to 14 Oct 2026

RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents

arXiv 2507.03112 ↗First posted 3 Jul 2025cs.CL cs.AI cs.CYcited by 25

On the map: LLM reasoning → Audio-language model reasoning

also in 2 more
Abstract (arXiv)

Large language models (LLMs) excel at logical and algorithmic reasoning, yet their emotional intelligence (EQ) still lags far behind their cognitive prowess. While reinforcement learning from verifiable rewards (RLVR) has advanced in other domains, its application to dialogue-especially for emotional intelligence-remains underexplored. In this work, we introduce RLVER, the first end-to-end reinforcement learning framework that leverages verifiable emotion rewards from simulated users to cultivate higher-order empathetic abilities in LLMs. Within this framework, self-consistent affective simulated users engage in dialogue rollouts and produce deterministic emotion scores during conversations, serving as reward signals to guide the LLM's learning. Fine-tuning publicly available Qwen2.5-7B-Instruct model with PPO boosts its Sentient-Benchmark score from 13.3 to 79.2 while largely preserving mathematical and coding competence. Extensive experiments reveal that: (i) RLVER consistently improves multiple dialogue capabilities; (ii) Thinking and non-thinking models show distinct trends--thinking models excel in empathy and insight, while non-thinking models favor action; (iii) GRPO often yields stable gains, while PPO can push certain capabilities to a higher ceiling; (iv) More challenging environments are not always better-moderate ones can yield stronger outcomes. Our results show that RLVER is a practical route toward emotionally intelligent and broadly capable language agents.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 45 of its 57 references that are in the map. First posted on arXiv 3 Jul 2025.

Builds on

45 of 57 references.

References · 45

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  9. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  10. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  11. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  12. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
33 more
  1. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  2. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 559
  3. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  4. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 485
  5. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 461
  6. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  7. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
  8. 2021Towards Emotional Support Dialog Systemscited by 299
  9. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  10. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 289
  11. 2025Reasoning with Exploration: An Entropy Perspectivecited by 281
  12. 2025Learning to Reason without External Rewardscited by 210
  13. 2020A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Supportcited by 182
  14. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 166
  15. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 162
  16. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domainscited by 156
  17. 2024NATURAL PLAN: Benchmarking LLMs on Natural Language Planningcited by 128
  18. 2025Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivizationcited by 128
  19. 2023SoulChat: Improving LLMs' Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy Conversationscited by 110
  20. 2025Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoningcited by 106
  21. 2021PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Supportcited by 92
  22. 2022MISC: A MIxed Strategy-Aware Model Integrating COMET for Emotional Support Conversationcited by 85
  23. 2025RLPR: Extrapolating RLVR to General Domains without Verifierscited by 85
  24. 2022AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversationcited by 79
  25. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspectivecited by 76
  26. 2022Control Globally, Understand Locally: A Global-to-Local Hierarchical Graph Network for Emotional Support Conversationcited by 56
  27. 2023SMILE: Single-turn to Multi-turn Inclusive Language Expansion via ChatGPT for Mental Health Supportcited by 46
  28. 2023ChatCounselor: A Large Language Models for Mental Health Supportcited by 44
  29. 2025S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learningcited by 39
  30. 2025Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewardscited by 38
  31. 2025DeepTheorem: Advancing LLM Reasoning for Theorem Proving Through Natural Language and Reinforcement Learningcited by 24
  32. 2025Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Modelscited by 21
  33. 2025Chain of Strategy Optimization Makes Large Language Models Better Emotional Supportercited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 21.

25 papers.

Sort:

By area:

  1. 2025How Far Are AI Scientists from Changing the World?cited by 15
  2. 2025DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Searchcited by 11
  3. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learningcited by 9
  4. 2025Enhancing Large Language Model Reasoning with Reward Models: An Analytical Surveycited by 8
  5. 2026Character-R1: Enhancing Role-Aware Reasoning in Role-Playing Agents via RLVRcited by 5
  6. 2026PERM: Psychology-grounded Empathetic Reward Modeling for Large Language Modelscited by 5
  7. 2026MICA: Multi-granularity Intertemporal Credit Assignment for Long-Horizon Emotional Support Dialoguecited by 3
  8. 2026EIBench: A Simulator-Based Benchmark and Turn-Credit RL for Emotion Managementcited by 3
  9. 2025Social Welfare Function Leaderboard: When LLM Agents Allocate Social Welfarecited by 2
  10. 2026Molecular Identifier Visual Prompt and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsingcited by 2
  11. 2026Listening to the Echo: User-Reaction Aware Policy Optimization via Scalar-Verbal Hybrid Reinforcement Learningcited by 1
  12. 2026TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbotscited by 1
13 more
  1. 2026Discourse Diversity in Multi-Turn Empathic Dialoguecited by 1
  2. 2026ESC-Skills: Discovering and Self-Evolving Skills for Emotional Support Conversationscited by 1
  3. 2026EmoLLM: Appraisal-Grounded Cognitive-Emotional Co-Reasoning in Large Language Modelscited by 0
  4. 2026LLMs Should Incorporate Explicit Mechanisms for Human Empathycited by 0
  5. 2026Can You Break RLVER? Probing Adversarial Robustness of RL-Trained Empathetic Agentscited by 0
  6. 2026CRPO: Character-centric Group Relative Policy Optimization for Role-aware Reasoning in Role-playing Agentscited by 0
  7. 2026From Empathy to Personalized Empathy: Adapting Empathetic Strategies to Individual Userscited by 0
  8. 2026When Clients Stop Following: A Cognitive Conceptualization Diagram-driven Framework for Strategic Counselingcited by 0
  9. 2026Can the Environment Speak for Itself? T2-GRPO: A Turn-Trajectory Group Relative Policy Optimization for Caregiver Agentscited by 0
  10. 2026From Pattern Recognizers to Personalized Companions: A Survey of Large Language Models in Mental Healthcited by 0
  11. 2026TRACER: Trajectory-Aligned Learning for Multi-Turn User Simulationcited by 0
  12. 2026Evolving Support Priorities in Empathetic Reinforcement Learningcited by 0
  13. 2026FACES: Fine-grained Gated Semantic Alignment for Multimodal Emotional Support Conversationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025PEER: Unified Process-Outcome Reinforcement Learning for Structured Empathetic Reasoning
  2. 2026Dual-Loop Self-Evolution via Verifiable Emotion Feedback for Multi-Turn Empathetic Dialogue
  3. 2026EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models
  4. 2026STRIDE-ED: A Strategy-Grounded Stepwise Reasoning Framework for Empathetic Dialogue Systems
  5. 2024Empathy Level Alignment via Reinforcement Learning for Empathetic Response Generation
  6. 2025Kardia-R1: Unleashing LLMs to Reason toward Understanding and Empathy for Emotional Support via Rubric-as-Judge Reinforcement Learning
  7. 2025Towards Open-Ended Emotional Support Conversations in LLMs via Reinforcement Learning with Future-Oriented Rewards
  8. 2024ReflectDiffu:Reflect between Emotion-intent Contagion and Mimicry for Empathetic Response Generation via a RL-Diffusion Framework