# PRIDE: Privileged Information-enhanced Distillation for Empathetic Dialogue Generation

arXiv: https://arxiv.org/abs/2606.23124 · HTML: https://mlsanity.com/paper/2606.23124/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 22 Jun 2026.

## Builds on (most cited first, 29 of 29)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023)
- [Self-Rewarding Language Models](https://mlsanity.com/paper/2401.10020/) (2024)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [MIME: MIMicking Emotions for Empathetic Response Generation](https://mlsanity.com/paper/2010.01454/) (2020)
- [MoEL: Mixture of Empathetic Listeners](https://mlsanity.com/paper/1908.07687/) (2019)
- [CEM: Commonsense-aware Empathetic Response Generation](https://mlsanity.com/paper/2109.05739/) (2021)
- [Is ChatGPT Equipped with Emotional Dialogue Capabilities?](https://mlsanity.com/paper/2304.09582/) (2023)
- [Knowledge Bridging for Empathetic Dialogue Generation](https://mlsanity.com/paper/2009.09708/) (2020)
- [LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning](https://mlsanity.com/paper/2004.12817/) (2020)
- [ABKD: Pursuing a Proper Allocation of the Probability Mass in Knowledge Distillation via α-β-Divergence](https://mlsanity.com/paper/2505.04560/) (2025)
- [STICKERCONV: Generating Multimodal Empathetic Responses from Scratch](https://mlsanity.com/paper/2402.01679/) (2024)
- [Empathetic Dialogue Generation via Sensitive Emotion Recognition and Sensible Knowledge Selection](https://mlsanity.com/paper/2210.11715/) (2022)
- [CASE: Aligning Coarse-to-Fine Cognition and Affection for Empathetic Response Generation](https://mlsanity.com/paper/2208.08845/) (2022)
- [Towards Multimodal Empathetic Response Generation: A Rich Text-Speech-Vision Avatar-based Benchmark](https://mlsanity.com/paper/2502.04976/) (2025)
- [EmpathyEar: An Open-source Avatar Multimodal Empathetic Chatbot](https://mlsanity.com/paper/2406.15177/) (2024)
- [Don't Lose Yourself! Empathetic Response Generation via Explicit Self-Other Awareness](https://mlsanity.com/paper/2210.03884/) (2022)
- [Empathetic Conversational Systems: A Review of Current Advances, Gaps, and Opportunities](https://mlsanity.com/paper/2206.05017/) (2022)
- [LLaVA-Gemma: Accelerating Multimodal Foundation Models with a Compact Language Model](https://mlsanity.com/paper/2404.01331/) (2024)
- [Talk With Human-like Agents: Empathetic Dialogue Through Perceptible Acoustic Reception and Reaction](https://mlsanity.com/paper/2406.12707/) (2024)
- [MEDIC: A Multimodal Empathy Dataset in Counseling](https://mlsanity.com/paper/2305.02842/) (2023)
- [An Iterative Associative Memory Model for Empathetic Response Generation](https://mlsanity.com/paper/2402.17959/) (2024)

## Abstract (arXiv)

Large language models have demonstrated significant capabilities in generating diverse and context-aware responses for empathetic dialogue. However, their computational demands severely limit their deployment in resource-constrained environments. While knowledge distillation offers a promising compression solution, it often fails to transfer the nuanced understanding essential for empathy, as it overlooks the implicit contextual cues that guide human connection. To bridge this gap, we propose a privileged information-enhanced knowledge distillation method for empathetic dialogue generation (PRIDE). Our method leverages privileged information, such as expert psychological annotations or future event summaries, which is available exclusively during training but unavailable at inference time. This allows us to transfer the teacher model's empathetic reasoning to smaller models without relying on extra inputs during deployment. Specifically, PRIDE has three key components: (1) An empathy-reasoning prompt that guides the teacher to explicitly decompose the empathetic process into understanding feelings and analyzing situations step-by-step; (2) A multi-source attention mechanism that directs the student to effectively integrate privileged information; (3) A dual-alignment loss that combines reversed Kullback-Leibler divergence and maximum mean discrepancy to ensure robust knowledge transfer at both logit and feature levels. Experiments on multi-modal and text-only datasets demonstrate that our method achieves competitive performance, and in some cases matches or even surpasses larger teacher models in terms of accuracy and semantic relevance.