# Towards a Unified Conversational Recommendation System: Multi-task Learning via Contextualized Knowledge Distillation

arXiv: https://arxiv.org/abs/2310.18119 · HTML: https://mlsanity.com/paper/2310.18119/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for recommender systems, one of 12 directions in Knowledge distillation.
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 27 Oct 2023.

## Placed in

- Knowledge distillation / [Distillation for recommender systems](https://mlsanity.com/distill/distillation-for-recommender-systems/)

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [Towards Deep Conversational Recommendations](https://mlsanity.com/paper/1812.07617/) (2018)
- [Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion](https://mlsanity.com/paper/2007.04032/) (2020)
- [Advances and Challenges in Conversational Recommender Systems: A Survey](https://mlsanity.com/paper/2101.09459/) (2021)
- [Towards Knowledge-Based Recommender Dialog System](https://mlsanity.com/paper/1908.05391/) (2019)
- [Few-shot Natural Language Generation for Task-Oriented Dialog](https://mlsanity.com/paper/2002.12328/) (2020)
- [Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning](https://mlsanity.com/paper/2206.09363/) (2022)
- [Training Neural Response Selection for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1906.01543/) (2019)
- [RevCore: Review-augmented Conversational Recommendation](https://mlsanity.com/paper/2106.00957/) (2021)
- [Pointing the Unknown Words](https://mlsanity.com/paper/1603.08148/) (2016)
- [A Unified Multi-task Learning Framework for Multi-goal Conversational Recommender Systems](https://mlsanity.com/paper/2204.06923/) (2022)
- [Extreme Adaptation for Personalized Neural Machine Translation](https://mlsanity.com/paper/1805.01817/) (2018)
- [Improving Conversational Recommendation Systems' Quality with Context-Aware Item Meta Information](https://mlsanity.com/paper/2112.08140/) (2021)
- [Learning Neural Templates for Recommender Dialogue System](https://mlsanity.com/paper/2109.12302/) (2021)
- [Training Deep AutoEncoders for Collaborative Filtering](https://mlsanity.com/paper/1708.01715/) (2017)

## Built on by (most cited first, 4 of 4)

- [IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce](https://mlsanity.com/paper/2406.10173/) (2024)
- [MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLU](https://mlsanity.com/paper/2408.08144/) (2024)
- [Efficient LLM-based Advertising via Model Compression and Parallel Verification](https://mlsanity.com/paper/2605.11582/) (2026)
- [LumiCRS: Asymmetric Contrastive Prototype Learning for Long-Tail Conversational Recommender Systems](https://mlsanity.com/paper/2507.04722/) (2025)

## Abstract (arXiv)

In Conversational Recommendation System (CRS), an agent is asked to recommend a set of items to users within natural language conversations. To address the need for both conversational capability and personalized recommendations, prior works have utilized separate recommendation and dialogue modules. However, such approach inevitably results in a discrepancy between recommendation results and generated responses. To bridge the gap, we propose a multi-task learning for a unified CRS, where a single model jointly learns both tasks via Contextualized Knowledge Distillation (ConKD). We introduce two versions of ConKD: hard gate and soft gate. The former selectively gates between two task-specific teachers, while the latter integrates knowledge from both teachers. Our gates are computed on-the-fly in a context-specific manner, facilitating flexible integration of relevant knowledge. Extensive experiments demonstrate that our single model significantly improves recommendation performance while enhancing fluency, and achieves comparable results in terms of diversity.