# Improving Conversational Recommendation System by Pretraining on Billions Scale of Knowledge Graph

arXiv: https://arxiv.org/abs/2104.14899 · HTML: https://mlsanity.com/paper/2104.14899/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space](https://mlsanity.com/paper/1902.10197/) (2019)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs](https://mlsanity.com/paper/1803.07294/) (2018)
- [Neural Approaches to Conversational AI](https://mlsanity.com/paper/1809.08267/) (2018)
- [Adaptive Graph Convolutional Neural Networks](https://mlsanity.com/paper/1801.03226/) (2018)
- [AliGraph: A Comprehensive Graph Neural Network Platform](https://mlsanity.com/paper/1902.08730/) (2019)
- [DKN: Deep Knowledge-Aware Network for News Recommendation](https://mlsanity.com/paper/1801.08284/) (2018)
- [GeniePath: Graph Neural Networks with Adaptive Receptive Paths](https://mlsanity.com/paper/1802.00910/) (2018)
- [TransEdge: Translating Relation-contextualized Embeddings for Knowledge Graphs](https://mlsanity.com/paper/2004.13579/) (2020)
- [Interaction Embeddings for Prediction and Explanation in Knowledge Graphs](https://mlsanity.com/paper/1903.04750/) (2019)
- [Can a Chatbot Determine My Diet?: Addressing Challenges of Chatbot Application for Meal Recommendation](https://mlsanity.com/paper/1802.09100/) (2018)

## Built on by (most cited first, 9 of 9)

- [A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multimodal](https://mlsanity.com/paper/2212.05767/) (2022)
- [Knowledge Perceived Multi-modal Pretraining in E-commerce](https://mlsanity.com/paper/2109.00895/) (2021)
- [Knowledge Graph Reasoning with Logics and Embeddings: Survey and Perspective](https://mlsanity.com/paper/2202.07412/) (2022)
- [Conversational Recommender System and Large Language Model Are Made for Each Other in E-commerce Pre-sales Dialogue](https://mlsanity.com/paper/2310.14626/) (2023)
- [Learning Instance-Level Representation for Large-Scale Multi-Modal Pretraining in E-commerce](https://mlsanity.com/paper/2304.02853/) (2023)
- [Multi-domain Knowledge Graph Collaborative Pre-training and Prompt Tuning for Diverse Downstream Tasks](https://mlsanity.com/paper/2405.13085/) (2024)
- [INSPIRED2: An Improved Dataset for Sociable Conversational Recommendation](https://mlsanity.com/paper/2208.04104/) (2022)
- [Knowledge Graph Embedding in E-commerce Applications: Attentive Reasoning, Explanations, and Transferable Rules](https://mlsanity.com/paper/2112.08589/) (2021)
- [Multimodal Representation Learning Conditioned on Semantic Relations](https://mlsanity.com/paper/2508.17497/) (2025)

## Abstract (arXiv)

Conversational Recommender Systems (CRSs) in E-commerce platforms aim to recommend items to users via multiple conversational interactions. Click-through rate (CTR) prediction models are commonly used for ranking candidate items. However, most CRSs are suffer from the problem of data scarcity and sparseness. To address this issue, we propose a novel knowledge-enhanced deep cross network (K-DCN), a two-step (pretrain and fine-tune) CTR prediction model to recommend items. We first construct a billion-scale conversation knowledge graph (CKG) from information about users, items and conversations, and then pretrain CKG by introducing knowledge graph embedding method and graph convolution network to encode semantic and structural information respectively.To make the CTR prediction model sensible of current state of users and the relationship between dialogues and items, we introduce user-state and dialogue-interaction representations based on pre-trained CKG and propose K-DCN.In K-DCN, we fuse the user-state representation, dialogue-interaction representation and other normal feature representations via deep cross network, which will give the rank of candidate items to be recommended.We experimentally prove that our proposal significantly outperforms baselines and show it's real application in Alime.