# Learning to Respond with Stickers: A Framework of Unifying Multi-Modality in Multi-Turn Dialog

arXiv: https://arxiv.org/abs/2003.04679 · HTML: https://mlsanity.com/paper/2003.04679/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2020.

## Built on by (most cited first, 13 of 13)

- [STICKERCONV: Generating Multimodal Empathetic Responses from Scratch](https://mlsanity.com/paper/2402.01679/) (2024)
- [Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots](https://mlsanity.com/paper/2103.09534/) (2021)
- [Towards Expressive Communication with Internet Memes: A New Multimodal Conversation Dataset and Benchmark](https://mlsanity.com/paper/2109.01839/) (2021)
- [EZInterviewer: To Improve Job Interview Performance with Mock Interview Generator](https://mlsanity.com/paper/2301.00972/) (2023)
- [Sticker820K: Empowering Interactive Retrieval with Stickers](https://mlsanity.com/paper/2306.06870/) (2023)
- [Emotion and Intention Guided Multi-Modal Learning for Sticker Response Selection](https://mlsanity.com/paper/2511.17587/) (2025)
- [Selecting Stickers in Open-Domain Dialogue through Multitask Learning](https://mlsanity.com/paper/2209.07697/) (2022)
- [PerSRV: Personalized Sticker Retrieval with Vision-Language Model](https://mlsanity.com/paper/2410.21801/) (2024)
- [RDTF: Resource-efficient Dual-mask Training Framework for Multi-frame Animated Sticker Generation](https://mlsanity.com/paper/2503.17735/) (2025)
- [Automatic Scene-based Topic Channel Construction System for E-Commerce](https://mlsanity.com/paper/2210.02643/) (2022)
- [Selecting Query-bag as Pseudo Relevance Feedback for Information-seeking Conversations](https://mlsanity.com/paper/2404.04272/) (2024)
- [MemeCMD: An Automatically Generated Chinese Multi-turn Dialogue Dataset with Contextually Retrieved Memes](https://mlsanity.com/paper/2507.00891/) (2025)
- [GSE: Evaluating Sticker Visual Semantic Similarity via a General Sticker Encoder](https://mlsanity.com/paper/2511.04977/) (2025)

## Abstract (arXiv)

Stickers with vivid and engaging expressions are becoming increasingly popular in online messaging apps, and some works are dedicated to automatically select sticker response by matching text labels of stickers with previous utterances. However, due to their large quantities, it is impractical to require text labels for the all stickers. Hence, in this paper, we propose to recommend an appropriate sticker to user based on multi-turn dialog context history without any external labels. Two main challenges are confronted in this task. One is to learn semantic meaning of stickers without corresponding text labels. Another challenge is to jointly model the candidate sticker with the multi-turn dialog context. To tackle these challenges, we propose a sticker response selector (SRS) model. Specifically, SRS first employs a convolutional based sticker image encoder and a self-attention based multi-turn dialog encoder to obtain the representation of stickers and utterances. Next, deep interaction network is proposed to conduct deep matching between the sticker with each utterance in the dialog history. SRS then learns the short-term and long-term dependency between all interaction results by a fusion network to output the the final matching score. To evaluate our proposed method, we collect a large-scale real-world dialog dataset with stickers from one of the most popular online chatting platform. Extensive experiments conducted on this dataset show that our model achieves the state-of-the-art performance for all commonly-used metrics. Experiments also verify the effectiveness of each component of SRS. To facilitate further research in sticker selection field, we release this dataset of 340K multi-turn dialog and sticker pairs.