# Towards Building an Open-Domain Dialogue System Incorporated with Internet Memes

arXiv: https://arxiv.org/abs/2203.03835 · HTML: https://mlsanity.com/paper/2203.03835/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Visual Dialog](https://mlsanity.com/paper/1611.08669/) (2016)
- [How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation](https://mlsanity.com/paper/1603.08023/) (2016)
- [Large-Scale Adversarial Training for Vision-and-Language Representation Learning](https://mlsanity.com/paper/2006.06195/) (2020)
- [UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning](https://mlsanity.com/paper/2012.15409/) (2020)
- [UNITER: UNiversal Image-TExt Representation Learning](https://mlsanity.com/paper/1909.11740/) (2019)
- [Training Millions of Personalized Dialogue Agents](https://mlsanity.com/paper/1809.01984/) (2018)
- [PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning](https://mlsanity.com/paper/2006.16779/) (2020)
- [Image Chat: Engaging Grounded Conversations](https://mlsanity.com/paper/1811.00945/) (2018)
- [Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation](https://mlsanity.com/paper/1701.08251/) (2017)
- [PhotoChat: A Human-Human Dialogue Dataset with Photo Sharing Behavior for Joint Image-Text Modeling](https://mlsanity.com/paper/2108.01453/) (2021)
- [Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network](https://mlsanity.com/paper/1902.01019/) (2019)
- [Graph Based Network with Contextualized Representations of Turns in Dialogue](https://mlsanity.com/paper/2109.04008/) (2021)
- [Towards Expressive Communication with Internet Memes: A New Multimodal Conversation Dataset and Benchmark](https://mlsanity.com/paper/2109.01839/) (2021)

## Built on by (most cited first, 5 of 5)

- [DialogCC: An Automated Pipeline for Creating High-Quality Multi-Modal Dialogue Dataset](https://mlsanity.com/paper/2212.04119/) (2022)
- [Tracking Emotional Dynamics in Chat Conversations: A Hybrid Approach using DistilBERT and Emoji Sentiment Analysis](https://mlsanity.com/paper/2408.01838/) (2024)
- [Facilitating Pornographic Text Detection for Open-Domain Dialogue Systems via Knowledge Distillation of Large Language Models](https://mlsanity.com/paper/2403.13250/) (2024)
- [Meme Similarity and Emotion Detection using Multimodal Analysis](https://mlsanity.com/paper/2503.17493/) (2025)
- [Facilitating NSFW Text Detection in Open-Domain Dialogue Systems via Knowledge Distillation](https://mlsanity.com/paper/2309.09749/) (2023)

## Abstract (arXiv)

In recent years, Internet memes have been widely used in online chatting. Compared with text-based communication, conversations become more expressive and attractive when Internet memes are incorporated. This paper presents our solutions for the Meme incorporated Open-domain Dialogue (MOD) Challenge of DSTC10, where three tasks are involved: text response modeling, meme retrieval, and meme emotion classification. Firstly, we leverage a large-scale pre-trained dialogue model for coherent and informative response generation. Secondly, based on interaction-based text-matching, our approach can retrieve appropriate memes with good generalization ability. Thirdly, we propose to model the emotion flow (EF) in conversations and introduce an auxiliary task of emotion description prediction (EDP) to boost the performance of meme emotion classification. Experimental results on the MOD dataset demonstrate that our methods can incorporate Internet memes into dialogue systems effectively.