# Multimodal Emoji Prediction

arXiv: https://arxiv.org/abs/1803.02392 · HTML: https://mlsanity.com/paper/1803.02392/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2018.

## Built on by (most cited first, 7 of 7)

- [A Survey on Multi-modal Summarization](https://mlsanity.com/paper/2109.05199/) (2021)
- [Towards Expressive Communication with Internet Memes: A New Multimodal Conversation Dataset and Benchmark](https://mlsanity.com/paper/2109.01839/) (2021)
- [Selecting Stickers in Open-Domain Dialogue through Multitask Learning](https://mlsanity.com/paper/2209.07697/) (2022)
- [Semantics Preserving Emoji Recommendation with Large Language Models](https://mlsanity.com/paper/2409.10760/) (2024)
- [The Overlooked Repetitive Lengthening Form in Sentiment Analysis](https://mlsanity.com/paper/2604.01268/) (2026)
- [Contrastive Learning-based Multi Modal Architecture for Emoticon Prediction by Employing Image-Text Pairs](https://mlsanity.com/paper/2408.02571/) (2024)
- [MemeCMD: An Automatically Generated Chinese Multi-turn Dialogue Dataset with Contextually Retrieved Memes](https://mlsanity.com/paper/2507.00891/) (2025)

## Abstract (arXiv)

Emojis are small images that are commonly included in social media text messages. The combination of visual and textual content in the same message builds up a modern way of communication, that automatic systems are not used to deal with. In this paper we extend recent advances in emoji prediction by putting forward a multimodal approach that is able to predict emojis in Instagram posts. Instagram posts are composed of pictures together with texts which sometimes include emojis. We show that these emojis can be predicted by using the text, but also using the picture. Our main finding is that incorporating the two synergistic modalities, in a combined model, improves accuracy in an emoji prediction task. This result demonstrates that these two modalities (text and images) encode different information on the use of emojis and therefore can complement each other.