# EmojiLM: Modeling the New Emoji Language

arXiv: https://arxiv.org/abs/2311.01751 · HTML: https://mlsanity.com/paper/2311.01751/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 18 references that are in the map.
- First posted on arXiv 3 Nov 2023.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [Does Synthetic Data Generation of LLMs Help Clinical Text Mining?](https://mlsanity.com/paper/2303.04360/) (2023)
- [Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hate](https://mlsanity.com/paper/2108.05921/) (2021)

## Built on by (most cited first, 6 of 6)

- [Emojis Decoded: Leveraging ChatGPT for Enhanced Understanding in Social Media Communications](https://mlsanity.com/paper/2402.01681/) (2024)
- [On-Device Emoji Classifier Trained with GPT-based Data Augmentation for a Mobile Keyboard](https://mlsanity.com/paper/2411.05031/) (2024)
- [When Smiley Turns Hostile: Interpreting How Emojis Trigger LLMs' Toxicity](https://mlsanity.com/paper/2509.11141/) (2025)
- [Semantics Preserving Emoji Recommendation with Large Language Models](https://mlsanity.com/paper/2409.10760/) (2024)
- [PerSRV: Personalized Sticker Retrieval with Vision-Language Model](https://mlsanity.com/paper/2410.21801/) (2024)
- [The Overlooked Repetitive Lengthening Form in Sentiment Analysis](https://mlsanity.com/paper/2604.01268/) (2026)

## Abstract (arXiv)

With the rapid development of the internet, online social media welcomes people with different backgrounds through its diverse content. The increasing usage of emoji becomes a noticeable trend thanks to emoji's rich information beyond cultural or linguistic borders. However, the current study on emojis is limited to single emoji prediction and there are limited data resources available for further study of the interesting linguistic phenomenon. To this end, we synthesize a large text-emoji parallel corpus, Text2Emoji, from a large language model. Based on the parallel corpus, we distill a sequence-to-sequence model, EmojiLM, which is specialized in the text-emoji bidirectional translation. Extensive experiments on public benchmarks and human evaluation demonstrate that our proposed model outperforms strong baselines and the parallel corpus benefits emoji-related downstream tasks.