# Semantics Preserving Emoji Recommendation with Large Language Models

arXiv: https://arxiv.org/abs/2409.10760 · HTML: https://mlsanity.com/paper/2409.10760/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 16 Sep 2024.

## Builds on (most cited first, 21 of 21)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models](https://mlsanity.com/paper/2304.03738/) (2023)
- [Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm](https://mlsanity.com/paper/1708.00524/) (2017)
- [Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks](https://mlsanity.com/paper/2304.10145/) (2023)
- [MojiTalk: Generating Emotional Responses at Scale](https://mlsanity.com/paper/1711.04090/) (2017)
- [SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning](https://mlsanity.com/paper/2307.10234/) (2023)
- [Image Sentiment Transfer](https://mlsanity.com/paper/2006.11337/) (2020)
- [Semi-automatic Generation of Multilingual Datasets for Stance Detection in Twitter](https://mlsanity.com/paper/2101.11978/) (2021)
- [Human vs. LMMs: Exploring the Discrepancy in Emoji Interpretation and Usage in Digital Communication](https://mlsanity.com/paper/2401.08212/) (2024)
- [Multimodal Emoji Prediction](https://mlsanity.com/paper/1803.02392/) (2018)
- [Emojis Decoded: Leveraging ChatGPT for Enhanced Understanding in Social Media Communications](https://mlsanity.com/paper/2402.01681/) (2024)
- [EmojiLM: Modeling the New Emoji Language](https://mlsanity.com/paper/2311.01751/) (2023)
- [Exploring Emoji Usage and Prediction Through a Temporal Variation Lens](https://mlsanity.com/paper/1805.00731/) (2018)
- [Neural Emoji Recommendation in Dialogue Systems](https://mlsanity.com/paper/1612.04609/) (2016)
- [A Semantics-Based Measure of Emoji Similarity](https://mlsanity.com/paper/1707.04653/) (2017)

## Built on by (most cited first, 1 of 1)

- [Irony in Emojis: A Comparative Study of Human and LLM Interpretation](https://mlsanity.com/paper/2501.11241/) (2025)

## Abstract (arXiv)

Emojis have become an integral part of digital communication, enriching text by conveying emotions, tone, and intent. Existing emoji recommendation methods are primarily evaluated based on their ability to match the exact emoji a user chooses in the original text. However, they ignore the essence of users' behavior on social media in that each text can correspond to multiple reasonable emojis. To better assess a model's ability to align with such real-world emoji usage, we propose a new semantics preserving evaluation framework for emoji recommendation, which measures a model's ability to recommend emojis that maintain the semantic consistency with the user's text. To evaluate how well a model preserves semantics, we assess whether the predicted affective state, demographic profile, and attitudinal stance of the user remain unchanged. If these attributes are preserved, we consider the recommended emojis to have maintained the original semantics. The advanced abilities of Large Language Models (LLMs) in understanding and generating nuanced, contextually relevant output make them well-suited for handling the complexities of semantics preserving emoji recommendation. To this end, we construct a comprehensive benchmark to systematically assess the performance of six proprietary and open-source LLMs using different prompting techniques on our task. Our experiments demonstrate that GPT-4o outperforms other LLMs, achieving a semantics preservation score of 79.23%. Additionally, we conduct case studies to analyze model biases in downstream classification tasks and evaluate the diversity of the recommended emojis.