# The Hidden Language of Harm: Examining the Role of Emojis in Harmful Online Communication and Content Moderation

arXiv: https://arxiv.org/abs/2506.00583 · HTML: https://mlsanity.com/paper/2506.00583/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 51 references that are in the map.
- First posted on arXiv 31 May 2025.

## Builds on (most cited first, 26 of 26)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification](https://mlsanity.com/paper/2010.12421/) (2020)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://mlsanity.com/paper/1905.12516/) (2019)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech](https://mlsanity.com/paper/2302.07736/) (2023)
- [Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm](https://mlsanity.com/paper/1708.00524/) (2017)
- [The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text](https://mlsanity.com/paper/2311.09807/) (2023)
- [Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks](https://mlsanity.com/paper/2304.10145/) (2023)
- [COLD: A Benchmark for Chinese Offensive Language Detection](https://mlsanity.com/paper/2201.06025/) (2022)
- [Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hate](https://mlsanity.com/paper/2108.05921/) (2021)
- ["HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social media](https://mlsanity.com/paper/2304.10619/) (2023)
- [Detecting Offensive Language in Tweets Using Deep Learning](https://mlsanity.com/paper/1801.04433/) (2018)
- [Cross-Cultural Transfer Learning for Chinese Offensive Language Detection](https://mlsanity.com/paper/2303.17927/) (2023)
- [Personalizing Content Moderation on Social Media: User Perspectives on Moderation Choices, Interface Design, and Labor](https://mlsanity.com/paper/2305.10374/) (2023)
- [MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEs](https://mlsanity.com/paper/2502.00997/) (2025)
- [Spice up Your Chat: The Intentions and Sentiment Effects of Using Emoji](https://mlsanity.com/paper/1703.02860/) (2017)
- [Emoji-Powered Representation Learning for Cross-Lingual Sentiment Classification](https://mlsanity.com/paper/1806.02557/) (2018)
- [Human vs. LMMs: Exploring the Discrepancy in Emoji Interpretation and Usage in Digital Communication](https://mlsanity.com/paper/2401.08212/) (2024)
- [Emojis Decoded: Leveraging ChatGPT for Enhanced Understanding in Social Media Communications](https://mlsanity.com/paper/2402.01681/) (2024)
- [Emoji Promotes Developer Participation and Issue Resolution on GitHub](https://mlsanity.com/paper/2308.16360/) (2023)
- [BEIKE NLP at SemEval-2022 Task 4: Prompt-Based Paragraph Classification for Patronizing and Condescending Language Detection](https://mlsanity.com/paper/2208.01312/) (2022)
- [Detecting and Reducing Bias in a High Stakes Domain](https://mlsanity.com/paper/1908.11474/) (2019)
- [From Adoption to Adaption: Tracing the Diffusion of New Emojis on Twitter](https://mlsanity.com/paper/2402.14187/) (2024)

## Abstract (arXiv)

Social media platforms have become central to modern communication, yet they also harbor offensive content that challenges platform safety and inclusivity. While prior research has primarily focused on textual indicators of offense, the role of emojis, ubiquitous visual elements in online discourse, remains underexplored. Emojis, despite being rarely offensive in isolation, can acquire harmful meanings through symbolic associations, sarcasm, and contextual misuse. In this work, we systematically examine emoji contributions to offensive Twitter messages, analyzing their distribution across offense categories and how users exploit emoji ambiguity. To address this, we propose an LLM-powered, multi-step moderation pipeline that selectively replaces harmful emojis while preserving the tweet's semantic intent. Human evaluations confirm our approach effectively reduces perceived offensiveness without sacrificing meaning. Our analysis also reveals heterogeneous effects across offense types, offering nuanced insights for online communication and emoji moderation.