# LLM-based Semantic Augmentation for Harmful Content Detection

arXiv: https://arxiv.org/abs/2504.15548 · HTML: https://mlsanity.com/paper/2504.15548/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 20 of its 74 references that are in the map.
- First posted on arXiv 22 Apr 2025.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [GIT: A Generative Image-to-text Transformer for Vision and Language](https://mlsanity.com/paper/2205.14100/) (2022)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024)
- [Generated Knowledge Prompting for Commonsense Reasoning](https://mlsanity.com/paper/2110.08387/) (2021)
- [Is GPT-3 a Good Data Annotator?](https://mlsanity.com/paper/2212.10450/) (2022)
- [Fine-Grained Analysis of Propaganda in News Articles](https://mlsanity.com/paper/1910.02517/) (2019)
- [Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physics](https://mlsanity.com/paper/2304.01938/) (2023)
- [Are Large Language Models Ready for Healthcare? A Comparative Study on Clinical Language Understanding](https://mlsanity.com/paper/2304.05368/) (2023)
- [Language Models for Text Classification: Is In-Context Learning Enough?](https://mlsanity.com/paper/2403.17661/) (2024)
- [A Survey on Data Augmentation in Large Model Era](https://mlsanity.com/paper/2401.15422/) (2024)
- [ApplicaAI at SemEval-2020 Task 11: On RoBERTa-CRF, Span CLS and Whether Self-Training Helps Them](https://mlsanity.com/paper/2005.07934/) (2020)
- [Deciphering Political Entity Sentiment in News with Large Language Models: Zero-Shot and Few-Shot Strategies](https://mlsanity.com/paper/2404.04361/) (2024)
- [A comparative study of zero-shot inference with large language models and supervised modeling in breast cancer pathology classification](https://mlsanity.com/paper/2401.13887/) (2024)

## Built on by (most cited first, 7 of 7)

- [Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM](https://mlsanity.com/paper/2508.05775/) (2025)
- [Boosting Large Language Models for Mental Manipulation Detection via Data Augmentation and Distillation](https://mlsanity.com/paper/2505.15255/) (2025)
- [MENTOR: A Metacognition-Driven Self-Evolution Framework for Uncovering and Mitigating Implicit Domain Risks in LLMs](https://mlsanity.com/paper/2511.07107/) (2025)
- [How DDAIR you? Disambiguated Data Augmentation for Intent Recognition](https://mlsanity.com/paper/2601.11234/) (2026)
- [Are Today's LLMs Ready to Explain Well-Being Concepts?](https://mlsanity.com/paper/2508.03990/) (2025)
- [Backdoor Attacks on Contrastive Continual Learning for IoT Systems](https://mlsanity.com/paper/2602.13062/) (2026)
- [Activation Steering for Synthetic Data Generation: The Role of Diversity in Downstream Safety Detection](https://mlsanity.com/paper/2605.28664/) (2026)

## Abstract (arXiv)

Recent advances in large language models (LLMs) have demonstrated strong performance on simple text classification tasks, frequently under zero-shot settings. However, their efficacy declines when tackling complex social media challenges such as propaganda detection, hateful meme classification, and toxicity identification. Much of the existing work has focused on using LLMs to generate synthetic training data, overlooking the potential of LLM-based text preprocessing and semantic augmentation. In this paper, we introduce an approach that prompts LLMs to clean noisy text and provide context-rich explanations, thereby enhancing training sets without substantial increases in data volume. We systematically evaluate on the SemEval 2024 multi-label Persuasive Meme dataset and further validate on the Google Jigsaw toxic comments and Facebook hateful memes datasets to assess generalizability. Our results reveal that zero-shot LLM classification underperforms on these high-context tasks compared to supervised models. In contrast, integrating LLM-based semantic augmentation yields performance on par with approaches that rely on human-annotated data, at a fraction of the cost. These findings underscore the importance of strategically incorporating LLMs into machine learning (ML) pipeline for social media classification tasks, offering broad implications for combating harmful content online.