# You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Content

arXiv: https://arxiv.org/abs/2308.05596 · HTML: https://mlsanity.com/paper/2308.05596/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 67 references that are in the map.
- First posted on arXiv 10 Aug 2023.

## Builds on (most cited first, 30 of 44)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference](https://mlsanity.com/paper/2001.07676/) (2020)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping](https://mlsanity.com/paper/2002.06305/) (2020)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://mlsanity.com/paper/2104.06599/) (2021)
- [True Few-Shot Learning with Language Models](https://mlsanity.com/paper/2105.11447/) (2021)
- [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://mlsanity.com/paper/2103.00453/) (2021)
- [HateCheck: Functional Tests for Hate Speech Detection Models](https://mlsanity.com/paper/2012.15606/) (2020)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks](https://mlsanity.com/paper/1903.05987/) (2019)
- [Constrained Language Models Yield Few-Shot Semantic Parsers](https://mlsanity.com/paper/2104.08768/) (2021)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [Deceiving Google's Perspective API Built for Detecting Toxic Comments](https://mlsanity.com/paper/1702.08138/) (2017)
- [Beyond BLEU: Training Neural Machine Translation with Semantic Similarity](https://mlsanity.com/paper/1909.06694/) (2019)
- [Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfer](https://mlsanity.com/paper/1908.09368/) (2019)
- [A Probabilistic Formulation of Unsupervised Text Style Transfer](https://mlsanity.com/paper/2002.03912/) (2020)

## Built on by (most cited first, 30 of 39)

- [A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2312.02003/) (2023)
- [PLeak: Prompt Leaking Attacks against Large Language Model Applications](https://mlsanity.com/paper/2405.06823/) (2024)
- [Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models](https://mlsanity.com/paper/2310.13828/) (2023)
- [Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems](https://mlsanity.com/paper/2401.05778/) (2024)
- [Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts](https://mlsanity.com/paper/2407.15050/) (2024)
- [VLM as Policy: Common-Law Content Moderation Framework for Short Video Platform](https://mlsanity.com/paper/2504.14904/) (2025)
- [An Investigation of Large Language Models for Real-World Hate Speech Detection](https://mlsanity.com/paper/2401.03346/) (2024)
- [Walking in Others' Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Bias](https://mlsanity.com/paper/2407.15366/) (2024)
- [Watch Your Language: Investigating Content Moderation with Large Language Models](https://mlsanity.com/paper/2309.14517/) (2023)
- [MultiParaDetox: Extending Text Detoxification with Parallel Data to New Languages](https://mlsanity.com/paper/2404.02037/) (2024)
- [Quantized Delta Weight Is Safety Keeper](https://mlsanity.com/paper/2411.19530/) (2024)
- [The Dark Side of Human Feedback: Poisoning Large Language Models via User Inputs](https://mlsanity.com/paper/2409.00787/) (2024)
- [Transformation vs Tradition: Artificial General Intelligence (AGI) for Arts and Humanities](https://mlsanity.com/paper/2310.19626/) (2023)
- [Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM](https://mlsanity.com/paper/2508.05775/) (2025)
- [From LLM-anation to LLM-orchestrator: Coordinating Small Models for Data Labeling](https://mlsanity.com/paper/2506.16393/) (2025)
- [LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedback](https://mlsanity.com/paper/2406.03363/) (2024)
- [Multimodal Large Language Models for Phishing Webpage Detection and Identification](https://mlsanity.com/paper/2408.05941/) (2024)
- [Supporting Human Raters with the Detection of Harmful Content using Large Language Models](https://mlsanity.com/paper/2406.12800/) (2024)
- [Paladin: Defending LLM-enabled Phishing Emails with a New Trigger-Tag Paradigm](https://mlsanity.com/paper/2509.07287/) (2025)
- [QGuard:Question-based Zero-shot Guard for Multi-modal LLM Safety](https://mlsanity.com/paper/2506.12299/) (2025)
- [OffensiveLang: A Community Based Implicit Offensive Language Dataset](https://mlsanity.com/paper/2403.02472/) (2024)
- [ToxiFrench: Benchmarking and Enhancing Language Models via CoT Fine-Tuning for French Toxicity Detection](https://mlsanity.com/paper/2508.11281/) (2025)
- [Something Just Like TRuST : Toxicity Recognition of Span and Target](https://mlsanity.com/paper/2506.02326/) (2025)
- [Conditioning Large Language Models on Legal Systems? Detecting Punishable Hate Speech](https://mlsanity.com/paper/2506.03009/) (2025)
- [Argument-Based Consistency in Toxicity Explanations of LLMs](https://mlsanity.com/paper/2506.19113/) (2025)
- [NLP Case Study on Predicting the Before and After of the Ukraine-Russia and Hamas-Israel Conflicts](https://mlsanity.com/paper/2410.06427/) (2024)
- [Soft-Label Integration for Robust Toxicity Classification](https://mlsanity.com/paper/2410.14894/) (2024)
- [Prompt-enhanced Network for Hateful Meme Classification](https://mlsanity.com/paper/2411.07527/) (2024)
- [Benign-to-Toxic Jailbreaking: Inducing Harmful Responses from Harmless Prompts](https://mlsanity.com/paper/2505.21556/) (2025)
- [VModA: An Effective Framework for Adaptive NSFW Image Moderation](https://mlsanity.com/paper/2505.23386/) (2025)

## Abstract (arXiv)

The spread of toxic content online is an important problem that has adverse effects on user experience online and in our society at large. Motivated by the importance and impact of the problem, research focuses on developing solutions to detect toxic content, usually leveraging machine learning (ML) models trained on human-annotated datasets. While these efforts are important, these models usually do not generalize well and they can not cope with new trends (e.g., the emergence of new toxic terms). Currently, we are witnessing a shift in the approach to tackling societal issues online, particularly leveraging large language models (LLMs) like GPT-3 or T5 that are trained on vast corpora and have strong generalizability. In this work, we investigate how we can use LLMs and prompt learning to tackle the problem of toxic content, particularly focusing on three tasks; 1) Toxicity Classification, 2) Toxic Span Detection, and 3) Detoxification. We perform an extensive evaluation over five model architectures and eight datasets demonstrating that LLMs with prompt learning can achieve similar or even better performance compared to models trained on these specific tasks. We find that prompt learning achieves around 10% improvement in the toxicity classification task compared to the baselines, while for the toxic span detection task we find better performance to the best baseline (0.643 vs. 0.640 in terms of F1-score). Finally, for the detoxification task, we find that prompt learning can successfully reduce the average toxicity score (from 0.775 to 0.213) while preserving semantic meaning.