# GPT-DETOX: An In-Context Learning-Based Paraphraser for Text Detoxification

arXiv: https://arxiv.org/abs/2404.03052 · HTML: https://mlsanity.com/paper/2404.03052/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 3 Apr 2024.

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023)
- [How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation](https://mlsanity.com/paper/1603.08023/) (2016)
- [Generating Training Data with Language Models: Towards Zero-Shot Language Understanding](https://mlsanity.com/paper/2202.04538/) (2022)
- [Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer](https://mlsanity.com/paper/1804.06437/) (2018)
- [GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation](https://mlsanity.com/paper/2104.08826/) (2021)
- [Beyond BLEU: Training Neural Machine Translation with Semantic Similarity](https://mlsanity.com/paper/1909.06694/) (2019)
- [A Probabilistic Formulation of Unsupervised Text Style Transfer](https://mlsanity.com/paper/2002.03912/) (2020)
- [Few-Shot Bot: Prompt-Based Learning for Dialogue Systems](https://mlsanity.com/paper/2110.08118/) (2021)
- [Text Detoxification using Large Pre-trained Neural Models](https://mlsanity.com/paper/2109.08914/) (2021)
- ["Mask and Infill" : Applying Masked Language Model to Sentiment Transfer](https://mlsanity.com/paper/1908.08039/) (2019)
- [APPDIA: A Discourse-aware Transformer-based Style Transfer Model for Offensive Social Media Conversations](https://mlsanity.com/paper/2209.08207/) (2022)
- [Stable Style Transformer: Delete and Generate Approach with Encoder-Decoder for Text Style Transfer](https://mlsanity.com/paper/2005.12086/) (2020)
- [Exploring Prompt-based Few-shot Learning for Grounded Dialog Generation](https://mlsanity.com/paper/2109.06513/) (2021)
- [Non-Parallel Text Style Transfer with Self-Parallel Supervision](https://mlsanity.com/paper/2204.08123/) (2022)
- [DiffuDetox: A Mixed Diffusion Model for Text Detoxification](https://mlsanity.com/paper/2306.08505/) (2023)

## Built on by (most cited first, 2 of 2)

- [K/DA: Automated Data Generation Pipeline for Detoxifying Implicitly Offensive Language in Korean](https://mlsanity.com/paper/2506.13513/) (2025)
- [GemDetox at TextDetox CLEF 2025: Enhancing a Massively Multilingual Model for Text Detoxification on Low-resource Languages](https://mlsanity.com/paper/2510.01250/) (2025)

## Abstract (arXiv)

Harmful and offensive communication or content is detrimental to social bonding and the mental state of users on social media platforms. Text detoxification is a crucial task in natural language processing (NLP), where the goal is removing profanity and toxicity from text while preserving its content. Supervised and unsupervised learning are common approaches for designing text detoxification solutions. However, these methods necessitate fine-tuning, leading to computational overhead. In this paper, we propose GPT-DETOX as a framework for prompt-based in-context learning for text detoxification using GPT-3.5 Turbo. We utilize zero-shot and few-shot prompting techniques for detoxifying input sentences. To generate few-shot prompts, we propose two methods: word-matching example selection (WMES) and context-matching example selection (CMES). We additionally take into account ensemble in-context learning (EICL) where the ensemble is shaped by base prompts from zero-shot and all few-shot settings. We use ParaDetox and APPDIA as benchmark detoxification datasets. Our experimental results show that the zero-shot solution achieves promising performance, while our best few-shot setting outperforms the state-of-the-art models on ParaDetox and shows comparable results on APPDIA. Our EICL solutions obtain the greatest performance, adding at least 10% improvement, against both datasets.