mlsanityData to 7 Oct 2026

Leashing the Inner Demons: Self-Detoxification for Language Models

arXiv 2203.03072 ↗First posted 6 Mar 2022cs.CL cs.AI cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

Language models (LMs) can reproduce (or amplify) toxic language seen during training, which poses a risk to their practical application. In this paper, we conduct extensive experiments to study this phenomenon. We analyze the impact of prompts, decoding strategies and training corpora on the output toxicity. Based on our findings, we propose a simple yet effective method for language models to "detoxify" themselves without an additional large corpus or external discriminator. Compared to a supervised baseline, our proposed method shows better toxicity reduction with good generation quality in the generated content under multiple settings. Warning: some examples shown in the paper may contain uncensored offensive content.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 24 of its 30 references that are in the map. First posted on arXiv 6 Mar 2022.

Builds on

24 of 30 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  6. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  7. 2018Hierarchical Neural Story Generationcited by 1,182
  8. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  9. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  10. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  11. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  12. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
12 more
  1. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  2. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  3. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
  4. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  5. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  6. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  7. 2015A Neural Conversational Modelcited by 195
  8. 2021Detoxifying Language Models Risks Marginalizing Minority Voicescited by 106
  9. 2018Learning a Deep Listwise Context Model for Ranking Refinementcited by 62
  10. 2018Feature-Wise Bias Amplificationcited by 31
  11. 2019Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoderscited by 27
  12. 2018Bias Amplification in Artificial Intelligence Systemscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 7; 2024: 4; 2025: 4; 2026: 1.

19 papers.

Sort:

By area:

  1. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  2. 2023Large Language Model Alignment: A Surveycited by 241
  3. 2023Attack Prompt Generation for Red Teaming and Defending Large Language Modelscited by 73
  4. 2023Self-Detoxifying Language Models via Toxification Reversalcited by 45
  5. 2022Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbotscited by 31
  6. 2023MedEval: A Multi-Level, Multi-Task, and Multi-Domain Medical Benchmark for Language Model Evaluationcited by 19
  7. 2022Controlling Bias Exposure for Fair Interpretable Predictionscited by 18
  8. 2024LLM-Personalize: Aligning LLM Planners with Human Preferences via Reinforced Self-Training for Housekeeping Robotscited by 18
  9. 2024Walking in Others' Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Biascited by 17
  10. 2022Synthetic Pre-Training Tasks for Neural Machine Translationcited by 8
  11. 2025UniDetox: Universal Detoxification of Large Language Models via Dataset Distillationcited by 8
  12. 2025IF-GUIDE: Influence Function-Guided Detoxification of LLMscited by 7
7 more
  1. 2023CMD: a framework for Context-aware Model self-Detoxificationcited by 6
  2. 2024Tiny Refinements Elicit Resilience: Toward Efficient Prefix-Model Against LLM Red-Teamingcited by 6
  3. 2024FrenchToxicityPrompts: a Large Benchmark for Evaluating and Mitigating Toxicity in French Textscited by 5
  4. 2023Interactive Text Generationcited by 2
  5. 2025IRepair: An Intent-Aware Approach to Repair Data-Driven Errors in Large Language Modelscited by 1
  6. 2025Combating Toxic Language: A Review of LLM-Based Strategies for Software Engineeringcited by 1
  7. 2026Detoxification for LLM: From Dataset Itselfcited by 0