mlsanityData to 14 Oct 2026

Detoxifying Language Models with a Toxic Corpus

arXiv 2205.00320 ↗First posted 30 Apr 2022cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Existing studies have investigated the tendency of autoregressive language models to generate contexts that exhibit undesired biases and toxicity. Various debiasing approaches have been proposed, which are primarily categorized into data-based and decoding-based. In our study, we investigate the ensemble of the two debiasing paradigms, proposing to use toxic corpus as an additional resource to reduce the toxicity. Our result shows that toxic corpus can indeed help to reduce the toxicity of the language generation process substantially, complementing the existing debiasing methods.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 17 of its 20 references that are in the map. First posted on arXiv 30 Apr 2022.

Builds on

17 of 20 references.

References · 17

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  3. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  4. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  5. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  6. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  7. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
  8. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  9. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
  10. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 296
  11. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  12. 2020Towards Debiasing Sentence Representationscited by 171
5 more
  1. 2021Challenges in Detoxifying Language Modelscited by 169
  2. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  3. 2019Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddingscited by 147
  4. 2017Affect-LM: A Neural Language Model for Customizable Affective Text Generationcited by 50
  5. 2020Mitigating Gender Bias for Neural Dialogue Generation with Adversarial Learningcited by 44

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 1; 2025: 0; 2026: 1.

5 papers.

  1. 2023Adversarial Fine-Tuning of Language Models: An Iterative Optimisation Approach for the Generation and Detection of Problematic Contentcited by 7
  2. 2023CMD: a framework for Context-aware Model self-Detoxificationcited by 6
  3. 2024FrenchToxicityPrompts: a Large Benchmark for Evaluating and Mitigating Toxicity in French Textscited by 5
  4. 2023An Introduction to Natural Language Processing Techniques and Framework for Clinical Implementation in Radiation Oncologycited by 2
  5. 2026Detoxification for LLM: From Dataset Itselfcited by 0