mlsanityData to 14 Oct 2026

Text Detoxification using Large Pre-trained Neural Models

arXiv 2109.08914 ↗First posted 18 Sep 2021cs.CL cs.LGcited by 52

Not on the map. Why?

Abstract (arXiv)

We present two novel unsupervised methods for eliminating toxicity in text. Our first method combines two recent ideas: (1) guidance of the generation process with small style-conditional language models and (2) use of paraphrasing models to perform style transfer. We use a well-performing paraphraser guided by style-trained language models to keep the text content and remove toxicity. Our second method uses BERT to replace toxic words with their non-offensive synonyms. We make the method more flexible by enabling BERT to replace mask tokens with a variable number of words. Finally, we present the first large-scale comparative study of style transfer models on the task of toxicity removal. We compare our models with a number of methods for style transfer. The models are evaluated in a reference-free way using a combination of unsupervised style transfer metrics. Both methods we suggest yield new SOTA results.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Builds on 31 of its 48 references that are in the map. First posted on arXiv 18 Sep 2021.

Builds on

31 of 48 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  5. 2018Neural Network Acceptability Judgmentscited by 958
  6. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  7. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
  8. 2017Toward Controlled Generation of Textcited by 272
  9. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 248
  10. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 211
  11. 2018Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfercited by 191
  12. 2020Reformulating Unsupervised Style Transfer as Paraphrase Generationcited by 183
19 more
  1. 2020SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)cited by 138
  2. 2018Disentangled Representation Learning for Non-Parallel Text Style Transfercited by 127
  3. 2018Style Transfer Through Back-Translationcited by 124
  4. 2018Conditional BERT Contextual Augmentationcited by 107
  5. 2019Beyond BLEU: Training Neural Machine Translation with Semantic Similaritycited by 84
  6. 2019Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfercited by 81
  7. 2020Politeness Transfer: A Tag and Generate Approachcited by 78
  8. 2020A Probabilistic Formulation of Unsupervised Text Style Transfercited by 70
  9. 2019A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfercited by 65
  10. 2018Fighting Offensive Language on Social Media with Unsupervised Text Style Transfercited by 55
  11. 2019"Mask and Infill" : Applying Masked Language Model to Sentiment Transfercited by 48
  12. 2017ParaNMT-50M: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translationscited by 44
  13. 2021Civil Rephrases Of Toxic Texts With Self-Supervised Transformerscited by 32
  14. 2020Style-transfer and Paraphrase: Looking for a Sensible Semantic Similarity Metriccited by 23
  15. 2020Stable Style Transformer: Delete and Generate Approach with Encoder-Decoder for Text Style Transfercited by 22
  16. 2020Don't Patronize Me! An Annotated Dataset with Patronizing and Condescending Language towards Vulnerable Communitiescited by 19
  17. 2020Towards A Friendly Online Community: An Unsupervised Style Transfer Framework for Profanity Redactioncited by 17
  18. 2019Towards non-toxic landscapes: Automatic toxic comment detection using DNNcited by 8
  19. 2021Detecting Inappropriate Messages on Sensitive Topics that Could Harm a Company's Reputationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 7; 2023: 12; 2024: 18; 2025: 13; 2026: 2.

52 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  2. 2023You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Contentcited by 39
  3. 2022Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Expertscited by 36
  4. 2022Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimizationcited by 33
  5. 2023ToViLaG: Your Visual-Language Generative Model is Also An Evildoercited by 27
  6. 2023Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Modelscited by 23
  7. 2024MERA: A Comprehensive LLM Evaluation in Russiancited by 22
  8. 2024Token-wise Influential Training Data Retrieval for Large Language Modelscited by 20
  9. 2023ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfercited by 18
  10. 2024From One to Many: Expanding the Scope of Toxicity Mitigation in Language Modelscited by 18
  11. 2024TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddingscited by 17
  12. 2024Walking in Others' Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Biascited by 17
38 more
  1. 2023Exploring Methods for Cross-lingual Text Style Transfer: The Case of Text Detoxificationcited by 15
  2. 2023Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvementscited by 14
  3. 2024MultiParaDetox: Extending Text Detoxification with Parallel Data to New Languagescited by 14
  4. 2024Distilling Text Style Transfer With Self-Explanation From LLMscited by 12
  5. 2024Multilingual and Explainable Text Detoxification with Parallel Corporacited by 12
  6. 2022Language Detoxification with Attribute-Discriminative Latent Spacecited by 11
  7. 2024Text Detoxification as Style Transfer in English and Hindicited by 11
  8. 2023Target-Agnostic Gender-Aware Contrastive Learning for Mitigating Bias in Multilingual Machine Translationcited by 10
  9. 2023Learn What NOT to Learn: Towards Generative Safety in Chatbotscited by 8
  10. 2025IF-GUIDE: Influence Function-Guided Detoxification of LLMscited by 7
  11. 2023Don't Take This Out of Context! On the Need for Contextual Models and Evaluations for Stylistic Rewritingcited by 6
  12. 2023DiffuDetox: A Mixed Diffusion Model for Text Detoxificationcited by 6
  13. 2023CMD: a framework for Context-aware Model self-Detoxificationcited by 6
  14. 2024MisgenderMender: A Community-Informed Approach to Interventions for Misgenderingcited by 6
  15. 2025Generative Data Refinement: Just Ask for Better Datacited by 6
  16. 2024FrenchToxicityPrompts: a Large Benchmark for Evaluating and Mitigating Toxicity in French Textscited by 5
  17. 2022The State of Profanity Obfuscation in Natural Language Processingcited by 4
  18. 2024Style Transfer with Multi-iteration Preference Optimizationcited by 4
  19. 2025SynthDetoxM: Modern LLMs are Few-Shot Parallel Detoxification Data Annotatorscited by 4
  20. 2025LLM in the Loop: Creating the ParaDeHate Dataset for Hate Speech Detoxificationcited by 4
  21. 2025Evaluating Text Style Transfer: A Nine-Language Benchmark for Text Detoxificationcited by 4
  22. 2022Exploring Cross-lingual Textual Style Transfer with Large Multilingual Language Modelscited by 3
  23. 2024Cross-cultural Inspiration Detection and Analysis in Real and LLM-generated Social Media Datacited by 3
  24. 2024Words Matter: Reducing Stigma in Online Conversations about Substance Use with Large Language Modelscited by 3
  25. 2024Learning from Response not Preference: A Stackelberg Approach for LLM Detoxification using Non-parallel Datacited by 3
  26. 2024GPT-DETOX: An In-Context Learning-Based Paraphraser for Text Detoxificationcited by 2
  27. 2024Mitigating Text Toxicity with Counterfactual Generationcited by 2
  28. 2024LaSEr-Edit: Localized Span-level Error Editing with Energy-based Localizationcited by 2
  29. 2025K/DA: Automated Data Generation Pipeline for Detoxifying Implicitly Offensive Language in Koreancited by 2
  30. 2025BALI: Enhancing Biomedical Language Representations through Knowledge Graph and Language Model Alignmentcited by 2
  31. 2022Rethinking Offensive Text Detection as a Multi-Hop Reasoning Problemcited by 1
  32. 2022Russian Texts Detoxification with Levenshtein Editingcited by 1
  33. 2025Detoxify: A framework for abusive text transformation using LLMscited by 0
  34. 2025ylmmcl at Multilingual Text Detoxification 2025: Lexicon-Guided Detoxification and Classifier-Gated Rewritingcited by 0
  35. 2025What's Taboo for You? - An Empirical Evaluation of LLMs Behavior Toward Sensitive Contentcited by 0
  36. 2025GemDetox at TextDetox CLEF 2025: Enhancing a Massively Multilingual Model for Text Detoxification on Low-resource Languagescited by 0
  37. 2025Beyond One-Size-Fits-All: Personalized Harmful Content Detection with In-Context Learningcited by 0
  38. 2025HatePrototypes: Interpretable and Transferable Representations for Implicit and Explicit Hate Speech Detectioncited by 0

50 most cited of 52 All 52 (JSON)