mlsanityData to 7 Oct 2026

Civil Rephrases Of Toxic Texts With Self-Supervised Transformers

arXiv 2102.05456 ↗First posted 1 Feb 2021cs.CL cs.AI cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Platforms that support online commentary, from social networks to news sites, are increasingly leveraging machine learning to assist their moderation efforts. But this process does not typically provide feedback to the author that would help them contribute according to the community guidelines. This is prohibitively time-consuming for human moderators to do, and computational approaches are still nascent. This work focuses on models that can help suggest rephrasings of toxic comments in a more civil manner. Inspired by recent progress in unpaired sequence-to-sequence tasks, a self-supervised learning model is introduced, called CAE-T5. CAE-T5 employs a pre-trained text-to-text transformer, which is fine tuned with a denoising and cyclic auto-encoder loss. Experimenting with the largest toxicity detection dataset to date (Civil Comments) our model generates sentences that are more fluent and better at preserving the initial content compared to earlier text style transfer systems which we compare with using several scoring systems and human evaluation.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 61 of its 86 references that are in the map. First posted on arXiv 1 Feb 2021.

Builds on

61 of 86 references.

References · 61

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  9. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  10. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  11. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  12. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
38 more
  1. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  2. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  3. 2018Universal Sentence Encodercited by 485
  4. 2017Word Translation Without Parallel Datacited by 480
  5. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  6. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  7. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  8. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  9. 2017Toward Controlled Generation of Textcited by 271
  10. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 246
  11. 2016Understanding Neural Networks through Representation Erasurecited by 223
  12. 2018Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfercited by 209
  13. 2016Ex Machina: Personal Attacks Seen at Scalecited by 208
  14. 2019SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)cited by 206
  15. 2018Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfercited by 191
  16. 2019FlauBERT: Unsupervised Language Model Pre-training for Frenchcited by 153
  17. 2017Unsupervised Neural Machine Translationcited by 148
  18. 2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 147
  19. 2016Dual Learning for Machine Translationcited by 146
  20. 2017Style Transfer in Text: Exploration and Evaluationcited by 145
  21. 2018Phrase-Based & Neural Unsupervised Machine Translationcited by 138
  22. 2018Disentangled Representation Learning for Non-Parallel Text Style Transfercited by 127
  23. 2018Style Transfer Through Back-Translationcited by 124
  24. 2019Automatically Neutralizing Subjective Bias in Textcited by 105
  25. 2019FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flowcited by 89
  26. 2019Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representationcited by 84
  27. 2018Conversations Gone Awry: Detecting Early Signs of Conversational Failurecited by 73
  28. 2018Unsupervised Text Style Transfer using Language Models as Discriminatorscited by 70
  29. 2020A Probabilistic Formulation of Unsupervised Text Style Transfercited by 70
  30. 2019A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfercited by 65
  31. 2018Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approachcited by 55
  32. 2018Fighting Offensive Language on Social Media with Unsupervised Text Style Transfercited by 55
  33. 2016Semantic Compositional Networks for Visual Captioningcited by 50
  34. 2019"Mask and Infill" : Applying Masked Language Model to Sentiment Transfercited by 48
  35. 2018Content preserving text generation with attribute controlscited by 47
  36. 2019Reinforcement Learning Based Text Style Transfer without Parallel Training Corpuscited by 38
  37. 2017A Deep Reinforcement Learning Chatbotcited by 36
  38. 2019Controllable Unsupervised Text Attribute Transfer via Editing Entangled Latent Representationcited by 34

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 9; 2024: 8; 2025: 3; 2026: 3.

32 papers.

Sort:

By area:

  1. 2021Challenges in Detoxifying Language Modelscited by 169
  2. 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Modelscited by 124
  3. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  4. 2021Text Detoxification using Large Pre-trained Neural Modelscited by 52
  5. 2023You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Contentcited by 37
  6. 2022Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Expertscited by 36
  7. 2022Listening to Affected Communities to Define Extreme Speech: Dataset and Experimentscited by 21
  8. 2024NLP for Counterspeech against Hate: A Survey and How-To Guidecited by 21
  9. 2023On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlookcited by 15
  10. 2023Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvementscited by 14
  11. 2023A Call for Standardization and Validation of Text Style Transfer Evaluationcited by 13
  12. 2023Can Language Model Moderators Improve the Health of Online Discourse?cited by 12
20 more
  1. 2024DetoxLLM: A Framework for Detoxification with Explanationscited by 10
  2. 2022ELF22: A Context-based Counter Trolling Dataset to Combat Internet Trollscited by 7
  3. 2024LLM-based Rewriting of Inappropriate Argumentation using Reinforcement Learning from Machine Feedbackcited by 7
  4. 2023DiffuDetox: A Mixed Diffusion Model for Text Detoxificationcited by 6
  5. 2025Generative Data Refinement: Just Ask for Better Datacited by 6
  6. 2022The State of Profanity Obfuscation in Natural Language Processingcited by 4
  7. 2024Northeastern Uni at Multilingual Counterspeech Generation: Enhancing Counter Speech Generation with LLM Alignment through Direct Preference Optimizationcited by 4
  8. 2025LLM in the Loop: Creating the ParaDeHate Dataset for Hate Speech Detoxificationcited by 4
  9. 2022Exploring Cross-lingual Textual Style Transfer with Large Multilingual Language Modelscited by 3
  10. 2024Learning from Response not Preference: A Stackelberg Approach for LLM Detoxification using Non-parallel Datacited by 3
  11. 2021Simple Text Detoxification by Identifying a Linear Toxic Subspace in Language Model Embeddingscited by 2
  12. 2022Exploiting Social Media Content for Self-Supervised Style Transfercited by 2
  13. 2024Mitigating Text Toxicity with Counterfactual Generationcited by 2
  14. 2023Sociocultural knowledge is needed for selection of shots in hate speech detection taskscited by 1
  15. 2025BanglaNirTox: A Large-scale Parallel Corpus for Explainable AI in Bengali Text Detoxificationcited by 1
  16. 2026Cleansing the Artificial Mind: A Self-Reflective Detoxification Framework for Large Language Modelscited by 1
  17. 2023Entity Identifier: A Natural Text Parsing-based Framework For Entity Relation Extractioncited by 0
  18. 2023Trustworthy Large Models in Vision: A Surveycited by 0
  19. 2026Teaching LLMs Human-Like Editing of Inappropriate Argumentation via Reinforcement Learningcited by 0
  20. 2026R^3: Advertisement Compliance Rectification via Group-Relative Experience Extractor and Curriculum Reinforcementcited by 0