mlsanityData to 7 Oct 2026

ToxiCraft: A Novel Framework for Synthetic Generation of Harmful Information

arXiv 2409.14740 ↗First posted 23 Sep 2024cs.CL cs.AIcited by 17

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

In different NLP tasks, detecting harmful content is crucial for online environments, especially with the growing influence of social media. However, previous research has two main issues: 1) a lack of data in low-resource settings, and 2) inconsistent definitions and criteria for judging harmful content, requiring classification models to be robust to spurious features and diverse. We propose Toxicraft, a novel framework for synthesizing datasets of harmful information to address these weaknesses. With only a small amount of seed data, our framework can generate a wide variety of synthetic, yet remarkably realistic, examples of toxic information. Experimentation across various datasets showcases a notable enhancement in detection model robustness and adaptability, surpassing or close to the gold labels.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 32 of its 55 references that are in the map. First posted on arXiv 23 Sep 2024.

Builds on

32 of 55 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  9. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  10. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  11. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
  12. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
20 more
  1. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  2. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  3. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  4. 2020Toxicity Detection: Does Context Really Matter?cited by 93
  5. 2023Reasoning Implicit Sentiment with Chain-of-Thought Promptingcited by 79
  6. 2023Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Methodcited by 76
  7. 2022ProGen: Progressive Zero-shot Dataset Generation via In-context Feedbackcited by 67
  8. 2022PromDA: Prompt-based Data Augmentation for Low-Resource NLU Taskscited by 51
  9. 2020Deep Learning Models for Multilingual Hate Speech Detectioncited by 49
  10. 2020Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generationcited by 47
  11. 2021Hate Towards the Political Opponent: A Twitter Corpus Study of the 2020 US Elections on the Basis of Offensive Speech and Stance Detectioncited by 45
  12. 2021Overview of the HASOC track at FIRE 2020: Hate Speech and Offensive Content Identification in Indo-European Languagescited by 38
  13. 2023Robust Hate Speech Detection in Social Media: A Cross-Dataset Empirical Evaluationcited by 28
  14. 2020Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from Instructionscited by 25
  15. 2023People Make Better Edits: Measuring the Efficacy of LLM-Generated Counterfactually Augmented Data for Harmful Language Detectioncited by 25
  16. 2023Mixture of Soft Prompts for Controllable Data Generationcited by 23
  17. 2021Fight Fire with Fire: Fine-tuning Hate Detectors using Large Samples of Generated Hate Speechcited by 13
  18. 2018Predictive Embeddings for Hate Speech Detection on Twittercited by 12
  19. 2020A little goes a long way: Improving toxic language classification despite data scarcitycited by 12
  20. 2018Examining a hate speech corpus for hate speech detection and popularity predictioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 9; 2026: 6.

17 papers.

Sort:

By area:

  1. 2026Value of Information: A Framework for Human-Agent Communicationcited by 10
  2. 2025When Personalization Meets Reality: A Multi-Faceted Analysis of Personalized Preference Learningcited by 9
  3. 2025Toward Safe and Human-Aligned Game Conversational Recommendation via Multi-Agent Decompositioncited by 9
  4. 2025A Survey on Efficient Large Language Model Training: From Data-centric Perspectivescited by 9
  5. 2025TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Lawcited by 7
  6. 2025Privacy-R1: Privacy-Aware Multi-LLM Agent Collaboration via Reinforcement Learningcited by 6
  7. 2024ToxiLab: How Well Do Open-Source LLMs Generate Synthetic Toxicity Data?cited by 5
  8. 2025A Review of Incorporating Psychological Theories in LLMscited by 5
  9. 2025Synthetic vs. Gold: The Role of LLM Generated Labels and Data in Cyberbullying Detectioncited by 4
  10. 2025ToxSyn-PT: A Synthetic Fine-Grained Dataset of Minority-Targeted Toxic Language in Portuguesecited by 2
  11. 2025SynBullying: A Multi LLM Synthetic Conversational Dataset for Cyberbullying Detectioncited by 1
  12. 2026ToxiGAN: Toxic Data Augmentation via LLM-Guided Directional Adversarial Generationcited by 1
5 more
  1. 2026Steer Model beyond Assistant: Controlling System Prompt Strength via Contrastive Decodingcited by 1
  2. 2024From Measurement Instruments to Data: Leveraging Theory-Driven Synthetic Training Data for Classifying Social Constructscited by 0
  3. 2026Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluationcited by 0
  4. 2026Lower-Resource, Higher Scores: Language Bias in LLM Evaluatorscited by 0
  5. 2026Do Social Patterns Hold in Synthetic Data? Analyzing Cyberbullying Dynamics in LLM-Generated and Authentic Dialoguescited by 0