mlsanityData to 7 Oct 2026

COLD: A Benchmark for Chinese Offensive Language Detection

arXiv 2201.06025 ↗First posted 16 Jan 2022cs.CL cs.AIcited by 74

Not on the map. Why?

Abstract (arXiv)

Offensive language detection is increasingly crucial for maintaining a civilized social media platform and deploying pre-trained language models. However, this task in Chinese is still under exploration due to the scarcity of reliable datasets. To this end, we propose a benchmark --COLD for Chinese offensive language analysis, including a Chinese Offensive Language Dataset --COLDATASET and a baseline detector --COLDETECTOR which is trained on the dataset. We show that the COLD benchmark contributes to Chinese offensive language detection which is challenging for existing resources. We then deploy the COLDETECTOR and conduct detailed analyses on popular Chinese pre-trained language models. We first analyze the offensiveness of existing generative models and show that these models inevitably expose varying degrees of offensive issues. Furthermore, we investigate the factors that influence the offensive generations, and we find that anti-bias contents and keywords referring to certain groups or revealing negative attitudes trigger offensive outputs easier.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Builds on 31 of its 70 references that are in the map. First posted on arXiv 16 Jan 2022.

Builds on

31 of 70 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Language Models as Knowledge Bases?cited by 1,922
  3. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  4. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  5. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  6. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  7. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  8. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  9. 2020Recipes for building an open-domain chatbotcited by 593
  10. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  11. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  12. 2019Predicting the Type and Target of Offensive Posts in Social Mediacited by 268
19 more
  1. 2016Ex Machina: Personal Attacks Seen at Scalecited by 208
  2. 2021Societal Biases in Language Generation: Progress and Challengescited by 201
  3. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  4. 2020Recipes for Safety in Open-domain Chatbotscited by 168
  5. 2020Directions in Abusive Language Training Data: Garbage In, Garbage Outcited by 139
  6. 2017Understanding Abuse: A Typology of Abusive Language Detection Subtaskscited by 115
  7. 2019Does Gender Matter? Towards Fairness in Dialogue Systemscited by 89
  8. 2020A Large-Scale Chinese Short-Text Conversation Datasetcited by 77
  9. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
  10. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  11. 2020CPM: A Large-scale Generative Chinese Pre-trained Language Modelcited by 60
  12. 2021Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contextscited by 56
  13. 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
  14. 2021SWSR: A Chinese Dataset and Lexicon for Online Sexism Detectioncited by 42
  15. 2021EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Trainingcited by 34
  16. 2021A systematic review of Hate Speech automatic detection using Natural Language Processingcited by 26
  17. 2021Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biasescited by 21
  18. 2021Assessing Political Prudence of Open-domain Chatbotscited by 20
  19. 2018Machine Learning Suites for Online Toxicity Detectioncited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 18; 2024: 16; 2025: 23; 2026: 13.

74 papers.

Sort:

By area:

  1. 2023Large Language Model Alignment: A Surveycited by 241
  2. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 189
  3. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  4. 2023Self-Guard: Empower the LLM to Safeguard Itselfcited by 52
  5. 2024ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectorscited by 48
  6. 2023Facilitating Fine-grained Detection of Chinese Toxic Language: Hierarchical Taxonomy, Resources, and Benchmarkscited by 45
  7. 2023Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysiscited by 44
  8. 2022Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarkscited by 36
  9. 2023Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?cited by 36
  10. 2023Flames: Benchmarking Value Alignment of LLMs in Chinesecited by 35
  11. 2022EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Trainingcited by 31
  12. 2023Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuningcited by 28
38 more
  1. 2024MultiHateClip: A Multilingual Benchmark Dataset for Hateful Video Detection on YouTube and Bilibilicited by 25
  2. 2023KoSBi: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Applicationcited by 24
  3. 2023Enhancing Psychological Counseling with Large Language Model: A Multifaceted Decision-Support System for Non-Professionalscited by 24
  4. 2025SafeDialBench: A Fine-Grained Safety Evaluation Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attackscited by 23
  5. 2023CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Modelscited by 22
  6. 2023Cross-Cultural Transfer Learning for Chinese Offensive Language Detectioncited by 21
  7. 2023Cultural Compass: Predicting Transfer Learning Success in Offensive Language Detection with Cultural Featurescited by 18
  8. 2023ChatPLUG: Open-Domain Generative Dialogue System with Internet-Augmented Instruction Tuning for Digital Humancited by 14
  9. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 14
  10. 2023CLEVA: Chinese Language Models EVAluation Platformcited by 13
  11. 2024Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPTcited by 13
  12. 2024SGHateCheck: Functional Tests for Detecting Hate Speech in Low-Resource Languages of Singaporecited by 10
  13. 2025Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findingscited by 10
  14. 2025Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacementcited by 10
  15. 2022PanGu-Bot: Efficient Generative Dialogue Pre-training from Pre-trained Language Modelcited by 9
  16. 2024Collaborative decoding of critical tokens for boosting factuality of large language modelscited by 9
  17. 2024OpenEval: Benchmarking Chinese LLMs across Capability, Alignment and Safetycited by 9
  18. 2025STATE ToxiCN: A Benchmark for Span-level Target-Aware Toxicity Extraction in Chinese Hate Speech Detectioncited by 9
  19. 2022Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generationcited by 8
  20. 2023Improved Instruction Ordering in Recipe-Grounded Conversationcited by 7
  21. 2024DeMod: A Holistic Tool with Explainable Detection and Personalized Modification for Toxicity Censorshipcited by 6
  22. 2025SCCD: A Session-based Dataset for Chinese Cyberbullying Detectioncited by 6
  23. 2024Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspectivecited by 5
  24. 2024Edu-Values: Towards Evaluating the Chinese Education Values of Large Language Modelscited by 5
  25. 2025LLM-C3MOD: A Human-LLM Collaborative System for Cross-Cultural Hate Speech Moderationcited by 5
  26. 2025ToxicTone: A Mandarin Audio Dataset Annotated for Toxicity and Toxic Utterance Tonalitycited by 5
  27. 2025Unified Game Moderation: Soft-Prompting and LLM-Assisted Label Transfer for Resource-Efficient Toxicity Detectioncited by 5
  28. 2024Chinese Offensive Language Detection:Current Status and Future Directionscited by 4
  29. 2024Purple-teaming LLMs with Adversarial Defender Trainingcited by 4
  30. 2023The Skipped Beat: A Study of Sociopragmatic Understanding in LLMs for 64 Languagescited by 3
  31. 2025Unified Multi-Task Learning & Model Fusion for Efficient Language Model Guardrailingcited by 3
  32. 2026Adversarial Humanities Benchmark: Results on Stylistic Robustness in Frontier Model Safetycited by 3
  33. 2024Cross-lingual Offensive Language Detection: A Systematic Review of Datasets, Transfer Approaches and Challengescited by 2
  34. 2024Social Orientation: A New Feature for Dialogue Analysiscited by 2
  35. 2024ChineseWebText 2.0: Large-Scale High-quality Chinese Web Text with Multi-dimensional and fine-grained informationcited by 2
  36. 2025PANDA -- Paired Anti-hate Narratives Dataset from Asia: Using an LLM-as-a-Judge to Create the First Chinese Counterspeech Datasetcited by 2
  37. 2025Chinese Toxic Language Mitigation via Sentiment Polarity Consistent Rewritescited by 2
  38. 2025Fine-Grained Chinese Hate Speech Understanding: Span-Level Resources, Coded Term Lexicon, and Enhanced Detection Frameworkscited by 2

50 most cited of 74 All 74 (JSON)