# CrowdCounter: A benchmark type-specific multi-target counterspeech dataset

arXiv: https://arxiv.org/abs/2410.01400 · HTML: https://mlsanity.com/paper/2410.01400/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://mlsanity.com/paper/1909.02622/) (2019)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [A Benchmark Dataset for Learning to Intervene in Online Hate Speech](https://mlsanity.com/paper/1909.04251/) (2019)
- [CONAN -- COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech](https://mlsanity.com/paper/1910.03270/) (2019)
- [Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech](https://mlsanity.com/paper/2107.08720/) (2021)
- [Thou shalt not hate: Countering Online Hate Speech](https://mlsanity.com/paper/1808.04409/) (2018)
- [Cross-topic Argument Mining from Heterogeneous Sources Using Attention-based Neural Networks](https://mlsanity.com/paper/1802.05758/) (2018)
- [Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech](https://mlsanity.com/paper/2106.01625/) (2021)
- [GRUEN for Evaluating Linguistic Quality of Generated Text](https://mlsanity.com/paper/2010.02498/) (2020)
- [Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints](https://mlsanity.com/paper/1809.01215/) (2018)
- [Knowledge-Grounded Dialogue Generation with a Unified Knowledge Representation](https://mlsanity.com/paper/2112.07924/) (2021)
- [CounterGeDi: A controllable approach to generate polite, detoxified and emotional counterspeech](https://mlsanity.com/paper/2205.04304/) (2022)
- [On Zero-Shot Counterspeech Generation by LLMs](https://mlsanity.com/paper/2403.14938/) (2024)
- [Spillover of Antisocial Behavior from Fringe Platforms: The Unintended Consequences of Community Banning](https://mlsanity.com/paper/2209.09803/) (2022)
- [Empowering NGOs in Countering Online Hate Messages](https://mlsanity.com/paper/2107.02472/) (2021)
- [Counterspeeches up my sleeve! Intent Distribution Learning and Persistent Fusion for Intent-Conditioned Counterspeech Generation](https://mlsanity.com/paper/2305.13776/) (2023)
- [Multilingual Counter Narrative Type Classification](https://mlsanity.com/paper/2109.13664/) (2021)

## Built on by (most cited first, 5 of 5)

- [Hope vs. Hate: Understanding User Interactions with LGBTQ+ News Content in Mainstream US News Media through the Lens of Hope Speech](https://mlsanity.com/paper/2502.09004/) (2025)
- [Can NLP Tackle Hate Speech in the Real World? Stakeholder-Informed Feedback and Survey on Counterspeech](https://mlsanity.com/paper/2508.04638/) (2025)
- [Counterspeech for Mitigating the Influence of Media Bias: Comparing Human and LLM-Generated Responses](https://mlsanity.com/paper/2508.15855/) (2025)
- [Counter with Evidence! A Multi-Agent Memory Efficient Reasoning Framework for Hate Category Informed Counterspeech Generation](https://mlsanity.com/paper/2608.23152/) (2026)
- [Deconstructing Stereotypes: Scope-Conditioned Generation for Effective Multilingual Counterspeech](https://mlsanity.com/paper/2609.16906/) (2026)

## Abstract (arXiv)

Counterspeech presents a viable alternative to banning or suspending users for hate speech while upholding freedom of expression. However, writing effective counterspeech is challenging for moderators/users. Hence, developing suggestion tools for writing counterspeech is the need of the hour. One critical challenge in developing such a tool is the lack of quality and diversity of the responses in the existing datasets. Hence, we introduce a new dataset - CrowdCounter containing 3,425 hate speech-counterspeech pairs spanning six different counterspeech types (empathy, humor, questioning, warning, shaming, contradiction), which is the first of its kind. The design of our annotation platform itself encourages annotators to write type-specific, non-redundant and high-quality counterspeech. We evaluate two frameworks for generating counterspeech responses - vanilla and type-controlled prompts - across four large language models. In terms of metrics, we evaluate the responses using relevance, diversity and quality. We observe that Flan-T5 is the best model in the vanilla framework across different models. Type-specific prompts enhance the relevance of the responses, although they might reduce the language quality. DialoGPT proves to be the best at following the instructions and generating the type-specific counterspeech accurately.