# Basque and Spanish Counter Narrative Generation: Data Creation and Evaluation

arXiv: https://arxiv.org/abs/2403.09159 · HTML: https://mlsanity.com/paper/2403.09159/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 40 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Builds on (most cited first, 15 of 15)

- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [A Benchmark Dataset for Learning to Intervene in Online Hate Speech](https://mlsanity.com/paper/1909.04251/) (2019)
- [CONAN -- COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech](https://mlsanity.com/paper/1910.03270/) (2019)
- [Translation Artifacts in Cross-lingual Transfer Learning](https://mlsanity.com/paper/2004.04721/) (2020)
- [Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech](https://mlsanity.com/paper/2107.08720/) (2021)
- [Thou shalt not hate: Countering Online Hate Speech](https://mlsanity.com/paper/1808.04409/) (2018)
- [Generating Counter Narratives against Online Hate Speech: Data and Strategies](https://mlsanity.com/paper/2004.04216/) (2020)
- [Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech](https://mlsanity.com/paper/2106.01625/) (2021)
- [Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech](https://mlsanity.com/paper/2106.11783/) (2021)
- [Using Pre-Trained Language Models for Producing Counter Narratives Against Hate Speech: a Comparative Study](https://mlsanity.com/paper/2204.01440/) (2022)
- [Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings](https://mlsanity.com/paper/2210.12623/) (2022)
- [T-Projection: High Quality Annotation Projection for Sequence Labeling Tasks](https://mlsanity.com/paper/2212.10548/) (2022)
- [Multilingual Counter Narrative Type Classification](https://mlsanity.com/paper/2109.13664/) (2021)

## Built on by (most cited first, 7 of 7)

- [Is Safer Better? The Impact of Guardrails on the Argumentative Strength of LLMs in Hate Speech Countering](https://mlsanity.com/paper/2410.03466/) (2024)
- [A LLM-Based Ranking Method for the Evaluation of Automatic Counter-Narrative Generation](https://mlsanity.com/paper/2406.15227/) (2024)
- [The #Somos600M Project: Generating NLP resources that represent the diversity of the languages from LATAM, the Caribbean, and Spain](https://mlsanity.com/paper/2407.17479/) (2024)
- [La Leaderboard: A Large Language Model Leaderboard for Spanish Varieties and Languages of Spain and Latin America](https://mlsanity.com/paper/2507.00999/) (2025)
- [Beating Harmful Stereotypes Through Facts: RAG-based Counter-speech Generation](https://mlsanity.com/paper/2510.12316/) (2025)
- [Multilingual Hate Speech Detection and Counterspeech Generation: A Comprehensive Survey and Practical Guide](https://mlsanity.com/paper/2603.19279/) (2026)
- [Can NLP Tackle Hate Speech in the Real World? Stakeholder-Informed Feedback and Survey on Counterspeech](https://mlsanity.com/paper/2508.04638/) (2025)

## Abstract (arXiv)

Counter Narratives (CNs) are non-negative textual responses to Hate Speech (HS) aiming at defusing online hatred and mitigating its spreading across media. Despite the recent increase in HS content posted online, research on automatic CN generation has been relatively scarce and predominantly focused on English. In this paper, we present CONAN-EUS, a new Basque and Spanish dataset for CN generation developed by means of Machine Translation (MT) and professional post-edition. Being a parallel corpus, also with respect to the original English CONAN, it allows to perform novel research on multilingual and crosslingual automatic generation of CNs. Our experiments on CN generation with mT5, a multilingual encoder-decoder model, show that generation greatly benefits from training on post-edited data, as opposed to relying on silver MT data only. These results are confirmed by their correlation with a qualitative manual evaluation, demonstrating that manually revised training data remains crucial for the quality of the generated CNs. Furthermore, multilingual data augmentation improves results over monolingual settings for structurally similar languages such as English and Spanish, while being detrimental for Basque, a language isolate. Similar findings occur in zero-shot crosslingual evaluations, where model transfer (fine-tuning in English and generating in a different target language) outperforms fine-tuning mT5 on machine translated data for Spanish but not for Basque. This provides an interesting insight into the asymmetry in the multilinguality of generative models, a challenging topic which is still open to research.