# Northeastern Uni at Multilingual Counterspeech Generation: Enhancing Counter Speech Generation with LLM Alignment through Direct Preference Optimization

arXiv: https://arxiv.org/abs/2412.15453 · HTML: https://mlsanity.com/paper/2412.15453/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 19 Dec 2024.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 18 of 18)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings](https://mlsanity.com/paper/1704.07130/) (2017)
- [Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective](https://mlsanity.com/paper/2012.12305/) (2020)
- [Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech](https://mlsanity.com/paper/2106.11783/) (2021)
- [Civil Rephrases Of Toxic Texts With Self-Supervised Transformers](https://mlsanity.com/paper/2102.05456/) (2021)
- [Beyond Denouncing Hate: Strategies for Countering Implied Biases and Stereotypes in Language](https://mlsanity.com/paper/2311.00161/) (2023)
- [NLP for Counterspeech against Hate: A Survey and How-To Guide](https://mlsanity.com/paper/2403.20103/) (2024)
- [On Zero-Shot Counterspeech Generation by LLMs](https://mlsanity.com/paper/2403.14938/) (2024)
- [Hope Speech Detection: A Computational Analysis of the Voice of Peace](https://mlsanity.com/paper/1909.12940/) (2019)
- [Intent-conditioned and Non-toxic Counterspeech Generation using Multi-Task Instruction Tuning with RLAIF](https://mlsanity.com/paper/2403.10088/) (2024)
- [Outcome-Constrained Large Language Models for Countering Hate Speech](https://mlsanity.com/paper/2403.17146/) (2024)

## Built on by (most cited first, 4 of 4)

- [Structuring the Space of Sociotechnical Alignment](https://mlsanity.com/paper/2607.01250/) (2026)
- [Can NLP Tackle Hate Speech in the Real World? Stakeholder-Informed Feedback and Survey on Counterspeech](https://mlsanity.com/paper/2508.04638/) (2025)
- [ART: Adaptive Reasoning Trees for Explainable Claim Verification](https://mlsanity.com/paper/2601.05455/) (2026)
- [A Multi-Stage Agentic Framework for Effective Counter-Narrative Generation and Refinement](https://mlsanity.com/paper/2609.14178/) (2026)

## Abstract (arXiv)

The automatic generation of counter-speech (CS) is a critical strategy for addressing hate speech by providing constructive and informed responses. However, existing methods often fail to generate high-quality, impactful, and scalable CS, particularly across diverse linguistic contexts. In this paper, we propose a novel methodology to enhance CS generation by aligning Large Language Models (LLMs) using Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). Our approach leverages DPO to align LLM outputs with human preferences, ensuring contextually appropriate and linguistically adaptable responses. Additionally, we incorporate knowledge grounding to enhance the factual accuracy and relevance of generated CS. Experimental results demonstrate that DPO-aligned models significantly outperform SFT baselines on CS benchmarks while scaling effectively to multiple languages. These findings highlight the potential of preference-based alignment techniques to advance CS generation across varied linguistic settings. The model supervision and alignment is done in English and the same model is used for reporting metrics across other languages like Basque, Italian, and Spanish.