# Reinforcement Learning-based Counter-Misinformation Response Generation: A Case Study of COVID-19 Vaccine Misinformation

arXiv: https://arxiv.org/abs/2303.06433 · HTML: https://mlsanity.com/paper/2303.06433/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 129 references that are in the map.
- First posted on arXiv 11 Mar 2023.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [Deep Reinforcement Learning for Dialogue Generation](https://mlsanity.com/paper/1606.01541/) (2016)
- [A Benchmark Dataset for Learning to Intervene in Online Hate Speech](https://mlsanity.com/paper/1909.04251/) (2019)
- [A Computational Approach to Politeness with Application to Social Factors](https://mlsanity.com/paper/1306.6078/) (2013)
- [COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic](https://mlsanity.com/paper/2106.03794/) (2021)
- [Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation](https://mlsanity.com/paper/1905.05621/) (2019)
- [Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach](https://mlsanity.com/paper/2101.07714/) (2021)
- [Community Interaction and Conflict on the Web](https://mlsanity.com/paper/1803.03697/) (2018)
- [Generating Counter Narratives against Online Hate Speech: Data and Strategies](https://mlsanity.com/paper/2004.04216/) (2020)
- [PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction](https://mlsanity.com/paper/2010.13816/) (2020)
- [Argument Generation with Retrieval, Planning, and Realization](https://mlsanity.com/paper/1906.03717/) (2019)
- [Aspect-Controlled Neural Argument Generation](https://mlsanity.com/paper/2005.00084/) (2020)
- [Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech](https://mlsanity.com/paper/2106.01625/) (2021)
- [Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech](https://mlsanity.com/paper/2106.11783/) (2021)
- [Racism is a Virus: Anti-Asian Hate and Counterspeech in Social Media during the COVID-19 Crisis](https://mlsanity.com/paper/2005.12423/) (2020)
- [False Information on Web and Social Media: A Survey](https://mlsanity.com/paper/1804.08559/) (2018)
- [Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions](https://mlsanity.com/paper/1702.01119/) (2017)
- [Better Conversations by Modeling,Filtering,and Optimizing for Coherence and Diversity](https://mlsanity.com/paper/1809.06873/) (2018)
- [The Rise of Guardians: Fact-checking URL Recommendation to Combat Fake News](https://mlsanity.com/paper/1806.07516/) (2018)
- [The Role of the Crowd in Countering Misinformation: A Case Study of the COVID-19 Infodemic](https://mlsanity.com/paper/2011.05773/) (2020)
- [Argument Undermining: Counter-Argument Generation by Attacking Weak Premises](https://mlsanity.com/paper/2105.11752/) (2021)
- [PETGEN: Personalized Text Generation Attack on Deep Sequence Embedding-based Classification Models](https://mlsanity.com/paper/2109.06777/) (2021)
- [Learning from Fact-checkers: Analysis and Generation of Fact-checking Language](https://mlsanity.com/paper/1910.02202/) (2019)
- [GraphNLI: A Graph-based Natural Language Inference Model for Polarity Prediction in Online Debates](https://mlsanity.com/paper/2202.08175/) (2022)

## Built on by (most cited first, 30 of 39)

- [DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detection](https://mlsanity.com/paper/2402.10426/) (2024)
- [Evidence-Driven Retrieval Augmented Response Generation for Online Misinformation](https://mlsanity.com/paper/2403.14952/) (2024)
- [Can Knowledge Editing Really Correct Hallucinations?](https://mlsanity.com/paper/2410.16251/) (2024)
- [FACT-GPT: Fact-Checking Augmentation via Claim Matching with LLMs](https://mlsanity.com/paper/2402.05904/) (2024)
- [Benchmarking the Generation of Fact Checking Explanations](https://mlsanity.com/paper/2308.15202/) (2023)
- [Retrieval Augmented Fact Verification by Synthesizing Contrastive Arguments](https://mlsanity.com/paper/2406.09815/) (2024)
- [Fact-checking information from large language models can decrease headline discernment](https://mlsanity.com/paper/2308.10800/) (2023)
- [NLP for Counterspeech against Hate: A Survey and How-To Guide](https://mlsanity.com/paper/2403.20103/) (2024)
- [NLP Systems That Can't Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helps](https://mlsanity.com/paper/2404.01651/) (2024)
- [MemeGuard: An LLM and VLM-based Framework for Advancing Content Moderation via Meme Intervention](https://mlsanity.com/paper/2406.05344/) (2024)
- [Automated Claim Matching with Large Language Models: Empowering Fact-Checkers in the Fight Against Misinformation](https://mlsanity.com/paper/2310.09223/) (2023)
- [Contextualized Counterspeech: Strategies for Adaptation, Personalization, and Evaluation](https://mlsanity.com/paper/2412.07338/) (2024)
- [Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Models](https://mlsanity.com/paper/2505.18596/) (2025)
- [EXPLAIN, EDIT, GENERATE: Rationale-Sensitive Counterfactual Data Augmentation for Multi-hop Fact Verification](https://mlsanity.com/paper/2310.14508/) (2023)
- [Countering Misinformation via Emotional Response Generation](https://mlsanity.com/paper/2311.10587/) (2023)
- [Customized large language models can outperform Community Notes in correcting misinformation](https://mlsanity.com/paper/2403.11169/) (2024)
- [MetaAdapt: Domain Adaptive Few-Shot Misinformation Detection via Meta Learning](https://mlsanity.com/paper/2305.12692/) (2023)
- [Global-Liar: Factuality of LLMs over Time and Geographic Regions](https://mlsanity.com/paper/2401.17839/) (2024)
- [COMPKE: Complex Question Answering under Knowledge Editing](https://mlsanity.com/paper/2506.00829/) (2025)
- [Harnessing Network Effect for Fake News Mitigation: Selecting Debunkers via Self-Imitation Learning](https://mlsanity.com/paper/2402.03357/) (2024)
- [On the Risk of Evidence Pollution for Malicious Social Text Detection in the Era of LLMs](https://mlsanity.com/paper/2410.12600/) (2024)
- [Face the Facts! Evaluating RAG-based Pipelines for Professional Fact-Checking](https://mlsanity.com/paper/2412.15189/) (2024)
- [Collaboration and Controversy Among Experts: Rumor Early Detection by Tuning a Comment Generator](https://mlsanity.com/paper/2504.04076/) (2025)
- [MemeSense: An Adaptive In-Context Framework for Social Commonsense Driven Meme Moderation](https://mlsanity.com/paper/2502.11246/) (2025)
- [Multi-Agent Retrieval-Augmented Framework for Evidence-Based Counterspeech Against Health Misinformation](https://mlsanity.com/paper/2507.07307/) (2025)
- [Speaking at the Right Level: Literacy-Controlled Counterspeech Generation with RAG-RL](https://mlsanity.com/paper/2509.01058/) (2025)
- [TrendFact: A Benchmark Towards Hotspot Perception in Automatic Fact-Checking](https://mlsanity.com/paper/2410.15135/) (2024)
- [A Scoping Review of Natural Language Processing in Addressing Medically Inaccurate Information: Errors, Misinformation, and Hallucination](https://mlsanity.com/paper/2505.00008/) (2025)
- [Self-Adaptive Paraphrasing and Preference Learning for Improved Claim Verifiability](https://mlsanity.com/paper/2412.11653/) (2024)
- [Generating Grounded Responses to Counter Misinformation via Learning Efficient Fine-Grained Critiques](https://mlsanity.com/paper/2506.05924/) (2025)

## Abstract (arXiv)

The spread of online misinformation threatens public health, democracy, and the broader society. While professional fact-checkers form the first line of defense by fact-checking popular false claims, they do not engage directly in conversations with misinformation spreaders. On the other hand, non-expert ordinary users act as eyes-on-the-ground who proactively counter misinformation -- recent research has shown that 96% counter-misinformation responses are made by ordinary users. However, research also found that 2/3 times, these responses are rude and lack evidence. This work seeks to create a counter-misinformation response generation model to empower users to effectively correct misinformation. This objective is challenging due to the absence of datasets containing ground-truth of ideal counter-misinformation responses, and the lack of models that can generate responses backed by communication theories. In this work, we create two novel datasets of misinformation and counter-misinformation response pairs from in-the-wild social media and crowdsourcing from college-educated students. We annotate the collected data to distinguish poor from ideal responses that are factual, polite, and refute misinformation. We propose MisinfoCorrect, a reinforcement learning-based framework that learns to generate counter-misinformation responses for an input misinformation post. The model rewards the generator to increase the politeness, factuality, and refutation attitude while retaining text fluency and relevancy. Quantitative and qualitative evaluation shows that our model outperforms several baselines by generating high-quality counter-responses. This work illustrates the promise of generative text models for social good -- here, to help create a safe and reliable information ecosystem. The code and data is accessible on https://github.com/claws-lab/MisinfoCorrect.