# WhatsApp Vaccine Discourse (WhaVax): An Expert-Annotated Dataset and Benchmark for Health Misinformation Detection

arXiv: https://arxiv.org/abs/2605.12510 · HTML: https://mlsanity.com/paper/2605.12510/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 44 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Builds on (most cited first, 17 of 17)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Text Classification Algorithms: A Survey](https://mlsanity.com/paper/1904.08067/) (2019)
- [Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning](https://mlsanity.com/paper/1811.12808/) (2018)
- [CoAID: COVID-19 Healthcare Misinformation Dataset](https://mlsanity.com/paper/2006.00885/) (2020)
- [Winds of Change: Impact of COVID-19 on Vaccine-related Opinions of Twitter users](https://mlsanity.com/paper/2111.10667/) (2021)
- [CAVES: A Dataset to facilitate Explainable Classification and Summarization of Concerns towards COVID Vaccines](https://mlsanity.com/paper/2204.13746/) (2022)
- [VaxxHesitancy: A Dataset for Studying Hesitancy towards COVID-19 Vaccination on Twitter](https://mlsanity.com/paper/2301.06660/) (2023)
- [MMCoVaR: Multimodal COVID-19 Vaccine Focused Data Repository for Fake News Detection and a Baseline Architecture for Classification](https://mlsanity.com/paper/2109.06416/) (2021)
- [A Dataset of Fact-Checked Images Shared on WhatsApp During the Brazilian and Indian Elections](https://mlsanity.com/paper/2005.02443/) (2020)
- [VaccinEU: COVID-19 vaccine conversations on Twitter in French, German and Italian](https://mlsanity.com/paper/2201.06293/) (2022)
- [Automatic Detection of COVID-19 Vaccine Misinformation with Graph Link Prediction](https://mlsanity.com/paper/2108.02314/) (2021)
- [From Hesitancy Framings to Vaccine Hesitancy Profiles: A Journey of Stance, Ontological Commitments and Moral Foundations](https://mlsanity.com/paper/2202.09456/) (2022)
- [Echoes through Time: Evolution of the Italian COVID-19 Vaccination Debate](https://mlsanity.com/paper/2204.12943/) (2022)
- [Can WhatsApp Counter Misinformation by Limiting Message Forwarding?](https://mlsanity.com/paper/1909.08740/) (2019)
- [Vax-Culture: A Dataset for Studying Vaccine Discourse on Twitter](https://mlsanity.com/paper/2304.06858/) (2023)
- [Helping Fact-Checkers Identify Fake News Stories Shared through Images on WhatsApp](https://mlsanity.com/paper/2308.14782/) (2023)
- [Global Patterns of Viral Content on WhatsApp](https://mlsanity.com/paper/2407.08172/) (2024)

## Abstract (arXiv)

We introduce WhaVax, a new expert-annotated dataset of vaccine-related WhatsApp messages collected from large Brazilian public groups spanning multiple pandemic years. The dataset was constructed through a rigorous, carefully designed pipeline that integrates keyword-based data collection, semantic deduplication to remove near-duplicate content, and a multi-stage annotation protocol conducted by medical specialists. This process produced a high-quality gold-standard corpus, characterized by substantial inter-annotator agreement and strong reliability for downstream analysis. Additionally, we provide a detailed characterization of WhatsApp misinformation, revealing distinctive linguistic, structural, lexical, temporal, and group-level patterns, as well as a meaningful layer of ambiguous cases that reflect the complexity of health discourse in private messaging. We also benchmark classical models, fine-tuned Small Language Models, and zero- or few-shot Large Language Models under realistic data-scarcity constraints, demonstrating that strong embeddings and LLM approaches perform competitively, while domain alignment and data availability remain critical factors. This study provides a rare, high-quality resource to support misinformation research and computational modeling in encrypted communication environments.