# Auditing and Robustifying COVID-19 Misinformation Datasets via Anticontent Sampling

arXiv: https://arxiv.org/abs/2310.07078 · HTML: https://mlsanity.com/paper/2310.07078/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 5 Aug 2023.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Neural Unsupervised Domain Adaptation in NLP---A Survey](https://mlsanity.com/paper/2006.00632/) (2020)
- [CoAID: COVID-19 Healthcare Misinformation Dataset](https://mlsanity.com/paper/2006.00885/) (2020)
- [Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society](https://mlsanity.com/paper/2005.00033/) (2020)
- [The COVID-19 Social Media Infodemic](https://mlsanity.com/paper/2003.05004/) (2020)
- [Characterizing COVID-19 Misinformation Communities Using a Novel Twitter Dataset](https://mlsanity.com/paper/2008.00791/) (2020)
- [The Role of the Crowd in Countering Misinformation: A Case Study of the COVID-19 Infodemic](https://mlsanity.com/paper/2011.05773/) (2020)
- [We Don't Speak the Same Language: Interpreting Polarization through Machine Translation](https://mlsanity.com/paper/2010.02339/) (2020)
- [PETGEN: Personalized Text Generation Attack on Deep Sequence Embedding-based Classification Models](https://mlsanity.com/paper/2109.06777/) (2021)
- [Are Chess Discussions Racist? An Adversarial Hate Speech Data Set](https://mlsanity.com/paper/2011.10280/) (2020)
- [Voice for the Voiceless: Active Sampling to Detect Comments Supporting the Rohingyas](https://mlsanity.com/paper/1910.03206/) (2019)
- [Fringe News Networks: Dynamics of US News Viewership following the 2020 Presidential Election](https://mlsanity.com/paper/2101.10112/) (2021)

## Built on by (most cited first, 3 of 3)

- [Infrastructure Ombudsman: Mining Future Failure Concerns from Structural Disaster Response](https://mlsanity.com/paper/2402.13528/) (2024)
- [Hope vs. Hate: Understanding User Interactions with LGBTQ+ News Content in Mainstream US News Media through the Lens of Hope Speech](https://mlsanity.com/paper/2502.09004/) (2025)
- [Investigating Vaccine Buyer's Remorse: Post-Vaccination Decision Regret in COVID-19 Social Media Using Politically Diverse Human Annotation](https://mlsanity.com/paper/2604.09626/) (2026)

## Abstract (arXiv)

This paper makes two key contributions. First, it argues that highly specialized rare content classifiers trained on small data typically have limited exposure to the richness and topical diversity of the negative class (dubbed anticontent) as observed in the wild. As a result, these classifiers' strong performance observed on the test set may not translate into real-world settings. In the context of COVID-19 misinformation detection, we conduct an in-the-wild audit of multiple datasets and demonstrate that models trained with several prominently cited recent datasets are vulnerable to anticontent when evaluated in the wild. Second, we present a novel active learning pipeline that requires zero manual annotation and iteratively augments the training data with challenging anticontent, robustifying these classifiers.