# Construction of Large-Scale Misinformation Labeled Datasets from Social Media Discourse using Label Refinement

arXiv: https://arxiv.org/abs/2202.12413 · HTML: https://mlsanity.com/paper/2202.12413/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 17 of 17)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [Unsupervised Label Noise Modeling and Loss Correction](https://mlsanity.com/paper/1904.11238/) (2019)
- [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018)
- [CSI: A Hybrid Deep Model for Fake News Detection](https://mlsanity.com/paper/1703.06959/) (2017)
- [Identifying Coordinated Accounts on Social Media through Hidden Influence and Group Behaviours](https://mlsanity.com/paper/2008.11308/) (2020)
- [ReCOVery: A Multimodal Repository for COVID-19 News Credibility Research](https://mlsanity.com/paper/2006.05557/) (2020)
- [Combating Fake News: A Survey on Identification and Mitigation Techniques](https://mlsanity.com/paper/1901.06437/) (2019)
- [Characterizing COVID-19 Misinformation Communities Using a Novel Twitter Dataset](https://mlsanity.com/paper/2008.00791/) (2020)
- [Weak Supervision for Fake News Detection via Reinforcement Learning](https://mlsanity.com/paper/1912.12520/) (2019)
- [Quantifying Controversy in Social Media](https://mlsanity.com/paper/1507.05224/) (2015)
- [Ginger Cannot Cure Cancer: Battling Fake Health News with a Comprehensive Data Repository](https://mlsanity.com/paper/2002.00837/) (2020)
- [NoiseRank: Unsupervised Label Noise Reduction with Dependence Models](https://mlsanity.com/paper/2003.06729/) (2020)
- [A first look at COVID-19 information and misinformation sharing on Twitter](https://mlsanity.com/paper/2003.13907/) (2020)
- [Detecting Fake News with Weak Social Supervision](https://mlsanity.com/paper/1910.11430/) (2019)
- [Characterizing Online Engagement with Disinformation and Conspiracies in the 2020 U.S. Presidential Election](https://mlsanity.com/paper/2107.08319/) (2021)

## Built on by (most cited first, 4 of 4)

- [FACT-GPT: Fact-Checking Augmentation via Claim Matching with LLMs](https://mlsanity.com/paper/2402.05904/) (2024)
- [Automated Claim Matching with Large Language Models: Empowering Fact-Checkers in the Fight Against Misinformation](https://mlsanity.com/paper/2310.09223/) (2023)
- [Interpretable Detection of Out-of-Context Misinformation with Neural-Symbolic-Enhanced Large Multimodal Model](https://mlsanity.com/paper/2304.07633/) (2023)
- [Leveraging Social Interactions to Detect Misinformation on Social Media](https://mlsanity.com/paper/2304.02983/) (2023)

## Abstract (arXiv)

Malicious accounts spreading misinformation has led to widespread false and misleading narratives in recent times, especially during the COVID-19 pandemic, and social media platforms struggle to eliminate these contents rapidly. This is because adapting to new domains requires human intensive fact-checking that is slow and difficult to scale. To address this challenge, we propose to leverage news-source credibility labels as weak labels for social media posts and propose model-guided refinement of labels to construct large-scale, diverse misinformation labeled datasets in new domains. The weak labels can be inaccurate at the article or social media post level where the stance of the user does not align with the news source or article credibility. We propose a framework to use a detection model self-trained on the initial weak labels with uncertainty sampling based on entropy in predictions of the model to identify potentially inaccurate labels and correct for them using self-supervision or relabeling. The framework will incorporate social context of the post in terms of the community of its associated user for surfacing inaccurate labels towards building a large-scale dataset with minimum human effort. To provide labeled datasets with distinction of misleading narratives where information might be missing significant context or has inaccurate ancillary details, the proposed framework will use the few labeled samples as class prototypes to separate high confidence samples into false, unproven, mixture, mostly false, mostly true, true, and debunk information. The approach is demonstrated for providing a large-scale misinformation dataset on COVID-19 vaccines.