# CoVERT: A Corpus of Fact-checked Biomedical COVID-19 Tweets

arXiv: https://arxiv.org/abs/2204.12164 · HTML: https://mlsanity.com/paper/2204.12164/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 26 Apr 2022.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/1902.07669/) (2019)
- [The Fact Extraction and VERification (FEVER) Shared Task](https://mlsanity.com/paper/1811.10971/) (2018)
- [DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning](https://mlsanity.com/paper/1809.06416/) (2018)
- [COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic](https://mlsanity.com/paper/2106.03794/) (2021)
- [Automated Fact Checking: Task formulations, methods and future directions](https://mlsanity.com/paper/1806.07687/) (2018)
- [Explainable Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2011.03870/) (2020)
- [Can Language Models be Biomedical Knowledge Bases?](https://mlsanity.com/paper/2109.07154/) (2021)
- [A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking](https://mlsanity.com/paper/1911.01214/) (2019)
- [The Limitations of Stylometry for Detecting Machine-Generated Fake News](https://mlsanity.com/paper/1908.09805/) (2019)
- [FakeCovid -- A Multilingual Cross-domain Fact Check News Dataset for COVID-19](https://mlsanity.com/paper/2006.11343/) (2020)
- [Language Models as Fact Checkers?](https://mlsanity.com/paper/2006.04102/) (2020)
- [An Exploratory Study of COVID-19 Misinformation on Twitter](https://mlsanity.com/paper/2005.05710/) (2020)
- [Claim Detection in Biomedical Twitter Posts](https://mlsanity.com/paper/2104.11639/) (2021)
- [A first look at COVID-19 information and misinformation sharing on Twitter](https://mlsanity.com/paper/2003.13907/) (2020)
- [Prevalence of Low-Credibility Information on Twitter During the COVID-19 Outbreak](https://mlsanity.com/paper/2004.14484/) (2020)

## Built on by (most cited first, 29 of 29)

- [Scientific Fact-Checking: A Survey of Resources and Approaches](https://mlsanity.com/paper/2305.16859/) (2023)
- [SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables](https://mlsanity.com/paper/2305.13186/) (2023)
- [If in a Crowdsourced Data Annotation Pipeline, a GPT-4](https://mlsanity.com/paper/2402.16795/) (2024)
- [HealthFC: Verifying Health Claims with Evidence-Based Medical Fact-Checking](https://mlsanity.com/paper/2309.08503/) (2023)
- [FactCHD: Benchmarking Fact-Conflicting Hallucination Detection](https://mlsanity.com/paper/2310.12086/) (2023)
- [Check-COVID: Fact-Checking COVID-19 News Claims with Scientific Evidence](https://mlsanity.com/paper/2305.18265/) (2023)
- [DataFlow: An LLM-Driven Framework for Unified Data Preparation and Workflow Automation in the Era of Data-Centric AI](https://mlsanity.com/paper/2512.16676/) (2025)
- [Modeling Information Change in Science Communication with Semantically Matched Paraphrases](https://mlsanity.com/paper/2210.13001/) (2022)
- [RARE: Retrieval-Augmented Reasoning Modeling](https://mlsanity.com/paper/2503.23513/) (2025)
- [What Makes Medical Claims (Un)Verifiable? Analyzing Entity and Relation Properties for Fact Verification](https://mlsanity.com/paper/2402.01360/) (2024)
- [Comparing Knowledge Sources for Open-Domain Scientific Claim Verification](https://mlsanity.com/paper/2402.02844/) (2024)
- [Understanding Fine-grained Distortions in Reports of Scientific Findings](https://mlsanity.com/paper/2402.12431/) (2024)
- [MuSciClaims: Multimodal Scientific Claim Verification](https://mlsanity.com/paper/2506.04585/) (2025)
- [Entity-based Claim Representation Improves Fact-Checking of Medical Content in Tweets](https://mlsanity.com/paper/2209.07834/) (2022)
- [The Next Phase of Scientific Fact-Checking: Advanced Evidence Retrieval from Complex Structured Academic Papers](https://mlsanity.com/paper/2506.20844/) (2025)
- [MedFact: A Large-scale Chinese Dataset for Evidence-based Medical Fact-checking of LLM Responses](https://mlsanity.com/paper/2509.17436/) (2025)
- [An Entity-based Claim Extraction Pipeline for Real-world Biomedical Fact-checking](https://mlsanity.com/paper/2304.05268/) (2023)
- [Quantifying Self-diagnostic Atomic Knowledge in Chinese Medical Foundation Model: A Computational Analysis](https://mlsanity.com/paper/2310.11722/) (2023)
- [Overview of Factify5WQA: Fact Verification through 5W Question-Answering](https://mlsanity.com/paper/2410.04236/) (2024)
- [NSF-SciFy: Mining the NSF Awards Database for Scientific Claims](https://mlsanity.com/paper/2503.08600/) (2025)
- [A Scoping Review of Natural Language Processing in Addressing Medically Inaccurate Information: Errors, Misinformation, and Hallucination](https://mlsanity.com/paper/2505.00008/) (2025)
- [ClimateViz: A Benchmark for Statistical Reasoning and Fact Verification on Scientific Charts](https://mlsanity.com/paper/2506.08700/) (2025)
- [Decide less, communicate more: On the construct validity of end-to-end fact-checking in medicine](https://mlsanity.com/paper/2506.20876/) (2025)
- [+VeriRel: Verification Feedback to Enhance Document Retrieval for Scientific Fact Checking](https://mlsanity.com/paper/2508.11122/) (2025)
- [IRIS: An Iterative and Integrated Framework for Verifiable Causal Discovery in the Absence of Tabular Data](https://mlsanity.com/paper/2510.09217/) (2025)
- [M2-Verify: A Large-Scale Multidomain Benchmark for Checking Multimodal Claim Consistency](https://mlsanity.com/paper/2604.01306/) (2026)
- [Assessing Automated Fact-Checking for Medical LLM Responses with Knowledge Graphs](https://mlsanity.com/paper/2511.12817/) (2025)
- [Angry but Accurate: Detecting and Profiling the Counter-Misinformation Ecosystem on Twitter](https://mlsanity.com/paper/2607.02900/) (2026)
- [Generating Biomedical Fact-Checking Reports with RL-Enhanced Agentic Search](https://mlsanity.com/paper/2608.23811/) (2026)

## Abstract (arXiv)

Over the course of the COVID-19 pandemic, large volumes of biomedical information concerning this new disease have been published on social media. Some of this information can pose a real danger to people's health, particularly when false information is shared, for instance recommendations on how to treat diseases without professional medical advice. Therefore, automatic fact-checking resources and systems developed specifically for the medical domain are crucial. While existing fact-checking resources cover COVID-19-related information in news or quantify the amount of misinformation in tweets, there is no dataset providing fact-checked COVID-19-related Twitter posts with detailed annotations for biomedical entities, relations and relevant evidence. We contribute CoVERT, a fact-checked corpus of tweets with a focus on the domain of biomedicine and COVID-19-related (mis)information. The corpus consists of 300 tweets, each annotated with medical named entities and relations. We employ a novel crowdsourcing methodology to annotate all tweets with fact-checking labels and supporting evidence, which crowdworkers search for online. This methodology results in moderate inter-annotator agreement. Furthermore, we use the retrieved evidence extracts as part of a fact-checking pipeline, finding that the real-world evidence is more useful than the knowledge indirectly available in pretrained language models.