# Overview of the CLEF--2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News

arXiv: https://arxiv.org/abs/2109.12987 · HTML: https://mlsanity.com/paper/2109.12987/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 101 references that are in the map.
- First posted on arXiv 23 Sep 2021.

## Builds on (most cited first, 30 of 35)

- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [Automated Fact-Checking for Assisting Human Fact-Checkers](https://mlsanity.com/paper/2103.07769/) (2021)
- [FANG: Leveraging Social Context for Fake News Detection Using Graph Representation](https://mlsanity.com/paper/2008.07939/) (2020)
- [That is a Known Lie: Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2005.06058/) (2020)
- [SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles](https://mlsanity.com/paper/2009.02696/) (2020)
- [SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours](https://mlsanity.com/paper/1704.05972/) (2017)
- [RumourEval 2019: Determining Rumour Veracity and Support for Rumours](https://mlsanity.com/paper/1809.06683/) (2018)
- [Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society](https://mlsanity.com/paper/2005.00033/) (2020)
- [A Survey on Natural Language Processing for Fake News Detection](https://mlsanity.com/paper/1811.00770/) (2018)
- [Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads](https://mlsanity.com/paper/1511.07487/) (2015)
- [FakeCovid -- A Multilingual Cross-domain Fact Check News Dataset for COVID-19](https://mlsanity.com/paper/2006.11343/) (2020)
- [SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images](https://mlsanity.com/paper/2105.09284/) (2021)
- [A Retrospective Analysis of the Fake News Challenge Stance Detection Task](https://mlsanity.com/paper/1806.05180/) (2018)
- [Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media](https://mlsanity.com/paper/2007.07997/) (2020)
- [What Was Written vs. Who Read It: News Media Profiling Using Text Analysis and Social Media Context](https://mlsanity.com/paper/2005.04518/) (2020)
- [Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthiness](https://mlsanity.com/paper/1808.05542/) (2018)
- [Fighting the COVID-19 Infodemic in Social Media: A Holistic Perspective and a Call to Arms](https://mlsanity.com/paper/2007.07996/) (2020)
- [An Exploratory Study of COVID-19 Misinformation on Twitter](https://mlsanity.com/paper/2005.05710/) (2020)
- [Fully Automated Fact Checking Using External Sources](https://mlsanity.com/paper/1710.00341/) (2017)
- [SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums](https://mlsanity.com/paper/1906.01727/) (2019)
- [The Role of Context in Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2104.07423/) (2021)
- [Accenture at CheckThat! 2020: If you say so: Post-hoc fact-checking of claims using transformer-based models](https://mlsanity.com/paper/2009.02431/) (2020)
- [It Takes Nine to Smell a Rat: Neural Multi-Task Learning for Check-Worthiness Prediction](https://mlsanity.com/paper/1908.07912/) (2019)
- [TweetCred: Real-Time Credibility Assessment of Content on Twitter](https://mlsanity.com/paper/1405.5490/) (2014)
- [SemEval-2016 Task 3: Community Question Answering](https://mlsanity.com/paper/1912.01972/) (2019)
- [People on Media: Jointly Identifying Credible News and Trustworthy Citizen Journalists in Online Communities](https://mlsanity.com/paper/1705.02667/) (2017)
- [Team Alex at CLEF CheckThat! 2020: Identifying Check-Worthy Tweets With Transformer Models](https://mlsanity.com/paper/2009.02931/) (2020)
- [Check_square at CheckThat! 2020: Claim Detection in Social Media via Fusion of Transformer and Syntactic Features](https://mlsanity.com/paper/2007.10534/) (2020)
- [AMUSED: An Annotation Framework of Multi-modal Social Media Data](https://mlsanity.com/paper/2010.00502/) (2020)

## Built on by (most cited first, 23 of 23)

- [DialFact: A Benchmark for Fact-Checking in Dialogue](https://mlsanity.com/paper/2110.08222/) (2021)
- [AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators](https://mlsanity.com/paper/2402.11073/) (2024)
- [Generative Large Language Models in Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2407.02351/) (2024)
- [Overview of the WANLP 2022 Shared Task on Propaganda Detection in Arabic](https://mlsanity.com/paper/2211.10057/) (2022)
- [Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages](https://mlsanity.com/paper/2112.09301/) (2021)
- [From Chaos to Clarity: Claim Normalization to Empower Fact-Checking](https://mlsanity.com/paper/2310.14338/) (2023)
- [Dataset of Fake News Detection and Fact Verification: A Survey](https://mlsanity.com/paper/2111.03299/) (2021)
- [Evaluating Generative Ad Hoc Information Retrieval](https://mlsanity.com/paper/2311.04694/) (2023)
- [MetaSumPerceiver: Multimodal Multi-Document Evidence Summarization for Fact-Checking](https://mlsanity.com/paper/2407.13089/) (2024)
- [CrowdChecked: Detecting Previously Fact-Checked Claims in Social Media](https://mlsanity.com/paper/2210.04447/) (2022)
- [Lost in Translation, Found in Spans: Identifying Claims in Multilingual Social Media](https://mlsanity.com/paper/2310.18205/) (2023)
- [Detecting Check-Worthy Claims in Political Debates, Speeches, and Interviews Using Audio Data](https://mlsanity.com/paper/2306.05535/) (2023)
- [Adversarial Attacks and Defenses for Social Network Text Processing Applications: Techniques, Challenges and Future Research Directions](https://mlsanity.com/paper/2110.13980/) (2021)
- [Multilingual Detection of Check-Worthy Claims using World Languages and Adapter Fusion](https://mlsanity.com/paper/2301.05494/) (2023)
- [ArCovidVac: Analyzing Arabic Tweets About COVID-19 Vaccination](https://mlsanity.com/paper/2201.06496/) (2022)
- [Cross-lingual Transfer Learning for Check-worthy Claim Identification over Twitter](https://mlsanity.com/paper/2211.05087/) (2022)
- [Leveraging Social Discourse to Measure Check-worthiness of Claims for Fact-checking](https://mlsanity.com/paper/2309.09274/) (2023)
- [TeamX@DravidianLangTech-ACL2022: A Comparative Analysis for Troll-Based Meme Classification](https://mlsanity.com/paper/2205.04404/) (2022)
- [Z-Index at CheckThat! Lab 2022: Check-Worthiness Identification on Tweet Text](https://mlsanity.com/paper/2207.07308/) (2022)
- [ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safety](https://mlsanity.com/paper/2508.20468/) (2025)
- [From Articles to Premises: Building PrimeFacts, an Extraction Methodology and Resource for Fact-Checking Evidence](https://mlsanity.com/paper/2605.06006/) (2026)
- [MythQA: Query-Based Large-Scale Check-Worthy Claim Detection through Multi-Answer Open-Domain Question Answering](https://mlsanity.com/paper/2307.11848/) (2023)
- [AKCIT-FN at CheckThat! 2025: Switching Fine-Tuned SLMs and LLM Prompting for Multilingual Claim Normalization](https://mlsanity.com/paper/2509.11496/) (2025)

## Abstract (arXiv)

We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality, and covers Arabic, Bulgarian, English, Spanish, and Turkish. Task 1 asks to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics (in all five languages). Task 2 asks to determine whether a claim in a tweet can be verified using a set of previously fact-checked claims (in Arabic and English). Task 3 asks to predict the veracity of a news article and its topical domain (in English). The evaluation is based on mean average precision or precision at rank k for the ranking tasks, and macro-F1 for the classification tasks. This was the most popular CLEF-2021 lab in terms of team registrations: 132 teams. Nearly one-third of them participated: 15, 5, and 25 teams submitted official runs for tasks 1, 2, and 3, respectively.