# Overview of the CLEF-2019 CheckThat!: Automatic Identification and Verification of Claims

arXiv: https://arxiv.org/abs/2109.15118 · HTML: https://mlsanity.com/paper/2109.15118/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 47 references that are in the map.
- First posted on arXiv 25 Sep 2021.

## Builds on (most cited first, 17 of 17)

- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [Combining Fact Extraction and Verification with Neural Semantic Matching Networks](https://mlsanity.com/paper/1811.07039/) (2018)
- [DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning](https://mlsanity.com/paper/1809.06416/) (2018)
- [Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads](https://mlsanity.com/paper/1511.07487/) (2015)
- [Integrating Stance Detection and Fact Checking in a Unified Corpus](https://mlsanity.com/paper/1804.08012/) (2018)
- [Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthiness](https://mlsanity.com/paper/1808.05542/) (2018)
- [Fully Automated Fact Checking Using External Sources](https://mlsanity.com/paper/1710.00341/) (2017)
- [ClaimRank: Detecting Check-Worthy Claims in Arabic and English](https://mlsanity.com/paper/1804.07587/) (2018)
- [SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums](https://mlsanity.com/paper/1906.01727/) (2019)
- [Automatic Fact-Checking Using Context and Discourse Information](https://mlsanity.com/paper/1908.01328/) (2019)
- [People on Media: Jointly Identifying Credible News and Trustworthy Citizen Journalists in Online Communities](https://mlsanity.com/paper/1705.02667/) (2017)
- [Fact Checking in Community Forums](https://mlsanity.com/paper/1803.03178/) (2018)
- [In Search of Credible News](https://mlsanity.com/paper/1911.08125/) (2019)
- [Neural Check-Worthiness Ranking with Weak Supervision: Finding Sentences for Fact-Checking](https://mlsanity.com/paper/1903.08404/) (2019)
- [We Built a Fake News & Click-bait Filter: What Happened Next Will Blow Your Mind!](https://mlsanity.com/paper/1803.03786/) (2018)

## Built on by (most cited first, 18 of 18)

- [NewsClaims: A New Benchmark for Claim Detection from News with Attribute Knowledge](https://mlsanity.com/paper/2112.08544/) (2021)
- [Overview of the WANLP 2022 Shared Task on Propaganda Detection in Arabic](https://mlsanity.com/paper/2211.10057/) (2022)
- [Dataset of Fake News Detection and Fact Verification: A Survey](https://mlsanity.com/paper/2111.03299/) (2021)
- [CrowdChecked: Detecting Previously Fact-Checked Claims in Social Media](https://mlsanity.com/paper/2210.04447/) (2022)
- [Lost in Translation, Found in Spans: Identifying Claims in Multilingual Social Media](https://mlsanity.com/paper/2310.18205/) (2023)
- [FactFinders at CheckThat! 2024: Refining Check-worthy Statement Detection with LLMs through Data Pruning](https://mlsanity.com/paper/2406.18297/) (2024)
- [Search, Examine and Early-Termination: Fake News Detection with Annotation-Free Evidences](https://mlsanity.com/paper/2407.07931/) (2024)
- [Cross-lingual Transfer Learning for Check-worthy Claim Identification over Twitter](https://mlsanity.com/paper/2211.05087/) (2022)
- [Multiverse: Multilingual Evidence for Fake News Detection](https://mlsanity.com/paper/2211.14279/) (2022)
- [Check-worthy Claim Detection across Topics for Automated Fact-checking](https://mlsanity.com/paper/2212.08514/) (2022)
- [Leveraging Social Discourse to Measure Check-worthiness of Claims for Fact-checking](https://mlsanity.com/paper/2309.09274/) (2023)
- [Z-Index at CheckThat! Lab 2022: Check-Worthiness Identification on Tweet Text](https://mlsanity.com/paper/2207.07308/) (2022)
- [MultiCaption: Detecting disinformation using multilingual visual claims](https://mlsanity.com/paper/2601.11220/) (2026)
- [One LLM to Train Them All: Multi-Task Learning Framework for Fact-Checking](https://mlsanity.com/paper/2601.11293/) (2026)
- [IAI Group at CheckThat! 2024: Transformer Models and Data Augmentation for Checkworthy Claim Detection](https://mlsanity.com/paper/2408.01118/) (2024)
- [AKCIT-FN at CheckThat! 2025: Switching Fine-Tuned SLMs and LLM Prompting for Multilingual Claim Normalization](https://mlsanity.com/paper/2509.11496/) (2025)
- [ClaimPT: A Portuguese Dataset of Annotated Claims in News Articles](https://mlsanity.com/paper/2601.19490/) (2026)
- [MultiCW: A Large-Scale Balanced Benchmark Dataset for Training Robust Check-Worthiness Detection Models](https://mlsanity.com/paper/2602.16298/) (2026)

## Abstract (arXiv)

We present an overview of the second edition of the CheckThat! Lab at CLEF 2019. The lab featured two tasks in two different languages: English and Arabic. Task 1 (English) challenged the participating systems to predict which claims in a political debate or speech should be prioritized for fact-checking. Task 2 (Arabic) asked to (A) rank a given set of Web pages with respect to a check-worthy claim based on their usefulness for fact-checking that claim, (B) classify these same Web pages according to their degree of usefulness for fact-checking the target claim, (C) identify useful passages from these pages, and (D) use the useful pages to predict the claim's factuality. CheckThat! provided a full evaluation framework, consisting of data in English (derived from fact-checking sources) and Arabic (gathered and annotated from scratch) and evaluation based on mean average precision (MAP) and normalized discounted cumulative gain (nDCG) for ranking, and F1 for classification. A total of 47 teams registered to participate in this lab, and fourteen of them actually submitted runs (compared to nine last year). The evaluation results show that the most successful approaches to Task 1 used various neural networks and logistic regression. As for Task 2, learning-to-rank was used by the highest scoring runs for subtask A, while different classifiers were used in the other subtasks. We release to the research community all datasets from the lab as well as the evaluation scripts, which should enable further research in the important tasks of check-worthiness estimation and automatic claim verification.