# Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthiness

arXiv: https://arxiv.org/abs/1808.05542 · HTML: https://mlsanity.com/paper/1808.05542/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2018.

## Built on by (most cited first, 30 of 41)

- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [Automated Fact-Checking for Assisting Human Fact-Checkers](https://mlsanity.com/paper/2103.07769/) (2021)
- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://mlsanity.com/paper/2103.00242/) (2021)
- [Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2109.11427/) (2021)
- [Multimodal Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2305.13507/) (2023)
- [SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images](https://mlsanity.com/paper/2105.09284/) (2021)
- [Claim Detection for Automated Fact-checking: A Survey on Monolingual, Multilingual and Cross-Lingual Research](https://mlsanity.com/paper/2401.11969/) (2024)
- [Zoom Out and Observe: News Environment Perception for Fake News Detection](https://mlsanity.com/paper/2203.10885/) (2022)
- [Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models](https://mlsanity.com/paper/2305.14623/) (2023)
- [Overview of the CLEF--2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News](https://mlsanity.com/paper/2109.12987/) (2021)
- [Assisting the Human Fact-Checkers: Detecting All Previously Fact-Checked Claims in a Document](https://mlsanity.com/paper/2109.07410/) (2021)
- [Overview of the CLEF-2019 CheckThat!: Automatic Identification and Verification of Claims](https://mlsanity.com/paper/2109.15118/) (2021)
- [Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checking](https://mlsanity.com/paper/2411.05375/) (2024)
- [The Role of Context in Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2104.07423/) (2021)
- [Dataset of Fake News Detection and Fact Verification: A Survey](https://mlsanity.com/paper/2111.03299/) (2021)
- [Environmental Claim Detection](https://mlsanity.com/paper/2209.00507/) (2022)
- [Multimodal Fact-Checking with Vision Language Models: A Probing Classifier based Solution with Embedding Strategies](https://mlsanity.com/paper/2412.05155/) (2024)
- [Lost in Translation, Found in Spans: Identifying Claims in Multilingual Social Media](https://mlsanity.com/paper/2310.18205/) (2023)
- [UPV at CheckThat! 2021: Mitigating Cultural Differences for Identifying Multilingual Check-worthy Claims](https://mlsanity.com/paper/2109.09232/) (2021)
- [FactFinders at CheckThat! 2024: Refining Check-worthy Statement Detection with LLMs through Data Pruning](https://mlsanity.com/paper/2406.18297/) (2024)
- [Constraint 2021: Machine Learning Models for COVID-19 Fake News Detection Shared Task](https://mlsanity.com/paper/2101.03717/) (2021)
- [Scalable Fact-checking with Human-in-the-Loop](https://mlsanity.com/paper/2109.10992/) (2021)
- [A Second Pandemic? Analysis of Fake News About COVID-19 Vaccines in Qatar](https://mlsanity.com/paper/2109.11372/) (2021)
- [Overview of the CLAIMSCAN-2023: Uncovering Truth in Social Media through Claim Detection and Identification of Claim Spans](https://mlsanity.com/paper/2310.19267/) (2023)
- [SciClaimHunt: A Large Dataset for Evidence-based Scientific Claim Verification](https://mlsanity.com/paper/2502.10003/) (2025)
- [HintsOfTruth: A Multimodal Checkworthiness Detection Dataset with Real and Synthetic Claims](https://mlsanity.com/paper/2502.11753/) (2025)
- [WhatTheWikiFact: Fact-Checking Claims Against Wikipedia](https://mlsanity.com/paper/2105.00826/) (2021)
- [Cross-lingual Transfer Learning for Check-worthy Claim Identification over Twitter](https://mlsanity.com/paper/2211.05087/) (2022)
- [Check-worthy Claim Detection across Topics for Automated Fact-checking](https://mlsanity.com/paper/2212.08514/) (2022)

## Abstract (arXiv)

We present an overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims, with focus on Task 1: Check-Worthiness. The task asks to predict which claims in a political debate should be prioritized for fact-checking. In particular, given a debate or a political speech, the goal was to produce a ranked list of its sentences based on their worthiness for fact checking. We offered the task in both English and Arabic, based on debates from the 2016 US Presidential Campaign, as well as on some speeches during and after the campaign. A total of 30 teams registered to participate in the Lab and seven teams actually submitted systems for Task~1. The most successful approaches used by the participants relied on recurrent and multi-layer neural networks, as well as on combinations of distributional representations, on matchings claims' vocabulary against lexicons, and on measures of syntactic dependency. The best systems achieved mean average precision of 0.18 and 0.15 on the English and on the Arabic test datasets, respectively. This leaves large room for further improvement, and thus we release all datasets and the scoring scripts, which should enable further research in check-worthiness estimation.