mlsanityData to 14 Oct 2026

Overview of the CLEF--2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News

arXiv 2109.12987 ↗First posted 23 Sep 2021cs.CL cs.IR cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality, and covers Arabic, Bulgarian, English, Spanish, and Turkish. Task 1 asks to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics (in all five languages). Task 2 asks to determine whether a claim in a tweet can be verified using a set of previously fact-checked claims (in Arabic and English). Task 3 asks to predict the veracity of a news article and its topical domain (in English). The evaluation is based on mean average precision or precision at rank k for the ranking tasks, and macro-F1 for the classification tasks. This was the most popular CLEF-2021 lab in terms of team registrations: 132 teams. Nearly one-third of them participated: 15, 5, and 25 teams submitted official runs for tasks 1, 2, and 3, respectively.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 35 of its 101 references that are in the map. First posted on arXiv 23 Sep 2021.

Builds on

35 of 101 references.

References · 35

  1. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  2. 2017Fake News Detection on Social Media: A Data Mining Perspectivecited by 336
  3. 2021Automated Fact-Checking for Assisting Human Fact-Checkerscited by 144
  4. 2020FANG: Leveraging Social Context for Fake News Detection Using Graph Representationcited by 99
  5. 2020That is a Known Lie: Detecting Previously Fact-Checked Claimscited by 82
  6. 2020SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articlescited by 81
  7. 2017SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumourscited by 77
  8. 2018RumourEval 2019: Determining Rumour Veracity and Support for Rumourscited by 75
  9. 2020Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Societycited by 72
  10. 2018A Survey on Natural Language Processing for Fake News Detectioncited by 67
  11. 2015Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threadscited by 64
  12. 2020FakeCovid -- A Multilingual Cross-domain Fact Check News Dataset for COVID-19cited by 61
23 more
  1. 2021SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Imagescited by 50
  2. 2018A Retrospective Analysis of the Fake News Challenge Stance Detection Taskcited by 49
  3. 2020Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Mediacited by 46
  4. 2020What Was Written vs. Who Read It: News Media Profiling Using Text Analysis and Social Media Contextcited by 42
  5. 2018Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthinesscited by 41
  6. 2020Fighting the COVID-19 Infodemic in Social Media: A Holistic Perspective and a Call to Armscited by 41
  7. 2020An Exploratory Study of COVID-19 Misinformation on Twittercited by 33
  8. 2017Fully Automated Fact Checking Using External Sourcescited by 27
  9. 2019SemEval-2019 Task 8: Fact Checking in Community Question Answering Forumscited by 18
  10. 2021The Role of Context in Detecting Previously Fact-Checked Claimscited by 17
  11. 2020Accenture at CheckThat! 2020: If you say so: Post-hoc fact-checking of claims using transformer-based modelscited by 15
  12. 2019It Takes Nine to Smell a Rat: Neural Multi-Task Learning for Check-Worthiness Predictioncited by 14
  13. 2014TweetCred: Real-Time Credibility Assessment of Content on Twittercited by 12
  14. 2019SemEval-2016 Task 3: Community Question Answeringcited by 12
  15. 2017People on Media: Jointly Identifying Credible News and Trustworthy Citizen Journalists in Online Communitiescited by 10
  16. 2020Team Alex at CLEF CheckThat! 2020: Identifying Check-Worthy Tweets With Transformer Modelscited by 8
  17. 2020Check_square at CheckThat! 2020: Claim Detection in Social Media via Fusion of Transformer and Syntactic Featurescited by 7
  18. 2020AMUSED: An Annotation Framework of Multi-modal Social Media Datacited by 7
  19. 2021Accenture at CheckThat! 2021: Interesting claim identification and ranking with contextually sensitive lexical training data augmentationcited by 6
  20. 2021UPV at CheckThat! 2021: Mitigating Cultural Differences for Identifying Multilingual Check-worthy Claimscited by 6
  21. 2021NoFake at CheckThat! 2021: Fake News Detection Using BERTcited by 3
  22. 2017Enriching Existing Test Collections with OXPathcited by 2
  23. 2021Tiplines to Combat Misinformation on Encrypted Platforms: A Case Study of the 2019 Indian Election on WhatsAppcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 6; 2023: 7; 2024: 3; 2025: 2; 2026: 1.

23 papers.

Sort:

  1. 2021DialFact: A Benchmark for Fact-Checking in Dialoguecited by 62
  2. 2024AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotatorscited by 28
  3. 2024Generative Large Language Models in Automated Fact-Checking: A Surveycited by 19
  4. 2022Overview of the WANLP 2022 Shared Task on Propaganda Detection in Arabiccited by 18
  5. 2021Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languagescited by 17
  6. 2023From Chaos to Clarity: Claim Normalization to Empower Fact-Checkingcited by 17
  7. 2021Dataset of Fake News Detection and Fact Verification: A Surveycited by 11
  8. 2023Evaluating Generative Ad Hoc Information Retrievalcited by 11
  9. 2024MetaSumPerceiver: Multimodal Multi-Document Evidence Summarization for Fact-Checkingcited by 10
  10. 2022CrowdChecked: Detecting Previously Fact-Checked Claims in Social Mediacited by 9
  11. 2023Lost in Translation, Found in Spans: Identifying Claims in Multilingual Social Mediacited by 8
  12. 2023Detecting Check-Worthy Claims in Political Debates, Speeches, and Interviews Using Audio Datacited by 7
11 more
  1. 2021Adversarial Attacks and Defenses for Social Network Text Processing Applications: Techniques, Challenges and Future Research Directionscited by 4
  2. 2023Multilingual Detection of Check-Worthy Claims using World Languages and Adapter Fusioncited by 4
  3. 2022ArCovidVac: Analyzing Arabic Tweets About COVID-19 Vaccinationcited by 3
  4. 2022Cross-lingual Transfer Learning for Check-worthy Claim Identification over Twittercited by 2
  5. 2023Leveraging Social Discourse to Measure Check-worthiness of Claims for Fact-checkingcited by 2
  6. 2022TeamX@DravidianLangTech-ACL2022: A Comparative Analysis for Troll-Based Meme Classificationcited by 1
  7. 2022Z-Index at CheckThat! Lab 2022: Check-Worthiness Identification on Tweet Textcited by 1
  8. 2025ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safetycited by 1
  9. 2026From Articles to Premises: Building PrimeFacts, an Extraction Methodology and Resource for Fact-Checking Evidencecited by 1
  10. 2023MythQA: Query-Based Large-Scale Check-Worthy Claim Detection through Multi-Answer Open-Domain Question Answeringcited by 0
  11. 2025AKCIT-FN at CheckThat! 2025: Switching Fine-Tuned SLMs and LLM Prompting for Multilingual Claim Normalizationcited by 0