# Credible, Unreliable or Leaked?: Evidence Verification for Enhanced Automated Fact-checking arXiv: https://arxiv.org/abs/2404.18971 ยท HTML: https://mlsanity.com/paper/2404.18971/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Builds on 29 of its 41 references that are in the map. - First posted on arXiv 29 Apr 2024. ## Builds on (most cited first, 29 of 29) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019) - [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020) - [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020) - [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018) - [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021) - [Multi-attentional Deepfake Detection](https://mlsanity.com/paper/2103.02406/) (2021) - [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018) - [LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://mlsanity.com/paper/2112.07916/) (2021) - [Explainable Automated Fact-Checking for Public Health Claims](https://mlsanity.com/paper/2010.09926/) (2020) - [MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims](https://mlsanity.com/paper/1909.03242/) (2019) - [AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Web](https://mlsanity.com/paper/2305.13117/) (2023) - [Visual News: Benchmark and Challenges in News Image Captioning](https://mlsanity.com/paper/2010.03743/) (2020) - [NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media](https://mlsanity.com/paper/2104.05893/) (2021) - [End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Models](https://mlsanity.com/paper/2205.12487/) (2022) - [DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning](https://mlsanity.com/paper/1809.06416/) (2018) - [Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources](https://mlsanity.com/paper/2112.00061/) (2021) - [A Survey on Natural Language Processing for Fake News Detection](https://mlsanity.com/paper/1811.00770/) (2018) - [A Survey on Multimodal Disinformation Detection](https://mlsanity.com/paper/2103.12541/) (2021) - [Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation](https://mlsanity.com/paper/2210.13865/) (2022) - [VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Bias](https://mlsanity.com/paper/2304.14133/) (2023) - [NELA-GT-2020: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles](https://mlsanity.com/paper/2102.04567/) (2021) - [Fully Automated Fact Checking Using External Sources](https://mlsanity.com/paper/1710.00341/) (2017) - [COSMOS: Catching Out-of-Context Misinformation with Self-Supervised Learning](https://mlsanity.com/paper/2101.06278/) (2021) - [NELA-GT-2018: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles](https://mlsanity.com/paper/1904.01546/) (2019) - [Sampling the News Producers: A Large News and Feature Data Set for the Study of the Complex Media Landscape](https://mlsanity.com/paper/1803.10124/) (2018) - [RED-DOT: Multimodal Fact-checking via Relevant Evidence Detection](https://mlsanity.com/paper/2311.09939/) (2023) - [NELA-GT-2019: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles](https://mlsanity.com/paper/2003.08444/) (2020) - [NELA-GT-2022: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles](https://mlsanity.com/paper/2203.05659/) (2022) ## Built on by (most cited first, 8 of 8) - [DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts](https://mlsanity.com/paper/2412.10510/) (2024) - [COVE: COntext and VEracity prediction for out-of-context images](https://mlsanity.com/paper/2502.01194/) (2025) - [Similarity over Factuality: Are we making progress on multimodal out-of-context misinformation detection?](https://mlsanity.com/paper/2407.13488/) (2024) - [A Reality Check on Context Utilisation for Retrieval-Augmented Generation](https://mlsanity.com/paper/2412.17031/) (2024) - [Latent Reconstruction from Generated Data for Multimodal Misinformation Detection](https://mlsanity.com/paper/2504.06010/) (2025) - [CoVLM: Leveraging Consensus from Vision-Language Models for Semi-supervised Multi-modal Fake News Detection](https://mlsanity.com/paper/2410.04426/) (2024) - [Evidence Triangulation for Multimodal Fact-Checking in the Wild](https://mlsanity.com/paper/2606.31367/) (2026) - [Can temporal article-level credibility signals improve domain-level credibility prediction?](https://mlsanity.com/paper/2607.04560/) (2026) ## Abstract (arXiv) Automated fact-checking (AFC) is garnering increasing attention by researchers aiming to help fact-checkers combat the increasing spread of misinformation online. While many existing AFC methods incorporate external information from the Web to help examine the veracity of claims, they often overlook the importance of verifying the source and quality of collected "evidence". One overlooked challenge involves the reliance on "leaked evidence", information gathered directly from fact-checking websites and used to train AFC systems, resulting in an unrealistic setting for early misinformation detection. Similarly, the inclusion of information from unreliable sources can undermine the effectiveness of AFC systems. To address these challenges, we present a comprehensive approach to evidence verification and filtering. We create the "CREDible, Unreliable or LEaked" (CREDULE) dataset, which consists of 91,632 articles classified as Credible, Unreliable and Fact checked (Leaked). Additionally, we introduce the EVidence VERification Network (EVVER-Net), trained on CREDULE to detect leaked and unreliable evidence in both short and long texts. EVVER-Net can be used to filter evidence collected from the Web, thus enhancing the robustness of end-to-end AFC systems. We experiment with various language models and show that EVVER-Net can demonstrate impressive performance of up to 91.5% and 94.4% accuracy, while leveraging domain credibility scores along with short or long texts, respectively. Finally, we assess the evidence provided by widely-used fact-checking datasets including LIAR-PLUS, MOCHEG, FACTIFY, NewsCLIPpings+ and VERITE, some of which exhibit concerning rates of leaked and unreliable evidence.