# CrediBench: Building Web-Scale Network Datasets for Information Integrity

arXiv: https://arxiv.org/abs/2509.23340 · HTML: https://mlsanity.com/paper/2509.23340/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 38 of its 91 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Builds on (most cited first, 30 of 38)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Inductive Representation Learning on Temporal Graphs](https://mlsanity.com/paper/2002.07962/) (2020)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs](https://mlsanity.com/paper/2103.09430/) (2021)
- [Matryoshka Representation Learning](https://mlsanity.com/paper/2205.13147/) (2022)
- [Representation Learning for Dynamic Graphs: A Survey](https://mlsanity.com/paper/1905.11485/) (2019)
- [The Surprising Power of Graph Neural Networks with Random Node Initialization](https://mlsanity.com/paper/2010.01179/) (2020)
- [Towards Better Dynamic Graph Learning: New Architecture and Unified Library](https://mlsanity.com/paper/2303.13047/) (2023)
- [EmbeddingGemma: Powerful and Lightweight Text Representations](https://mlsanity.com/paper/2509.20354/) (2025)
- [MMTEB: Massive Multilingual Text Embedding Benchmark](https://mlsanity.com/paper/2502.13595/) (2025)
- [CSI: A Hybrid Deep Model for Fake News Detection](https://mlsanity.com/paper/1703.06959/) (2017)
- [The spread of low-credibility content by social bots](https://mlsanity.com/paper/1707.07592/) (2017)
- [Position: Graph Learning Will Lose Relevance Due To Poor Benchmarks](https://mlsanity.com/paper/2502.14546/) (2025)
- [PyG 2.0: Scalable Learning on Real World Graphs](https://mlsanity.com/paper/2507.16991/) (2025)
- [Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4](https://mlsanity.com/paper/2305.14928/) (2023)
- [Disinformation Detection: An Evolving Challenge in the Age of LLMs](https://mlsanity.com/paper/2309.15847/) (2023)
- [TGB 2.0: A Benchmark for Learning on Temporal Knowledge Graphs and Heterogeneous Graphs](https://mlsanity.com/paper/2406.09639/) (2024)
- [DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphs](https://mlsanity.com/paper/2406.12072/) (2024)
- [Accuracy and Political Bias of News Source Credibility Ratings by Large Language Models](https://mlsanity.com/paper/2304.00228/) (2023)
- [Adversarial Active Learning based Heterogeneous Graph Neural Network for Fake News Detection](https://mlsanity.com/paper/2101.11206/) (2021)
- [Some Like it Hoax: Automated Fake News Detection in Social Networks](https://mlsanity.com/paper/1704.07506/) (2017)
- [GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Data](https://mlsanity.com/paper/2409.14500/) (2024)
- [Cross-SEAN: A Cross-Stitch Semi-Supervised Neural Attention Model for COVID-19 Fake News Detection](https://mlsanity.com/paper/2102.08924/) (2021)
- [TweetCred: Real-Time Credibility Assessment of Content on Twitter](https://mlsanity.com/paper/1405.5490/) (2014)
- [A Golden Age: Conspiracy Theories' Relationship with Misinformation Outlets, News Media, and the Wider Internet](https://mlsanity.com/paper/2301.10880/) (2023)
- [Echo Chambers on Social Media: A comparative analysis](https://mlsanity.com/paper/2004.09603/) (2020)
- [No Calm in The Storm: Investigating QAnon Website Relationships](https://mlsanity.com/paper/2106.15715/) (2021)

## Abstract (arXiv)

Automatically assessing the credibility of online sources presents an invaluable tool for navigating today's information ecosystem. However, existing approaches either depend on scarce and costly human annotations, or focus exclusively on assessments at the level of individual claims. Misinformation often spreads via interlinked web domains, whose connections evolve over time. Focusing on claims alone ignores these structural and temporal credibility signals evident in the changing web topology. Existing datasets fail to capture these central modalities in web domain credibility prediction: namely, internet topology, temporality and text (webpage) content. To address this gap, we present CrediBench, a dataset containing eights months of web graph data; of which we analyze the three months surrounding the 2024 U.S. federal elections, a time of heightened misinformation propagation online. Each monthly snapshot contains over 40 million nodes, their scraped webpage content, and over 1 billion hyperlink edges. CrediBench supports credibility prediction as both a regression (continuous credibility score) and a binary classification task (credible or not). For classification, we curate a novel binary label set containing 662,575 web domains labelled for boolean credibility, spanning four areas (misinformation, crowd-sourced, malware and phishing). Our empirical experiments support that all task modalities-graph, text and time-contribute significantly to achieving the best performance. In particular, our multi-modal regression model trained on CrediBench outperforms other configurations and existing baselines, decreasing Mean Average Error from 0.162 to 0.107 on the regression task, while the multi-modal classifier improves accuracy from 56% to 85% on the classification one. CrediBench, our proposed web-scale multi-modal dataset, is available on Huggingface for future research.