# The Impact of Data Characteristics on GNN Evaluation for Detecting Fake News

arXiv: https://arxiv.org/abs/2512.06638 · HTML: https://mlsanity.com/paper/2512.06638/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 7 Dec 2025.

## Builds on (most cited first, 13 of 13)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018)
- [Fake News Detection on Social Media using Geometric Deep Learning](https://mlsanity.com/paper/1902.06673/) (2019)
- [User Preference-aware Fake News Detection](https://mlsanity.com/paper/2104.12259/) (2021)
- [MuMiN: A Large-Scale Multilingual Multimodal Fact-Checked Misinformation Social Network Dataset](https://mlsanity.com/paper/2202.11684/) (2022)
- [Graph Neural Networks with Continual Learning for Fake News Detection from Social Media](https://mlsanity.com/paper/2007.03316/) (2020)
- [GAMC: An Unsupervised Method for Fake News Detection using Graph Autoencoder with Masking](https://mlsanity.com/paper/2312.05739/) (2023)
- [DECOR: Degree-Corrected Social Graph Refinement for Fake News Detection](https://mlsanity.com/paper/2307.00077/) (2023)
- [Nothing Stands Alone: Relational Fake News Detection with Hypergraph Neural Networks](https://mlsanity.com/paper/2212.12621/) (2022)
- [Modelling Social Context for Fake News Detection: A Graph Neural Network Based Approach](https://mlsanity.com/paper/2207.13500/) (2022)

## Abstract (arXiv)

Graph neural networks (GNNs) are widely used for the detection of fake news by modeling the content and propagation structure of news articles on social media. We show that two of the most commonly used benchmark data sets - GossipCop and PolitiFact - are poorly suited to evaluating the utility of models that use propagation structure. Specifically, these data sets exhibit shallow, ego-like graph topologies that provide little or no ability to differentiate among modeling methods. We systematically benchmark five GNN architectures against a structure-agnostic multilayer perceptron (MLP) that uses the same node features. We show that MLPs match or closely trail the performance of GNNs, with performance gaps often within 1-2% and overlapping confidence intervals. To isolate the contribution of structure in these datasets, we conduct controlled experiments where node features are shuffled or edge structures randomized. We find that performance collapses under feature shuffling but remains stable under edge randomization. This suggests that structure plays a negligible role in these benchmarks. Structural analysis further reveals that over 75% of nodes are only one hop from the root, exhibiting minimal structural diversity. In contrast, on synthetic datasets where node features are noisy and structure is informative, GNNs significantly outperform MLPs. These findings provide strong evidence that widely used benchmarks do not meaningfully test the utility of modeling structural features, and they motivate the development of datasets with richer, more diverse graph topologies.