mlsanityData to 7 Oct 2026

Uncertainty in GNN Learning Evaluations: A Comparison Between Measures for Quantifying Randomness in GNN Community Detection

arXiv 2312.09015 ↗First posted 14 Dec 2023cs.LG cs.SI physics.soc-phcited by 3

Not on the map. Why?

Abstract (arXiv)

(1) The enhanced capability of Graph Neural Networks (GNNs) in unsupervised community detection of clustered nodes is attributed to their capacity to encode both the connectivity and feature information spaces of graphs. The identification of latent communities holds practical significance in various domains, from social networks to genomics. Current real-world performance benchmarks are perplexing due to the multitude of decisions influencing GNN evaluations for this task. (2) Three metrics are compared to assess the consistency of algorithm rankings in the presence of randomness. The consistency and quality of performance between the results under a hyperparameter optimisation with the default hyperparameters is evaluated. (3) The results compare hyperparameter optimisation with default hyperparameters, revealing a significant performance loss when neglecting hyperparameter investigation. A comparison of metrics indicates that ties in ranks can substantially alter the quantification of randomness. (4) Ensuring adherence to the same evaluation criteria may result in notable differences in the reported performance of methods for this task. The W Randomness coefficient, based on the Wasserstein distance, is identified as providing the most robust assessment of randomness.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 30 of its 53 references that are in the map. First posted on arXiv 14 Dec 2023.

Builds on

30 of 53 references.

References · 30

  1. 2016Variational Graph Auto-Encoderscited by 1,373
  2. 2018Deep Graph Infomaxcited by 1,101
  3. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  4. 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 718
  5. 2020Benchmarking Graph Neural Networkscited by 672
  6. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  7. 2020Deep Graph Contrastive Representation Learningcited by 535
  8. 2016Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filteringcited by 517
  9. 2018From Louvain to Leiden: guaranteeing well-connected communitiescited by 379
  10. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 328
  11. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
  12. 2019A Fair Comparison of Graph Neural Networks for Graph Classificationcited by 244
18 more
  1. 2020On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practicecited by 181
  2. 2020Graph Clustering with Graph Neural Networkscited by 163
  3. 2019Attributed Graph Clustering: A Deep Attentional Embedding Approachcited by 149
  4. 2019Fake News Detection on Social Media using Geometric Deep Learningcited by 115
  5. 2022Towards Unsupervised Deep Graph Structure Learningcited by 112
  6. 2021A Comprehensive Survey on Community Detection with Deep Learningcited by 63
  7. 2022GraphWorld: Fake Graphs Bring Real Insights for GNNscited by 56
  8. 2021A Survey of Community Detection Approaches: From Statistical Modeling to Deep Learningcited by 55
  9. 2014Community Detection in Networks with Node Attributescited by 44
  10. 2020Deep Learning for Community Detection: Progress, Challenges and Opportunitiescited by 42
  11. 2019N2D: (Not Too) Deep Clustering via Clustering the Local Manifold of an Autoencoded Embeddingcited by 30
  12. 2021Self-supervised Graph Neural Networks without explicit negative samplingcited by 25
  13. 2019Community detection in node-attributed social networks: a surveycited by 24
  14. 2022VGAER: Graph Neural Network Reconstruction based Community Detectioncited by 14
  15. 2021A Systematic Comparison Study on Hyperparameter Optimisation of Graph Neural Networks for Molecular Property Predictioncited by 9
  16. 2022NCAGC: A Neighborhood Contrast Framework for Attributed Graph Clusteringcited by 5
  17. 2015Detecting Community Structures in Hi-C Genomic Datacited by 4
  18. 2023Uncertainty in GNN Learning Evaluations: The Importance of a Consistent Benchmark for Community Detectioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2025Community detection robustness of graph neural networkscited by 1
  2. 2026Conditional Neural Bayes Ratio Estimation for Experimental Design Optimisationcited by 1
  3. 2024Unsupervised Optimisation of GNNs for Node Clusteringcited by 0