# GNNEvaluator: Evaluating GNN Performance On Unseen Graphs Without Labels

arXiv: https://arxiv.org/abs/2310.14586 · HTML: https://mlsanity.com/paper/2310.14586/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 47 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Builds on (most cited first, 30 of 38)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning](https://mlsanity.com/paper/2310.01061/) (2023)
- [SSD: A Unified Framework for Self-Supervised Outlier Detection](https://mlsanity.com/paper/2103.12051/) (2021)
- [Graph Neural Networks for Graphs with Heterophily: A Survey](https://mlsanity.com/paper/2202.07082/) (2022)
- [Handling Distribution Shifts on Graphs: An Invariance Perspective](https://mlsanity.com/paper/2202.02466/) (2022)
- [Leveraging Unlabeled Data to Predict Out-of-Distribution Performance](https://mlsanity.com/paper/2201.04234/) (2022)
- [Assessing Generalization of SGD via Disagreement](https://mlsanity.com/paper/2106.13799/) (2021)
- [Predicting with Confidence on Unseen Distributions](https://mlsanity.com/paper/2107.03315/) (2021)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [Are Labels Always Necessary for Classifier Accuracy Evaluation?](https://mlsanity.com/paper/2007.02915/) (2020)
- [Out-Of-Distribution Generalization on Graphs: A Survey](https://mlsanity.com/paper/2202.07987/) (2022)
- [Trustworthy Graph Neural Networks: Aspects, Methods and Trends](https://mlsanity.com/paper/2205.07424/) (2022)
- [Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Data](https://mlsanity.com/paper/2108.01099/) (2021)
- [Structure-free Graph Condensation: From Large-scale Graphs to Condensed Graph-free Data](https://mlsanity.com/paper/2306.02664/) (2023)
- [Rethinking and Scaling Up Graph Contrastive Learning: An Extremely Efficient Approach with Group Discrimination](https://mlsanity.com/paper/2206.01535/) (2022)
- [What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?](https://mlsanity.com/paper/2106.05961/) (2021)
- [Beyond Smoothing: Unsupervised Graph Representation Learning with Edge Heterophily Discriminating](https://mlsanity.com/paper/2211.14065/) (2022)
- [Mandoline: Model Evaluation under Distribution Shift](https://mlsanity.com/paper/2107.00643/) (2021)
- [Predicting Out-of-Distribution Error with the Projection Norm](https://mlsanity.com/paper/2202.05834/) (2022)
- [GOOD-D: On Unsupervised Graph Out-Of-Distribution Detection](https://mlsanity.com/paper/2211.04208/) (2022)
- [DANE: Domain Adaptive Network Embedding](https://mlsanity.com/paper/1906.00684/) (2019)
- [Learning Strong Graph Neural Networks with Weak Information](https://mlsanity.com/paper/2305.18457/) (2023)
- [Finding the Missing-half: Graph Complementary Learning for Homophily-prone and Heterophily-prone Graphs](https://mlsanity.com/paper/2306.07608/) (2023)

## Built on by (most cited first, 17 of 17)

- [A Label-Free Heterophily-Guided Approach for Unsupervised Graph Fraud Detection](https://mlsanity.com/paper/2502.13308/) (2025)
- [BiMark: Unbiased Multilayer Watermarking for Large Language Models](https://mlsanity.com/paper/2506.21602/) (2025)
- [Graph Learning under Distribution Shifts: A Comprehensive Survey on Domain Adaptation, Out-of-distribution, and Continual Learning](https://mlsanity.com/paper/2402.16374/) (2024)
- [GOODAT: Towards Test-time Graph Out-of-Distribution Detection](https://mlsanity.com/paper/2401.06176/) (2024)
- [Dynamic Graph Unlearning: A General and Efficient Post-Processing Method via Gradient Transformation](https://mlsanity.com/paper/2405.14407/) (2024)
- [Online GNN Evaluation Under Test-time Graph Distribution Shifts](https://mlsanity.com/paper/2403.09953/) (2024)
- [Noise-Resilient Unsupervised Graph Representation Learning via Multi-Hop Feature Quality Estimation](https://mlsanity.com/paper/2407.19944/) (2024)
- [Beyond Generalization: A Survey of Out-Of-Distribution Adaptation on Graphs](https://mlsanity.com/paper/2402.11153/) (2024)
- [DSTSA-GCN: Advancing Skeleton-Based Gesture Recognition with Semantic-Aware Spatio-Temporal Topology Modeling](https://mlsanity.com/paper/2501.12086/) (2025)
- [An Efficient and Effective Evaluator for Text2SQL Models on Unseen and Unlabeled Data](https://mlsanity.com/paper/2603.07841/) (2026)
- [Shapley-Guided Utility Learning for Effective Graph Inference Data Valuation](https://mlsanity.com/paper/2503.18195/) (2025)
- [OASIS: Harnessing Diffusion Adversarial Network for Ocean Salinity Imputation using Sparse Drifter Trajectories](https://mlsanity.com/paper/2508.21570/) (2025)
- [Test-time GNN Model Evaluation on Dynamic Graphs](https://mlsanity.com/paper/2509.23816/) (2025)
- [Generator-based Graph Generation via Heat Diffusion](https://mlsanity.com/paper/2602.03612/) (2026)
- [Learning to Evaluate: Cost-Effective Model Evaluation on Unlabeled Data with Meta-Learning](https://mlsanity.com/paper/2605.23595/) (2026)
- [ALSA: Anchors in Logit Space for Out-of-Distribution Accuracy Estimation](https://mlsanity.com/paper/2508.19613/) (2025)
- [Temporal Memory-Aware Online Test-Time Adaptation on Dynamic Graphs](https://mlsanity.com/paper/2608.27948/) (2026)

## Abstract (arXiv)

Evaluating the performance of graph neural networks (GNNs) is an essential task for practical GNN model deployment and serving, as deployed GNNs face significant performance uncertainty when inferring on unseen and unlabeled test graphs, due to mismatched training-test graph distributions. In this paper, we study a new problem, GNN model evaluation, that aims to assess the performance of a specific GNN model trained on labeled and observed graphs, by precisely estimating its performance (e.g., node classification accuracy) on unseen graphs without labels. Concretely, we propose a two-stage GNN model evaluation framework, including (1) DiscGraph set construction and (2) GNNEvaluator training and inference. The DiscGraph set captures wide-range and diverse graph data distribution discrepancies through a discrepancy measurement function, which exploits the outputs of GNNs related to latent node embeddings and node class predictions. Under the effective training supervision from the DiscGraph set, GNNEvaluator learns to precisely estimate node classification accuracy of the to-be-evaluated GNN model and makes an accurate inference for evaluating GNN model performance. Extensive experiments on real-world unseen and unlabeled test graphs demonstrate the effectiveness of our proposed method for GNN model evaluation.