# What Makes Graph Neural Networks Miscalibrated?

arXiv: https://arxiv.org/abs/2210.06391 · HTML: https://mlsanity.com/paper/2210.06391/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 57 references that are in the map.
- First posted on arXiv 12 Oct 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Geom-GCN: Geometric Graph Convolutional Networks](https://mlsanity.com/paper/2002.05287/) (2020)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Ranking](https://mlsanity.com/paper/1707.03815/) (2017)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [A Fair Comparison of Graph Neural Networks for Graph Classification](https://mlsanity.com/paper/1912.09893/) (2019)
- [Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learning](https://mlsanity.com/paper/2003.07329/) (2020)
- [Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches](https://mlsanity.com/paper/1803.04386/) (2018)
- [Calibration of Neural Networks using Splines](https://mlsanity.com/paper/2006.12800/) (2020)
- [Uncertainty Aware Semi-Supervised Learning on Graph Data](https://mlsanity.com/paper/2010.12783/) (2020)
- [Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration](https://mlsanity.com/paper/2109.14285/) (2021)
- [Bayesian graph convolutional neural networks for semi-supervised classification](https://mlsanity.com/paper/1811.11103/) (2018)
- [Bayesian Graph Neural Networks with Adaptive Connection Sampling](https://mlsanity.com/paper/2006.04064/) (2020)
- [Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classification](https://mlsanity.com/paper/2110.14012/) (2021)
- [Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks](https://mlsanity.com/paper/2003.06820/) (2020)

## Built on by (most cited first, 30 of 40)

- [Uncertainty Quantification over Graph with Conformalized Graph Neural Networks](https://mlsanity.com/paper/2305.14535/) (2023)
- [Uncertainty in Graph Neural Networks: A Survey](https://mlsanity.com/paper/2403.07185/) (2024)
- [Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness](https://mlsanity.com/paper/2506.05735/) (2025)
- [Uncertainty for Active Learning on Graphs](https://mlsanity.com/paper/2405.01462/) (2024)
- [SimCalib: Graph Neural Network Calibration based on Similarity between Nodes](https://mlsanity.com/paper/2312.11858/) (2023)
- [Conformalized Link Prediction on Graph Neural Networks](https://mlsanity.com/paper/2406.18763/) (2024)
- [Accurate and Scalable Estimation of Epistemic Uncertainty for Graph Neural Networks](https://mlsanity.com/paper/2401.03350/) (2024)
- [Similarity-Navigated Conformal Prediction for Graph Neural Networks](https://mlsanity.com/paper/2405.14303/) (2024)
- [A Graph Is More Than Its Nodes: Towards Structured Uncertainty-Aware Learning on Graphs](https://mlsanity.com/paper/2210.15575/) (2022)
- [Mitigating Label Noise on Graph via Topological Sample Selection](https://mlsanity.com/paper/2403.01942/) (2024)
- [AdapTable: Test-Time Adaptation for Tabular Data via Shift-Aware Uncertainty Calibrator and Label Distribution Handler](https://mlsanity.com/paper/2407.10784/) (2024)
- [Enhancing Trustworthiness of Graph Neural Networks with Rank-Based Conformal Training](https://mlsanity.com/paper/2501.02767/) (2025)
- [Understanding and Mitigating Under-Confidence in GNNs from the Final Layer](https://mlsanity.com/paper/2505.11335/) (2025)
- [On the Temperature of Bayesian Graph Neural Networks for Conformal Prediction](https://mlsanity.com/paper/2310.11479/) (2023)
- [Uncertainty Quantification on Graph Learning: A Survey](https://mlsanity.com/paper/2404.14642/) (2024)
- [GETS: Ensemble Temperature Scaling for Calibration in Graph Neural Networks](https://mlsanity.com/paper/2410.09570/) (2024)
- [Uncertainty-Aware Graph Structure Learning](https://mlsanity.com/paper/2502.12618/) (2025)
- [Pairwise is Not Enough: Hypergraph Neural Networks for Multi-Agent Pathfinding](https://mlsanity.com/paper/2602.06733/) (2026)
- [In-n-Out: Calibrating Graph Neural Networks for Link Prediction](https://mlsanity.com/paper/2403.04605/) (2024)
- [Conditional Uncertainty Quantification for Tensorized Topological Neural Networks](https://mlsanity.com/paper/2410.15241/) (2024)
- [Multimodal Graph Representation Learning for Robust Surgical Workflow Recognition with Adversarial Feature Disentanglement](https://mlsanity.com/paper/2505.01766/) (2025)
- [BANGS: Game-Theoretic Node Selection for Graph Self-Training](https://mlsanity.com/paper/2410.09348/) (2024)
- [Structural Alignment Improves Graph Test-Time Adaptation](https://mlsanity.com/paper/2502.18334/) (2025)
- [Residual Reweighted Conformal Prediction for Graph Neural Networks](https://mlsanity.com/paper/2506.07854/) (2025)
- [WATS: Calibrating Graph Neural Networks with Wavelet-Aware Temperature Scaling](https://mlsanity.com/paper/2506.23782/) (2025)
- [PREF-Gate: Provenance-Constrained Relational Evidence Fusion with Validation-Gated Selection for Graph Fraud Detection](https://mlsanity.com/paper/2607.11212/) (2026)
- [Conditional Shift-Robust Conformal Prediction for Graph Neural Network](https://mlsanity.com/paper/2405.11968/) (2024)
- [Conformal Prediction: A Theoretical Note and Benchmarking Transductive Node Classification in Graphs](https://mlsanity.com/paper/2409.18332/) (2024)
- [Towards Precise Prediction Uncertainty in GNNs: Refining GNNs with Topology-grouping Strategy](https://mlsanity.com/paper/2412.14223/) (2024)
- [Hierarchical Uncertainty-Aware Graph Neural Network](https://mlsanity.com/paper/2504.19820/) (2025)

## Abstract (arXiv)

Given the importance of getting calibrated predictions and reliable uncertainty estimations, various post-hoc calibration methods have been developed for neural networks on standard multi-class classification tasks. However, these methods are not well suited for calibrating graph neural networks (GNNs), which presents unique challenges such as accounting for the graph structure and the graph-induced correlations between the nodes. In this work, we conduct a systematic study on the calibration qualities of GNN node predictions. In particular, we identify five factors which influence the calibration of GNNs: general under-confident tendency, diversity of nodewise predictive distributions, distance to training nodes, relative confidence level, and neighborhood similarity. Furthermore, based on the insights from this study, we design a novel calibration method named Graph Attention Temperature Scaling (GATS), which is tailored for calibrating graph neural networks. GATS incorporates designs that address all the identified influential factors and produces nodewise temperature scaling using an attention-based architecture. GATS is accuracy-preserving, data-efficient, and expressive at the same time. Our experiments empirically verify the effectiveness of GATS, demonstrating that it can consistently achieve state-of-the-art calibration results on various graph datasets for different GNN backbones.