mlsanityData to 7 Oct 2026

GOOD: A Graph Out-of-Distribution Benchmark

arXiv 2206.08452 ↗First posted 16 Jun 2022cs.LG cs.AIcited by 140

Not on the map. Why?

Abstract (arXiv)

Out-of-distribution (OOD) learning deals with scenarios in which training and test data follow different distributions. Although general OOD problems have been intensively studied in machine learning, graph OOD is only an emerging area of research. Currently, there lacks a systematic benchmark tailored to graph OOD method evaluation. In this work, we aim at developing an OOD benchmark, known as GOOD, for graphs specifically. We explicitly make distinctions between covariate and concept shifts and design data splits that accurately reflect different shifts. We consider both graph and node prediction tasks as there are key differences in designing shifts. Overall, GOOD contains 11 datasets with 17 domain selections. When combined with covariate, concept, and no shifts, we obtain 51 different splits. We provide performance results on 10 commonly used baseline methods with 10 random runs. This results in 510 dataset-model combinations in total. Our results show significant performance gaps between in-distribution and OOD settings. Our results also shed light on different performance trends between covariate and concept shifts by different methods. Our GOOD benchmark is a growing project and expects to expand in both quantity and variety of resources as the area develops. The GOOD benchmark can be accessed via https://github.com/divelab/GOOD/.

Key facts. Cited by 140 papers in the map (data through 7 Oct 2026). Builds on 42 of its 63 references that are in the map. First posted on arXiv 16 Jun 2022.

Builds on

42 of 63 references.

References · 42

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2018How Powerful are Graph Neural Networks?cited by 4,355
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  7. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  8. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  9. 2019Invariant Risk Minimizationcited by 1,832
  10. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  11. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  12. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
30 more
  1. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  2. 2020In Search of Lost Domain Generalizationcited by 1,042
  3. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  4. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  5. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  6. 2019GNNExplainer: Generating Explanations for Graph Neural Networkscited by 735
  7. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 647
  8. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  9. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  10. 2019Graph U-Netscited by 489
  11. 2016Geometric deep learning on graphs and manifolds using mixture model CNNscited by 468
  12. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  13. 2020Explainability in Graph Neural Networks: A Taxonomic Surveycited by 383
  14. 2017Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Rankingcited by 332
  15. 2017Grammar Variational Autoencodercited by 283
  16. 2020The Risks of Invariant Risk Minimizationcited by 262
  17. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  18. 2022Discovering Invariant Rationales for Graph Neural Networkscited by 186
  19. 2021Spherical Message Passing for 3D Graph Networkscited by 166
  20. 2022Handling Distribution Shifts on Graphs: An Invariance Perspectivecited by 163
  21. 2021Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Datacited by 86
  22. 2022NICO++: Towards Better Benchmarking for Domain Generalizationcited by 86
  23. 2021Size-Invariant Graph Representations for Graph Classification Extrapolationscited by 82
  24. 2021OOD-GNN: Out-of-Distribution Generalized Graph Neural Networkcited by 78
  25. 2021DIG: A Turnkey Library for Diving into Graph Deep Learning Researchcited by 76
  26. 2021Generalizing Graph Neural Networks on Out-Of-Distribution Graphscited by 69
  27. 2022DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotationscited by 54
  28. 2020Topology-Aware Graph Pooling Networkscited by 44
  29. 2022OOD Link Prediction Generalization Capabilities of Message-Passing GNNs in Larger Test Graphscited by 38
  30. 2020Deep Learning of High-Order Interactions for Protein Interface Predictioncited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 32; 2024: 46; 2025: 30; 2026: 26.

140 papers.

Sort:

By area:

  1. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  2. 2024OpenGraph: Towards Open Graph Foundation Modelscited by 75
  3. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  4. 2022Empowering Graph Representation Learning with Test-Time Graph Transformationcited by 66
  5. 2022Learning Hierarchical Protein Representations via Complete 3D Graph Networkscited by 54
  6. 2024A Survey of Graph Neural Networks in Real world: Imbalance, Noise, Privacy and OOD Challengescited by 54
  7. 2023Structural Re-weighting Improves Graph Domain Adaptationcited by 45
  8. 2023Does Invariant Graph Learning via Environment Augmentation Learn Invariance?cited by 42
  9. 2023Demystifying Structural Disparity in Graph Neural Networks: Can One Size Fit All?cited by 41
  10. 2024The Clever Hans Mirage: A Comprehensive Survey on Spurious Correlations in Machine Learningcited by 39
  11. 2022Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shiftcited by 38
  12. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
38 more
  1. 2023Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalizationcited by 29
  2. 2024Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learningcited by 28
  3. 2023Environment-Aware Dynamic Graph Learning for Out-of-Distribution Generalizationcited by 26
  4. 2024Optimizing OOD Detection in Molecular Graphs: A Novel Approach with Diffusion Modelscited by 21
  5. 2022FlowX: Towards Explainable Graph Neural Networks via Message Flowscited by 20
  6. 2024Uncertainty in Graph Neural Networks: A Surveycited by 20
  7. 2024GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Datacited by 20
  8. 2023Learning Invariant Molecular Representation in Latent Discrete Spacecited by 19
  9. 2023GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Expertscited by 19
  10. 2024Adapting Multi-modal Large Language Model to Concept Drift From Pre-training Onwardscited by 19
  11. 2024Active Test-Time Adaptation: Theoretical Analyses and An Algorithmcited by 18
  12. 2024Equivariance via Minimal Frame Averaging for More Symmetries and Efficiencycited by 18
  13. 2024Unifying Unsupervised Graph-Level Anomaly Detection and Out-of-Distribution Detection: A Benchmarkcited by 18
  14. 2024GOODAT: Towards Test-time Graph Out-of-Distribution Detectioncited by 17
  15. 2024Structure-based out-of-distribution (OOD) materials property prediction: a benchmark studycited by 17
  16. 2024Subgraph Pooling: Tackling Negative Transfer on Graphscited by 17
  17. 2024A Survey of Deep Graph Learning under Distribution Shifts: from Graph Out-of-Distribution Generalization to Adaptationcited by 16
  18. 2025GOLD: Graph Out-of-Distribution Detection via Implicit Adversarial Latent Generationcited by 15
  19. 2023Explaining and Adapting Graph Conditional Shiftcited by 14
  20. 2024Reconsidering Faithfulness in Regular, Self-Explainable and Domain Invariant GNNscited by 14
  21. 2023Conformal Drug Property Prediction with Density Estimation under Covariate Shiftcited by 13
  22. 2024Investigating Out-of-Distribution Generalization of GNNs: An Architecture Perspectivecited by 13
  23. 2024How Interpretable Are Interpretable Graph Neural Networks?cited by 13
  24. 2024Safety in Graph Machine Learning: Threats and Safeguardscited by 11
  25. 2023Predicting the Silent Majority on Graphs: Knowledge Transferable Graph Neural Networkcited by 10
  26. 2023Graph Structure and Feature Extrapolation for Out-of-Distribution Generalizationcited by 10
  27. 2023Bridged-GNN: Knowledge Bridge Learning for Effective Knowledge Transfercited by 10
  28. 2023Lo-Hi: Practical ML Drug Discovery Benchmarkcited by 10
  29. 2023Identifying Semantic Component for Robust Molecular Property Predictioncited by 10
  30. 2024Learning Invariant Representations of Graph Neural Networks via Cluster Generalizationcited by 10
  31. 2025Beyond Topological Self-Explainable GNNs: A Formal Explainability Perspectivecited by 10
  32. 2024Can Modifying Data Address Graph Domain Adaptation?cited by 9
  33. 2024Beyond Efficiency: Molecular Data Pruning for Enhanced Generalizationcited by 9
  34. 2025Raising the Bar in Graph OOD Generalization: Invariant Learning Beyond Explicit Environment Modelingcited by 9
  35. 2023Instructor-inspired Machine Learning for Robust Molecular Property Predictioncited by 8
  36. 2024Accurate and Scalable Estimation of Epistemic Uncertainty for Graph Neural Networkscited by 8
  37. 2023Evaluating Robustness and Uncertainty of Graph Models Under Structural Distributional Shiftscited by 7
  38. 2023MARIO: Model Agnostic Recipe for Improving OOD Generalization of Graph Contrastive Learningcited by 7

50 most cited of 140 All 140 (JSON)