mlsanityData to 14 Oct 2026

Deep Ensembles for Graphs with Higher-order Dependencies

arXiv 2205.13988 ↗First posted 27 May 2022cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Graph neural networks (GNNs) continue to achieve state-of-the-art performance on many graph learning tasks, but rely on the assumption that a given graph is a sufficient approximation of the true neighborhood structure. When a system contains higher-order sequential dependencies, we show that the tendency of traditional graph representations to underfit each node's neighborhood causes existing GNNs to generalize poorly. To address this, we propose a novel Deep Graph Ensemble (DGE), which captures neighborhood variance by training an ensemble of GNNs on different neighborhood subspaces of the same node within a higher-order network structure. We show that DGE consistently outperforms existing GNNs on semisupervised and supervised tasks on six real-world data sets with known higher-order dependencies, even under a similar parameter budget. We demonstrate that learning diverse and accurate base classifiers is central to DGE's success, and discuss the implications of these findings for future work on ensembles of GNNs.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 33 of its 66 references that are in the map. First posted on arXiv 27 May 2022.

Builds on

33 of 66 references.

References · 33

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2017Graph Attention Networkscited by 8,923
  3. 2017Inductive Representation Learning on Large Graphscited by 6,320
  4. 2018How Powerful are Graph Neural Networks?cited by 4,393
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  6. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  7. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  8. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  9. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,102
  10. 2018Pitfalls of Graph Neural Network Evaluationcited by 1,007
  11. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 881
  12. 2020Simple and Deep Graph Convolutional Networkscited by 860
21 more
  1. 2021How Attentive are Graph Attention Networks?cited by 835
  2. 2018Link Prediction Based on Graph Neural Networkscited by 832
  3. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  4. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
  5. 2020Higher-Order Explanations of Graph Neural Networks via Relevant Walkscited by 138
  6. 2020Networks beyond pairwise interactions: structure and dynamicscited by 130
  7. 2016Higher-order organization of complex networkscited by 110
  8. 2022Decoupling the Depth and Scope of Graph Neural Networkscited by 94
  9. 2021The physics of higher-order interactions in complex systemscited by 63
  10. 2022Deep Ensembles Work, But Are They Necessary?cited by 63
  11. 2015Evaluating Link Prediction Methodscited by 31
  12. 2020The why, how, and when of representations for complex systemscited by 23
  13. 2013Memory in network flows and its effects on spreading dynamics and community detectioncited by 18
  14. 2021Graph Ensemble Learning over Multiple Dependency Trees for Aspect-level Sentiment Classificationcited by 13
  15. 2020Calendar Graph Neural Networks for Modeling Time Structures in Spatiotemporal User Behaviorscited by 12
  16. 2013Causality-Driven Slow-Down and Speed-Up of Diffusion in Non-Markovian Temporal Networkscited by 10
  17. 2015Representing higher-order dependencies in networkscited by 10
  18. 2015Modeling sequences and temporal networks with dynamic community structurescited by 9
  19. 2019HONEM: Learning Embedding for Higher Order Networkscited by 4
  20. 2019Nonlinearity + Networks: A 2020 Visioncited by 3
  21. 2020Inferring Network Structure From Datacited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 0; 2026: 1.

3 papers.

  1. 2022De Bruijn goes Neural: Causality-Aware Graph Neural Networks for Time Series Data on Dynamic Graphscited by 10
  2. 2026Do Deep Ensembles Actually Capture Uncertainty in Graph Neural Networks?cited by 2
  3. 2023Representing Outcome-driven Higher-order Dependencies in Graphs of Disease Trajectoriescited by 0