mlsanityData to 7 Oct 2026

GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Experts

arXiv 2312.04693 ↗First posted 7 Dec 2023cs.LGcited by 19

On the map: Mixture-of-experts → MoE for time series and graphs

Abstract (arXiv)

Graph data are inherently complex and heterogeneous, leading to a high natural diversity of distributional shifts. However, it remains unclear how to build machine learning architectures that generalize to the complex distributional shifts naturally occurring in the real world. Here, we develop GraphMETRO, a Graph Neural Network architecture that models natural diversity and captures complex distributional shifts. GraphMETRO employs a Mixture-of-Experts (MoE) architecture with a gating model and multiple expert models, where each expert model targets a specific distributional shift to produce a referential representation w.r.t. a reference model, and the gating model identifies shift components. Additionally, we design a novel objective that aligns the representations from different expert models to ensure reliable optimization. GraphMETRO achieves state-of-the-art results on four datasets from the GOOD benchmark, which is comprised of complex and natural real-world distribution shifts, improving by 67% and 4.2% on the WebKB and Twitch datasets. Code and data are available at https://github.com/Wuyxin/GraphMETRO.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts. Builds on 66 of its 87 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

66 of 87 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modeling6 Apr

Other references · 65

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2017Inductive Representation Learning on Large Graphscited by 6,284
  4. 2018How Powerful are Graph Neural Networks?cited by 4,355
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  8. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  9. 2019Invariant Risk Minimizationcited by 1,832
  10. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  11. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  12. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
38 more
  1. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  2. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  3. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  4. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  5. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
  6. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  7. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  8. 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 718
  9. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  10. 2020Benchmarking Graph Neural Networkscited by 672
  11. 2020Long-tail learning via logit adjustmentcited by 605
  12. 2020Explainability in Graph Neural Networks: A Taxonomic Surveycited by 383
  13. 2020MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embeddingcited by 304
  14. 2020Graph Random Neural Network for Semi-Supervised Learning on Graphscited by 213
  15. 2020Data Augmentation for Graph Neural Networkscited by 200
  16. 2022Discovering Invariant Rationales for Graph Neural Networkscited by 186
  17. 2022Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanismcited by 171
  18. 2019Understanding Attention and Generalization in Graph Neural Networkscited by 169
  19. 2022Handling Distribution Shifts on Graphs: An Invariance Perspectivecited by 163
  20. 2022A Review of Sparse Expert Models in Deep Learningcited by 147
  21. 2022G-Mixup: Graph Data Augmentation for Graph Classificationcited by 145
  22. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  23. 2022GOOD: A Graph Out-of-Distribution Benchmarkcited by 140
  24. 2022Data Augmentation for Deep Graph Learning: A Surveycited by 132
  25. 2021Generative Causal Explanations for Graph Neural Networkscited by 124
  26. 2020From Local Structures to Size Generalization in Graph Neural Networkscited by 119
  27. 2020Characteristic Functions on Graphs: Birds of a Feather, from Statistical Descriptors to Parametric Modelscited by 116
  28. 2021Causal Attention for Interpretable and Generalizable Graph Classificationcited by 108
  29. 2022Out-Of-Distribution Generalization on Graphs: A Surveycited by 97
  30. 2021Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training Datacited by 86
  31. 2022Sparse Mixture-of-Experts are Domain Generalizable Learnerscited by 85
  32. 2021Size-Invariant Graph Representations for Graph Classification Extrapolationscited by 82
  33. 2020Factorizable Graph Convolutional Networkscited by 79
  34. 2021OOD-GNN: Out-of-Distribution Generalized Graph Neural Networkcited by 78
  35. 2022Graph Rationalization with Environment-based Augmentationscited by 70
  36. 2021Subgroup Generalization and Fairness of Graph Neural Networkscited by 69
  37. 2021Generalizing Graph Neural Networks on Out-Of-Distribution Graphscited by 69
  38. 2022Empowering Graph Representation Learning with Test-Time Graph Transformationcited by 66

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 9; 2026: 5.

19 papers.

Sort:

By area:

  1. 2025GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuningcited by 13
  2. 2025A Survey of Large Language Models for Text-Guided Molecular Discovery: from Molecule Generation to Optimizationcited by 12
  3. 2024Safety in Graph Machine Learning: Threats and Safeguardscited by 11
  4. 2024Revisiting, Benchmarking and Understanding Unsupervised Graph Domain Adaptationcited by 10
  5. 2024Hierarchical Mixture of Experts: Generalizable Learning for High-Level Synthesiscited by 9
  6. 2024DA-MoE: Addressing Depth-Sensitivity in Graph-Level Analysis through Mixture of Expertscited by 7
  7. 2024Efficient Task Transfer for HLS DSEcited by 4
  8. 2025Graph Few-Shot Learning via Adaptive Spectrum Experts and Cross-Set Distribution Calibrationcited by 3
  9. 2025Can Molecular Foundation Models Know What They Don't Know? A Simple Remedy with Preference Optimizationcited by 2
  10. 2025Leveraging Manifold Embeddings for Enhanced Graph Transformer Representations and Learningcited by 1
  11. 2026Context-aware Graph Causality Inference for Few-Shot Molecular Property Predictioncited by 1
  12. 2026Topology-Aware Multiscale Mixture of Experts for Efficient Molecular Property Predictioncited by 1
7 more
  1. 2026Generalizing GNNs with Tokenized Mixture of Expertscited by 1
  2. 2025Backdoor or Manipulation? Graph Mixture of Experts Can Defend Against Various Graph Adversarial Attackscited by 0
  3. 2025Training Diverse Graph Experts for Ensembles: A Systematic Empirical Studycited by 0
  4. 2025IDOL: Meeting Diverse Distribution Shifts with Prior Physics for Tropical Cyclone Multi-Task Estimationcited by 0
  5. 2025Self-Adaptive Graph Mixture of Modelscited by 0
  6. 2026MalMoE: Mixture-of-Experts Enhanced Encrypted Malicious Traffic Detection Under Graph Driftcited by 0
  7. 2026MoEMambaMIL: Structure-Aware Selective State Space Modeling for Whole-Slide Image Analysiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Out-of-Distribution Graph Models Merging
  2. 2025Adaptive Graph Mixture of Residual Experts: Unsupervised Learning on Diverse Graphs with Heterogeneous Specialization
  3. 2024Learning Divergence Fields for Shift-Robust Graph Representations
  4. 2025GMoPE:A Prompt-Expert Mixture Framework for Graph Foundation Models
  5. 2024Graph Out-of-Distribution Generalization via Causal Intervention
  6. 2024AnyGraph: Graph Foundation Model in the Wild
  7. 2024MLDGG: Meta-Learning for Domain Generalization on Graphs
  8. 2024GDDA: Semantic OOD Detection on Graphs under Covariate Shift via Score-Based Diffusion Models