mlsanityData to 7 Oct 2026

Generalizing GNNs with Tokenized Mixture of Experts

arXiv 2602.09258 ↗First posted 9 Feb 2026cs.LGcited by 1

On the map: Mixture-of-experts → MoE for time series and graphs

Abstract (arXiv)

Deployed graph neural networks (GNNs) are frozen at deployment yet must fit clean data, generalize under distribution shifts, and remain stable to perturbations. We show that static inference induces a fundamental tradeoff: improving stability requires reducing reliance on shift-sensitive features, leaving an irreducible worst-case generalization floor. Instance-conditional routing can break this ceiling, but is fragile because shifts can mislead routing and perturbations can make routing fluctuate. We capture these effects via two decompositions separating coverage vs selection, and base sensitivity vs fluctuation amplification. Based on these insights, we propose STEM-GNN, a pretrain-then-finetune framework with a mixture-of-experts encoder for diverse computation paths, a vector-quantized token interface to stabilize encoder-to-head signals, and a Lipschitz-regularized head to bound output amplification. Across nine node, link, and graph benchmarks, STEM-GNN achieves a stronger three-way balance, improving robustness to degree/homophily shifts and to feature/edge corruptions while remaining competitive on clean graphs.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts. Builds on 56 of its 86 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

56 of 86 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Experts7 Dec

Other references · 55

  1. 2017Graph Attention Networkscited by 8,854
  2. 2017Inductive Representation Learning on Large Graphscited by 6,284
  3. 2017Neural Discrete Representation Learningcited by 4,650
  4. 2018How Powerful are Graph Neural Networks?cited by 4,355
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  7. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  8. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  9. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  10. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  11. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  12. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
38 more
  1. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  2. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  3. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  4. 2018Deep Graph Infomaxcited by 1,101
  5. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  6. 2016Geometric deep learning: going beyond Euclidean datacited by 1,042
  7. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  8. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  9. 2020Simple and Deep Graph Convolutional Networkscited by 857
  10. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  11. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  12. 2018FastGCN: Fast Learning with Graph Convolutional Networks via Importance Samplingcited by 583
  13. 2020Deep Graph Contrastive Representation Learningcited by 535
  14. 2019DeepGCNs: Can GCNs Go as Deep as CNNs?cited by 527
  15. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  16. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 461
  17. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  18. 2018Adversarial Attacks on Neural Networks for Graph Datacited by 395
  19. 2020Graph Structure Learning for Robust Graph Neural Networkscited by 368
  20. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 239
  21. 2020Graph Random Neural Network for Semi-Supervised Learning on Graphscited by 213
  22. 2020GNNGuard: Defending Graph Neural Networks against Adversarial Attackscited by 195
  23. 2021Large-Scale Representation Learning on Graphs via Bootstrappingcited by 184
  24. 2015Conditional Computation in Neural Networks for faster modelscited by 166
  25. 2018GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphscited by 164
  26. 2022GOOD: A Graph Out-of-Distribution Benchmarkcited by 140
  27. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 137
  28. 2022Out-Of-Distribution Generalization on Graphs: A Surveycited by 97
  29. 2024GFT: Graph Foundation Model with Transferable Tree Vocabularycited by 82
  30. 2022Heterogeneous Graph Masked Autoencoderscited by 47
  31. 2024LiGNN: Graph Neural Networks at LinkedIncited by 31
  32. 2024Graph Out-of-Distribution Generalization via Causal Interventioncited by 30
  33. 2024Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Treescited by 30
  34. 2023Some Fundamental Aspects about Lipschitz Continuity of Neural Networkscited by 25
  35. 2022Self-Supervised Graph Structure Refinement for Graph Neural Networkscited by 20
  36. 2024Subgraph Pooling: Tackling Negative Transfer on Graphscited by 17
  37. 2025Beyond Message Passing: Neural Graph Pattern Machinecited by 17
  38. 2026Graph is a Substrate Across Data Modalitiescited by 13

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026On the Safety of Graph Representation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025GMoPE:A Prompt-Expert Mixture Framework for Graph Foundation Models
  2. 2025Training Diverse Graph Experts for Ensembles: A Systematic Empirical Study
  3. 2024Mixture of Experts Meets Decoupled Message Passing: Towards General and Adaptive Node Classification
  4. 2025Adaptive Graph Mixture of Residual Experts: Unsupervised Learning on Diverse Graphs with Heterogeneous Specialization
  5. 2025GNN-MoE: Context-Aware Patch Routing using GNNs for Parameter-Efficient Domain Generalization
  6. 2025Out-of-Distribution Graph Models Merging
  7. 2025Are LLMs Better GNN Helpers? Rethinking Robust Graph Learning under Deficiencies with Iterative Refinement
  8. 2026Training-Free Cross-Architecture Merging for Graph Neural Networks