mlsanityData to 14 Oct 2026

Out-of-Distribution Graph Models Merging

arXiv 2506.03674 ↗First posted 4 Jun 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper studies a novel problem of out-of-distribution graph models merging, which aims to construct a generalized model from multiple graph models pre-trained on different domains with distribution discrepancy. This problem is challenging because of the difficulty in learning domain-invariant knowledge implicitly in model parameters and consolidating expertise from potentially heterogeneous GNN backbones. In this work, we propose a graph generation strategy that instantiates the mixture distribution of multiple domains. Then, we merge and fine-tune the pre-trained graph models via a MoE module and a masking mechanism for generalized adaptation. Our framework is architecture-agnostic and can operate without any source/target domain data. Both theoretical analysis and experimental results demonstrate the effectiveness of our approach in addressing the model generalization problem.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 58 of its 69 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

58 of 69 references.

References · 58

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  2. 2017Graph Attention Networkscited by 8,923
  3. 2018How Powerful are Graph Neural Networks?cited by 4,393
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  5. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  6. 2019Invariant Risk Minimizationcited by 1,854
  7. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  8. 2022Editing Models with Task Arithmeticcited by 1,120
  9. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  10. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  11. 2021Merging Models with Fisher-Weighted Averagingcited by 621
  12. 2019Multi-scale Attributed Node Embeddingcited by 548
38 more
  1. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  2. 2023AdaMerging: Adaptive Model Merging for Multi-Task Learningcited by 252
  3. 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 222
  4. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  5. 2023ZipIt! Merging Models from Different Tasks without Trainingcited by 179
  6. 2021Graph Condensation for Graph Neural Networkscited by 150
  7. 2020Invariant Rationalizationcited by 131
  8. 2020Dynamic Model Pruning with Feedbackcited by 125
  9. 2022Fusing finetuned models for better pretrainingcited by 110
  10. 2021Causal Attention for Interpretable and Generalizable Graph Classificationcited by 108
  11. 2020Neural Networks Fail to Learn Periodic Functions and How to Fix Itcited by 100
  12. 2020Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Maskscited by 98
  13. 2022Out-Of-Distribution Generalization on Graphs: A Surveycited by 97
  14. 2021Towards a Theoretical Framework of Out-of-Distribution Generalizationcited by 94
  15. 2020Masking as an Efficient Alternative to Finetuning for Pretrained Language Modelscited by 93
  16. 2024Mixture of A Million Expertscited by 72
  17. 2021Generalizing Graph Neural Networks on Out-Of-Distribution Graphscited by 71
  18. 2022A Survey on Deep Graph Generation: Methods and Applicationscited by 69
  19. 2023Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modelingcited by 64
  20. 2022Graph Condensation via Receptive Field Distribution Matchingcited by 60
  21. 2024A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learningcited by 52
  22. 2021Graph-Free Knowledge Distillation for Graph Neural Networkscited by 46
  23. 2023Does Invariant Graph Learning via Environment Augmentation Learn Invariance?cited by 42
  24. 2020How fine can fine-tuning be? Learning efficient language modelscited by 41
  25. 2023Graph Condensation for Inductive Node Representation Learningcited by 31
  26. 2023Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalizationcited by 29
  27. 2021Learngene: From Open-World to Your Learning Taskcited by 27
  28. 2023Semi-supervised Domain Adaptation in Graph Transfer Learningcited by 27
  29. 2023Environment-Aware Dynamic Graph Learning for Out-of-Distribution Generalizationcited by 26
  30. 2023Graph Invariant Learning with Subgraph Co-mixup for Out-Of-Distribution Generalizationcited by 18
  31. 2024Graph Learning under Distribution Shifts: A Comprehensive Survey on Domain Adaptation, Out-of-distribution, and Continual Learningcited by 18
  32. 2024A Survey of Deep Graph Learning under Distribution Shifts: from Graph Out-of-Distribution Generalization to Adaptationcited by 17
  33. 2022On Balancing Bias and Variance in Unsupervised Multi-Source-Free Domain Adaptationcited by 16
  34. 2024One Model for One Graph: A New Perspective for Pretraining with Cross-domain Graphscited by 12
  35. 2023Graph Structure and Feature Extrapolation for Out-of-Distribution Generalizationcited by 10
  36. 2023Graph Out-of-Distribution Generalization with Controllable Data Augmentationcited by 9
  37. 2024GALA: Graph Diffusion-based Alignment with Jigsaw for Source-free Domain Adaptationcited by 8
  38. 2025GCAL: Adapting Graph Models to Evolving Domain Shiftscited by 8

50 most cited of 58 All 58 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Graph Out-of-Distribution Generalization via Causal Intervention
  2. 2026Generalizing GNNs with Tokenized Mixture of Experts
  3. 2025Self-Adaptive Graph Mixture of Models
  4. 2025Training Diverse Graph Experts for Ensembles: A Systematic Empirical Study
  5. 2025GMoPE:A Prompt-Expert Mixture Framework for Graph Foundation Models
  6. 2026Training-Free Cross-Architecture Merging for Graph Neural Networks
  7. 2025Do Graph Diffusion Models Accurately Capture and Generate Substructure Distributions?
  8. 2023D4Explainer: In-Distribution GNN Explanations via Discrete Denoising Diffusion