mlsanityData to 7 Oct 2026

Can Graph Foundation Models Generalize Over Architecture?

arXiv 2603.22984 ↗First posted 24 Mar 2026cs.LG cs.AI cs.SIcited by 0

Not on the map. Why?

Abstract (arXiv)

Graph foundation models (GFMs) have recently attracted interest due to the promise of graph neural network (GNN) architectures that generalize zero-shot across graphs of arbitrary scales, feature dimensions, and domains. While existing work has demonstrated this ability empirically across diverse real-world benchmarks, these tasks share a crucial hidden limitation: they admit a narrow set of effective GNN architectures. In particular, current domain-agnostic GFMs rely on fixed architectural backbones, implicitly assuming that a single message-passing regime suffices across tasks. In this paper, we argue that architecture adaptivity is a necessary requirement for true GFMs. We show that existing approaches are non-robust to task-dependent architectural attributes and, as a case study, use range as a minimal and measurable axis along which this limitation becomes explicit. With theoretical analysis and controlled synthetic experiments, we demonstrate that fixed-backbone GFMs provably under-reach on tasks whose architectural requirements differ from those seen at training time. To address this issue, we introduce a framework that adapts effective GNN architecture at inference time by discovering and mixing task-specific linear graph operators, enabling zero-shot generalization across tasks with heterogeneous architectural requirements, without retraining. We validate our approach on arbitrary-range synthetic tasks and a suite of real-world benchmarks, demonstrating improved performance and robustness over existing domain-agnostic GFMs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 47 of its 58 references that are in the map. First posted on arXiv 24 Mar 2026.

Builds on

47 of 58 references.

References · 47

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2017Graph Attention Networkscited by 8,854
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  7. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  8. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  9. 2017Deep Setscited by 1,394
  10. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
  11. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  12. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
35 more
  1. 2020A Generalization of Transformer Networks to Graphscited by 653
  2. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
  3. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 592
  4. 2019Multi-scale Attributed Node Embeddingcited by 543
  5. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  6. 2017Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Rankingcited by 332
  7. 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 332
  8. 2020SIGN: Scalable Inception Graph Neural Networkscited by 283
  9. 2020Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networkscited by 253
  10. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 223
  11. 2021Weisfeiler and Lehman Go Cellular: CW Networkscited by 218
  12. 2017struc2vec: Learning Node Representations from Structural Identitycited by 214
  13. 2023On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topologycited by 163
  14. 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 112
  15. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
  16. 2023Where Did the Gap Go? Reassessing the Long-Range Graph Benchmarkcited by 83
  17. 2023Towards Foundation Models for Knowledge Graph Reasoningcited by 83
  18. 2023DRew: Dynamically Rewired Message Passing with Delaycited by 81
  19. 2024Efficient Tuning and Inference for Large Language Models on Textual Graphscited by 54
  20. 2024AnyGraph: Graph Foundation Model in the Wildcited by 51
  21. 2019On Universal Equivariant Set Networkscited by 45
  22. 2025On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learningcited by 42
  23. 2020PANE: scalable and effective attributed network embeddingcited by 23
  24. 2025GraphPFN: A Prior-Data Fitted Graph Foundation Modelcited by 20
  25. 2025Towards Quantifying Long-Range Interactions in Graph Machine Learning: a Large Graph Dataset and a Measurementcited by 18
  26. 2025Oversmoothing, Oversquashing, Heterophily, Long-Range, and more: Demystifying Common Beliefs in Graph Machine Learningcited by 18
  27. 2025Equivariance Everywhere All At Once: A Recipe for Graph Foundation Modelscited by 17
  28. 2020Neural Architecture Search in Graph Neural Networkscited by 15
  29. 2025Bringing Graphs to the Table: Zero-shot Node Classification via Tabular Foundation Modelscited by 12
  30. 2021Adaptive Gaussian Processes on Graphs via Spectral Graph Waveletscited by 11
  31. 2023TransWorldNG: Traffic Simulation via Foundation Modelcited by 9
  32. 2025On Measuring Long-Range Interactions in Graph Neural Networkscited by 9
  33. 2025Can You Hear Me Now? A Benchmark for Long-Range Graph Propagationcited by 8
  34. 2025Towards Graph Foundation Models: A Transferability Perspectivecited by 5
  35. 2025Graph Foundation Models for Recommendation: A Comprehensive Surveycited by 3

Built on by

Not cited yet.