Can Graph Foundation Models Generalize Over Architecture?
Not on the map. Why?
Abstract (arXiv)
Graph foundation models (GFMs) have recently attracted interest due to the promise of graph neural network (GNN) architectures that generalize zero-shot across graphs of arbitrary scales, feature dimensions, and domains. While existing work has demonstrated this ability empirically across diverse real-world benchmarks, these tasks share a crucial hidden limitation: they admit a narrow set of effective GNN architectures. In particular, current domain-agnostic GFMs rely on fixed architectural backbones, implicitly assuming that a single message-passing regime suffices across tasks. In this paper, we argue that architecture adaptivity is a necessary requirement for true GFMs. We show that existing approaches are non-robust to task-dependent architectural attributes and, as a case study, use range as a minimal and measurable axis along which this limitation becomes explicit. With theoretical analysis and controlled synthetic experiments, we demonstrate that fixed-backbone GFMs provably under-reach on tasks whose architectural requirements differ from those seen at training time. To address this issue, we introduce a framework that adapts effective GNN architecture at inference time by discovering and mixing task-specific linear graph operators, enabling zero-shot generalization across tasks with heterogeneous architectural requirements, without retraining. We validate our approach on arbitrary-range synthetic tasks and a suite of real-world benchmarks, demonstrating improved performance and robustness over existing domain-agnostic GFMs.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 47 of its 58 references that are in the map. First posted on arXiv 24 Mar 2026.
Builds on
47 of 58 references.
References · 47
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2017Graph Attention Networkscited by 8,854
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
- 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
- 2017Deep Setscited by 1,394
- 2018Pitfalls of Graph Neural Network Evaluationcited by 994
- 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
- 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 821
35 more
- 2020A Generalization of Transformer Networks to Graphscited by 653
- 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 624
- 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 592
- 2019Multi-scale Attributed Node Embeddingcited by 543
- 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
- 2017Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Rankingcited by 332
- 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 332
- 2020SIGN: Scalable Inception Graph Neural Networkscited by 283
- 2020Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networkscited by 253
- 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 223
- 2021Weisfeiler and Lehman Go Cellular: CW Networkscited by 218
- 2017struc2vec: Learning Node Representations from Structural Identitycited by 214
- 2023On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topologycited by 163
- 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 112
- 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
- 2023Where Did the Gap Go? Reassessing the Long-Range Graph Benchmarkcited by 83
- 2023Towards Foundation Models for Knowledge Graph Reasoningcited by 83
- 2023DRew: Dynamically Rewired Message Passing with Delaycited by 81
- 2024Efficient Tuning and Inference for Large Language Models on Textual Graphscited by 54
- 2024AnyGraph: Graph Foundation Model in the Wildcited by 51
- 2019On Universal Equivariant Set Networkscited by 45
- 2025On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learningcited by 42
- 2020PANE: scalable and effective attributed network embeddingcited by 23
- 2025GraphPFN: A Prior-Data Fitted Graph Foundation Modelcited by 20
- 2025Towards Quantifying Long-Range Interactions in Graph Machine Learning: a Large Graph Dataset and a Measurementcited by 18
- 2025Oversmoothing, Oversquashing, Heterophily, Long-Range, and more: Demystifying Common Beliefs in Graph Machine Learningcited by 18
- 2025Equivariance Everywhere All At Once: A Recipe for Graph Foundation Modelscited by 17
- 2020Neural Architecture Search in Graph Neural Networkscited by 15
- 2025Bringing Graphs to the Table: Zero-shot Node Classification via Tabular Foundation Modelscited by 12
- 2021Adaptive Gaussian Processes on Graphs via Spectral Graph Waveletscited by 11
- 2023TransWorldNG: Traffic Simulation via Foundation Modelcited by 9
- 2025On Measuring Long-Range Interactions in Graph Neural Networkscited by 9
- 2025Can You Hear Me Now? A Benchmark for Long-Range Graph Propagationcited by 8
- 2025Towards Graph Foundation Models: A Transferability Perspectivecited by 5
- 2025Graph Foundation Models for Recommendation: A Comprehensive Surveycited by 3
Built on by
Not cited yet.