mlsanityData to 14 Oct 2026

Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees

arXiv 2412.16441 ↗First posted 21 Dec 2024cs.LG cs.AI cs.SIcited by 32

Not on the map. Why?

Abstract (arXiv)

Foundation models are pretrained on large-scale corpora to learn generalizable patterns across domains and tasks -- such as contours, textures, and edges in images, or tokens and sentences in text. In contrast, discovering such generalities in graph-structured data, especially across heterogeneous graph tasks, remains an open challenge. To address this, we propose a novel approach to cross-task generalization in graphs via task-trees, which serve as unified learning instances aligning node-, edge-, and graph-level tasks. We theoretically analyze the stability, transferability, and generalization properties of task-trees, showing that pretraining a graph neural network (GNN) on diverse task-trees with a reconstruction objective induces transferable knowledge. This enables efficient adaptation to downstream tasks with minimal fine-tuning. To validate our framework, we introduce Graph Generality Identifier on Task-Trees (GIT), a graph foundation model that demonstrates strong performance on over 30 graphs across five domains via fine-tuning, in-context learning, and zero-shot generalization. Code and data are available at https://github.com/Zehong-Wang/GIT.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 71 of its 88 references that are in the map. First posted on arXiv 21 Dec 2024.

Builds on

71 of 88 references.

References · 71

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2017Graph Attention Networkscited by 8,923
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2017Inductive Representation Learning on Large Graphscited by 6,320
  7. 2018How Powerful are Graph Neural Networks?cited by 4,393
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  10. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  11. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  12. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 881
38 more
  1. 2021Florence: A New Foundation Model for Computer Visioncited by 804
  2. 2022Galactica: A Large Language Model for Sciencecited by 744
  3. 2020Deep Graph Contrastive Representation Learningcited by 539
  4. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 448
  5. 2023Preference Ranking Optimization for Human Alignmentcited by 285
  6. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 252
  7. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 243
  8. 2020Can Graph Neural Networks Count Substructures?cited by 236
  9. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 226
  10. 2020Generalization and Representational Limits of Graph Neural Networkscited by 218
  11. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 210
  12. 2021Large-Scale Representation Learning on Graphs via Bootstrappingcited by 186
  13. 2021Is Homophily a Necessity for Graph Neural Networks?cited by 182
  14. 2024LLaGA: Large Language and Graph Assistantcited by 181
  15. 2022Revisiting Heterophily For Graph Neural Networkscited by 169
  16. 2023GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networkscited by 167
  17. 2020Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learningcited by 149
  18. 2023All in One: Multi-task Prompting for Graph Neural Networkscited by 148
  19. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 131
  20. 2020Graphon Neural Networks and the Transferability of Graph Neural Networkscited by 118
  21. 2020Self-supervised Learning on Graphs: Deep Insights and New Directioncited by 108
  22. 2023PRODIGY: Enabling In-context Learning Over Graphscited by 98
  23. 2020Transfer Learning of Graph Neural Networks with Ego-graph Information Maximizationcited by 93
  24. 2019Transferability of Spectral Graph Convolutional Neural Networkscited by 92
  25. 2020Graph Prototypical Networks for Few-shot Learning on Attributed Networkscited by 90
  26. 2024Position: Graph Foundation Models are Already Herecited by 86
  27. 2024All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretrainingcited by 86
  28. 2024ZeroG: Investigating Cross-dataset Zero-shot Transferability in Graphscited by 85
  29. 2024GFT: Graph Foundation Model with Transferable Tree Vocabularycited by 85
  30. 2024OpenGraph: Towards Open Graph Foundation Modelscited by 78
  31. 2023GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learningcited by 70
  32. 2024A Comprehensive Survey on Graph Reduction: Sparsification, Coarsening, and Condensationcited by 57
  33. 2024AnyGraph: Graph Foundation Model in the Wildcited by 53
  34. 2024Can we Soft Prompt LLMs for Graph Learning Tasks?cited by 52
  35. 2021Learning Theory Can (Sometimes) Explain Generalisation in Graph Neural Networkscited by 51
  36. 2024Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insightscited by 44
  37. 2022Task-Adaptive Few-shot Node Classificationcited by 43
  38. 2024Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Modelscited by 42

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 23.

32 papers.

Sort:

By area:

  1. 2025Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglementcited by 14
  2. 2026Graph is a Substrate Across Data Modalitiescited by 14
  3. 2025Generative Graph Pattern Machinecited by 11
  4. 2025One Prompt Fits All: Universal Graph Adaptation for Pretrained Modelscited by 11
  5. 2025FedBook: A Unified Federated Graph Foundation Codebook with Intra-domain and Inter-domain Knowledge Modelingcited by 8
  6. 2025Controllable Graph Generation with Diffusion Models via Inference-Time Tree Search Guidancecited by 7
  7. 2025Semantic Refinement with LLMs for Graph Representationscited by 6
  8. 2026Molecular Representations in Implicit Functional Space via Hyper-Networkscited by 5
  9. 2026Multi-Domain Riemannian Graph Gluing for Building Graph Foundation Modelscited by 5
  10. 2026Structure-Centric Graph Foundation Model via Geometric Basescited by 5
  11. 2026Temporal Graph Pattern Machinecited by 3
  12. 2026BHyGNN+: Unsupervised Representation Learning for Heterophilic Hypergraphscited by 2
20 more
  1. 2025Relation-Aware Graph Foundation Modelcited by 1
  2. 2026Generalizing GNNs with Tokenized Mixture of Expertscited by 1
  3. 2026OPBench: A Graph Benchmark to Combat the Opioid Crisiscited by 1
  4. 2026Riemannian Geometry Speaks Louder Than Words: From Graph Foundation Model to Next-Generation Graph Intelligencecited by 1
  5. 2026SCGFM-ART: Amortized Relational Transport for Structure-Centric Graph Foundation Modelscited by 1
  6. 2026PermuFormer: Multi-Task Pretraining for Permutation Representation in Algebraic Combinatoricscited by 1
  7. 2025Two Facets of the Same Optimization Coin: Model Degradation and Representation Collapse in Graph Foundation Modelscited by 0
  8. 2025Enhancing Graph Representations with Neighborhood-Contextualized Message-Passingcited by 0
  9. 2026Out-of-Distribution Generalization in Graph Foundation Modelscited by 0
  10. 2026Mochi: Aligning Pre-training and Inference for Efficient Graph Foundation Models via Meta-Learningcited by 0
  11. 2026On the Safety of Graph Representation Learningcited by 0
  12. 2026Are Common Substructures Transferable? Riemannian Graph Foundation Model with Neural Vector Bundlescited by 0
  13. 2026SPR: Toward a Graph Foundation Model for Transferable Graph Cognition via Spectral Patterns and Relational Geometrycited by 0
  14. 2026Same Graph Cross-Task Transfer in GNNs: Protocols and Predictorscited by 0
  15. 2026MUGEN: Generating Unlearnable Graph Examples for Multiple Learning Taskscited by 0
  16. 2026Training Graph Foundation Models on The Web Graphcited by 0
  17. 2026CalibHyper: Chance-Corrected Relational Hypergraphs for Few-Shot Molecular Property Predictioncited by 0
  18. 2026Message Passing Does More with Less for In-Context Learning on Graphscited by 0
  19. 2026Toward Omni Multimodal Graph Foundation Model: A Topology-Driven Binding Approachcited by 0
  20. 2026More Domains, No Gains? FLO: Fast-Slow Reciprocal Learning for Multi-Domain Graph Pre-trainingcited by 0