mlsanityData to 14 Oct 2026

Towards Graph Foundation Models: A Transferability Perspective

arXiv 2503.09363 ↗First posted 12 Mar 2025cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

In recent years, Graph Foundation Models (GFMs) have gained significant attention for their potential to generalize across diverse graph domains and tasks. Some works focus on Domain-Specific GFMs, which are designed to address a variety of tasks within a specific domain, while others aim to create General-Purpose GFMs that extend the capabilities of domain-specific models to multiple domains. Regardless of the type, transferability is crucial for applying GFMs across different domains and tasks. However, achieving strong transferability is a major challenge due to the structural, feature, and distributional variations in graph data. To date, there has been no systematic research examining and analyzing GFMs from the perspective of transferability. To bridge the gap, we present the first comprehensive taxonomy that categorizes and analyzes existing GFMs through the lens of transferability, structuring GFMs around their application scope (domain-specific vs. general-purpose) and their approaches to knowledge acquisition and transfer. We provide a structured perspective on current progress and identify potential pathways for advancing GFM generalization across diverse graph datasets and tasks. We aims to shed light on the current landscape of GFMs and inspire future research directions in GFM development.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 69 of its 98 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

69 of 98 references.

References · 69

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2017Graph Attention Networkscited by 8,923
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  9. 2017Inductive Representation Learning on Large Graphscited by 6,320
  10. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  11. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  12. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
38 more
  1. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  2. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  3. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  4. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  6. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  7. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  8. 2016node2vec: Scalable Feature Learning for Networkscited by 1,918
  9. 2014DeepWalk: Online Learning of Social Representationscited by 1,547
  10. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  11. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  12. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  13. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  14. 2018Deep Graph Infomaxcited by 1,106
  15. 2021Florence: A New Foundation Model for Computer Visioncited by 804
  16. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  17. 2015LINE: Large-scale Information Network Embeddingcited by 582
  18. 2020Graph Neural Networks in Recommender Systems: A Surveycited by 494
  19. 2021Rethinking Graph Transformers with Spectral Attentioncited by 473
  20. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 463
  21. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 448
  22. 2018Deep Learning on Graphs: A Surveycited by 393
  23. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  24. 2021Graph Self-Supervised Learning: A Surveycited by 289
  25. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 274
  26. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 252
  27. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 210
  28. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 192
  29. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  30. 2024LLaGA: Large Language and Graph Assistantcited by 181
  31. 2020Graph-Bert: Only Attention is Needed for Learning Graph Representationscited by 177
  32. 2018Fast Abstractive Summarization with Reinforce-Selected Sentence Rewritingcited by 158
  33. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 138
  34. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
  35. 2023GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learnercited by 132
  36. 2023GraphText: Graph Reasoning in Text Spacecited by 111
  37. 2023PRODIGY: Enabling In-context Learning Over Graphscited by 98
  38. 2024ZeroG: Investigating Cross-dataset Zero-shot Transferability in Graphscited by 85

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuningcited by 13
  2. 2025One Prompt Fits All: Universal Graph Adaptation for Pretrained Modelscited by 11
  3. 2025From Sequence to Structure: Uncovering Substructure Reasoning in Transformerscited by 3
  4. 2026Can Graph Foundation Models Generalize Over Architecture?cited by 0
  5. 2026A Unified Graph Language Model for Multi-Domain Multi-Task Graph Alignment Instruction Tuningcited by 0
  6. 2026AdaLoop: Adaptive-Depth Latent Reasoning for Audio Language Modelscited by 0
  7. 2026SkillFormer: Skill-Decomposed Adaptation for Audio Language Modelscited by 0