mlsanityData to 14 Oct 2026

Graph Foundation Models: Concepts, Opportunities and Challenges

arXiv 2310.11829 ↗First posted 18 Oct 2023cs.LGcited by 103

Not on the map. Why?

Abstract (arXiv)

Foundation models have emerged as critical components in a variety of artificial intelligence applications, and showcase significant success in natural language processing and several other domains. Meanwhile, the field of graph machine learning is witnessing a paradigm transition from shallow methods to more sophisticated deep learning approaches. The capabilities of foundation models in generalization and adaptation motivate graph machine learning researchers to discuss the potential of developing a new graph learning paradigm. This paradigm envisions models that are pre-trained on extensive graph data and can be adapted for various graph tasks. Despite this burgeoning interest, there is a noticeable lack of clear definitions and systematic analyses pertaining to this new domain. To this end, this article introduces the concept of Graph Foundation Models (GFMs), and offers an exhaustive explanation of their key characteristics and underlying technologies. We proceed to classify the existing work related to GFMs into three distinct categories, based on their dependence on graph neural networks and large language models. In addition to providing a thorough review of the current state of GFMs, this article also outlooks potential avenues for future research in this rapidly evolving domain.

Key facts. Cited by 103 papers in the map (data through 14 Oct 2026). Builds on 153 of its 233 references that are in the map. First posted on arXiv 18 Oct 2023.

Builds on

153 of 233 references.

References · 153

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2022Training language models to follow instructions with human feedbackcited by 15,971
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  10. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  11. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  12. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
38 more
  1. 2017Graph Attention Networkscited by 8,923
  2. 2017Inductive Representation Learning on Large Graphscited by 6,320
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2018How Powerful are Graph Neural Networks?cited by 4,393
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  10. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  11. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  12. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  13. 2022Emergent Abilities of Large Language Modelscited by 2,512
  14. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  15. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  16. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  17. 2016node2vec: Scalable Feature Learning for Networkscited by 1,918
  18. 2018Graph Neural Networks: A Review of Methods and Applicationscited by 1,576
  19. 2014DeepWalk: Online Learning of Social Representationscited by 1,547
  20. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  21. 2016Variational Graph Auto-Encoderscited by 1,382
  22. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  23. 2018Deep Graph Infomaxcited by 1,106
  24. 2018Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learningcited by 1,102
  25. 2019How Can We Know What Language Models Know?cited by 959
  26. 2016Complex Embeddings for Simple Link Predictioncited by 902
  27. 2018Link Prediction Based on Graph Neural Networkscited by 832
  28. 2019Heterogeneous Graph Attention Networkcited by 734
  29. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 713
  30. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  31. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 680
  32. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  33. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 631
  34. 2020Deep Graph Contrastive Representation Learningcited by 539
  35. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  36. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  37. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  38. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 505

50 most cited of 153 All 153 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 33; 2025: 21; 2026: 42.

103 papers.

Sort:

By area:

  1. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 192
  2. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 172
  3. 2023InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discoverycited by 86
  4. 2024Position: Graph Foundation Models are Already Herecited by 86
  5. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 85
  6. 2024Position: Topological Deep Learning is the New Frontier for Relational Learningcited by 66
  7. 2023Graph Prompt Learning: A Comprehensive Survey and Beyondcited by 61
  8. 2024Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insightscited by 44
  9. 2025GFM-RAG: Graph Foundation Model for Retrieval Augmented Generationcited by 40
  10. 2024GLBench: A Comprehensive Benchmark for Graph with Large Language Modelscited by 36
  11. 2024A Survey of Deep Learning and Foundation Models for Time Series Forecastingcited by 33
  12. 2024The Heterophilic Graph Learning Handbook: Benchmarks, Models, Theoretical Analysis, Applications and Challengescited by 29
38 more
  1. 2024Exploring Large Language Models for Feature Selection: A Data-centric Perspectivecited by 27
  2. 2024MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Predictioncited by 24
  3. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  4. 2024textttMiniMol: A Parameter-Efficient Foundation Model for Molecular Learningcited by 20
  5. 2025GraphMaster: Automated Graph Synthesis via LLM Agents in Data-Limited Environmentscited by 19
  6. 2024A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applicationscited by 17
  7. 2024Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniquescited by 16
  8. 2024Large Language Model-driven Meta-structure Discovery in Heterogeneous Information Networkcited by 16
  9. 2023Fine-tuning Graph Neural Networks by Preserving Graph Generative Patternscited by 13
  10. 2025GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuningcited by 13
  11. 2023Large Knowledge Model: Perspectives and Challengescited by 12
  12. 2024Towards Graph Foundation Models for Personalizationcited by 12
  13. 2024GLaM: Fine-Tuning Large Language Models for Domain Knowledge Graph Alignment via Neighborhood Partitioning and Generative Subgraph Encodingcited by 11
  14. 2025One Prompt Fits All: Universal Graph Adaptation for Pretrained Modelscited by 11
  15. 2024Similarity-based Neighbor Selection for Graph LLMscited by 10
  16. 2024Cross-Modality Program Representation Learning for Electronic Design Automation with High-Level Synthesiscited by 10
  17. 2024Exploring the Potential of Large Language Models for Heterophilic Graphscited by 10
  18. 2024Intruding with Words: Towards Understanding Graph Injection Attacks at the Text Levelcited by 9
  19. 2023Graph Agent: Explicit Reasoning Agent for Graphscited by 8
  20. 2024Enhance Graph Alignment for Large Language Modelscited by 7
  21. 2026Toward Effective Multimodal Graph Foundation Model: A Divide-and-Conquer Based Approachcited by 7
  22. 2026TFMLinker: Universal Link Predictor by Graph In-Context Learning with Tabular Foundation Modelscited by 7
  23. 2026Toward Graph-Tokenizing Large Language Models with Reconstructive Graph Instruction Tuningcited by 7
  24. 2024Towards Foundation Models on Graphs: An Analysis on Cross-Dataset Transfer of Pretrained GNNscited by 5
  25. 2026Multi-Domain Riemannian Graph Gluing for Building Graph Foundation Modelscited by 5
  26. 2026Structure-Centric Graph Foundation Model via Geometric Basescited by 5
  27. 2024GraphFM: A Comprehensive Benchmark for Graph Foundation Modelcited by 4
  28. 2024Towards Graph Foundation Models: A Study on the Generalization of Positional and Structural Encodingscited by 4
  29. 2024Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated Expertscited by 3
  30. 2024Generalist Multimodal AI: A Review of Architectures, Challenges and Opportunitiescited by 3
  31. 2024A Survey on Graph Neural Networks for Remaining Useful Life Prediction: Methodologies, Evaluation and Future Trendscited by 3
  32. 2025Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphscited by 3
  33. 2025FUSAR-KLIP: Towards Multimodal Foundation Models for Remote Sensingcited by 3
  34. 2026Evaluating Progress in Graph Foundation Models: A Comprehensive Benchmark and New Insightscited by 3
  35. 2024NetFlowGen: Leveraging Generative Pre-training for Network Traffic Dynamicscited by 2
  36. 2025GLIP-OOD: Zero-Shot Graph OOD Detection with Graph Foundation Modelcited by 2
  37. 2025Fairness in Augmented Graph Learning: A Surveycited by 2
  38. 2025GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learningcited by 2

50 most cited of 103 All 103 (JSON)