mlsanityData to 14 Oct 2026

Position: Graph Foundation Models are Already Here

arXiv 2402.02216 ↗First posted 3 Feb 2024cs.LGcited by 86

Not on the map. Why?

Abstract (arXiv)

Graph Foundation Models (GFMs) are emerging as a significant research topic in the graph domain, aiming to develop graph models trained on extensive and diverse data to enhance their applicability across various tasks and domains. Developing GFMs presents unique challenges over traditional Graph Neural Networks (GNNs), which are typically trained from scratch for specific tasks on particular datasets. The primary challenge in constructing GFMs lies in effectively leveraging vast and diverse graph data to achieve positive transfer. Drawing inspiration from existing foundation models in the CV and NLP domains, we propose a novel perspective for the GFM development by advocating for a ``graph vocabulary'', in which the basic transferable units underlying graphs encode the invariance on graphs. We ground the graph vocabulary construction from essential aspects including network analysis, expressiveness, and stability. Such a vocabulary perspective can potentially advance the future GFM design in line with the neural scaling laws. All relevant resources with GFM design can be found here.

Key facts. Cited by 86 papers in the map (data through 14 Oct 2026). Builds on 128 of its 194 references that are in the map. First posted on arXiv 3 Feb 2024.

Builds on

128 of 194 references.

References · 128

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2017Graph Attention Networkscited by 8,923
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2017Neural Discrete Representation Learningcited by 4,712
  8. 2018How Powerful are Graph Neural Networks?cited by 4,393
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  11. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  12. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
38 more
  1. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  2. 2016Variational Graph Auto-Encoderscited by 1,382
  3. 2020Graph Contrastive Learning with Augmentationscited by 1,208
  4. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,138
  5. 2018Deep Graph Infomaxcited by 1,106
  6. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 993
  7. 2019Strategies for Pre-training Graph Neural Networkscited by 971
  8. 2018Link Prediction Based on Graph Neural Networkscited by 832
  9. 2022Galactica: A Large Language Model for Sciencecited by 744
  10. 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 728
  11. 2020Contrastive Multi-View Representation Learning on Graphscited by 652
  12. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  13. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  14. 2020Adaptive Universal Generalized PageRank Graph Neural Networkcited by 595
  15. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 520
  16. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  17. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  18. 2021Understanding over-squashing and bottlenecks on graphs via curvaturecited by 478
  19. 2021Rethinking Graph Transformers with Spectral Attentioncited by 473
  20. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 448
  21. 2019Composition-based Multi-Relational Graph Convolutional Networkscited by 396
  22. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 337
  23. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  24. 2021Large Scale Learning on Non-Homophilous Graphs: New Benchmarks and Strong Simple Methodscited by 311
  25. 2022Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equationscited by 283
  26. 2021Neural Bellman-Ford Networks: A General Graph Neural Network Framework for Link Predictioncited by 275
  27. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 274
  28. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 252
  29. 2020A Note on Over-Smoothing for Graph Neural Networkscited by 249
  30. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 243
  31. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 226
  32. 2023Language is All a Graph Needscited by 220
  33. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  34. 2023NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classificationcited by 210
  35. 2020Hierarchical Generation of Molecular Graphs using Structural Motifscited by 208
  36. 2019Position-aware Graph Neural Networkscited by 203
  37. 2021Identity-aware Graph Neural Networkscited by 198
  38. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 192

50 most cited of 128 All 128 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 20; 2025: 37; 2026: 29.

86 papers.

Sort:

By area:

  1. 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
  2. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 172
  3. 2025Position: Graph Learning Will Lose Relevance Due To Poor Benchmarkscited by 53
  4. 2025GFM-RAG: Graph Foundation Model for Retrieval Augmented Generationcited by 40
  5. 2025RiemannGFM: Learning a Graph Foundation Model from Riemannian Geometrycited by 35
  6. 2024Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Treescited by 32
  7. 2024Foundation Models for the Electric Power Gridcited by 28
  8. 2025How Expressive are Knowledge Graph Foundation Models?cited by 23
  9. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  10. 2025LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Modelscited by 22
  11. 2024GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Datacited by 20
  12. 2024Learning Graph Quantized Tokenizerscited by 19
38 more
  1. 2025GraphMaster: Automated Graph Synthesis via LLM Agents in Data-Limited Environmentscited by 19
  2. 2024A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applicationscited by 17
  3. 2025Equivariance Everywhere All At Once: A Recipe for Graph Foundation Modelscited by 17
  4. 2025Bringing Graphs to the Table: Zero-shot Node Classification via Tabular Foundation Modelscited by 16
  5. 2024GraphFM: A generalist graph transformer that learns transferable representations across diverse domainscited by 14
  6. 2025A Survey of Graph Transformers: Architectures, Theories and Applicationscited by 14
  7. 2024LangGFM: A Large Language Model Alone Can be a Powerful Graph Foundation Modelcited by 13
  8. 2025GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuningcited by 13
  9. 2024Exploring the Potential of Large Language Models for Heterophilic Graphscited by 10
  10. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  11. 2024Scalable Training of Trustworthy and Energy-Efficient Predictive Graph Foundation Models for Atomistic Materials Modeling: A Case Study with HydraGNNcited by 8
  12. 2025SEMMA: A Semantic Aware Knowledge Graph Foundation Modelcited by 8
  13. 2025HYPER: A Foundation Model for Inductive Link Prediction with Knowledge Hypergraphscited by 7
  14. 2025Position: Foundation Models for Tabular Data within Systemic Contexts Need Groundingcited by 6
  15. 2024Multi-Scale Heterogeneous Text-Attributed Graph Datasets From Diverse Domainscited by 5
  16. 2024Towards Foundation Models on Graphs: An Analysis on Cross-Dataset Transfer of Pretrained GNNscited by 5
  17. 2025Invariant Link Selector for Spatial-Temporal Out-of-Distribution Problemcited by 5
  18. 2026Billion-Scale Graph Foundation Modelscited by 5
  19. 2024Uncertainty Quantification on Graph Learning: A Surveycited by 4
  20. 2024Towards Graph Foundation Models: A Study on the Generalization of Positional and Structural Encodingscited by 4
  21. 2025Graph Foundation Models for Recommendation: A Comprehensive Surveycited by 4
  22. 2025DELPHYNE: A Pre-Trained Model for General and Financial Time Seriescited by 4
  23. 2026RAG-GFM: Overcoming In-Memory Bottlenecks in Graph Foundation Models via Retrieval-Augmented Generationcited by 4
  24. 2026OpenRFM: Dissecting Relational In-Context Learningcited by 4
  25. 2024A Structure-Aware Framework for Learning Device Placements on Computation Graphscited by 3
  26. 2025Unveiling Mode Connectivity in Graph Neural Networkscited by 3
  27. 2025Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphscited by 3
  28. 2025Towards Effective, Stealthy, and Persistent Backdoor Attacks Targeting Graph Foundation Modelscited by 3
  29. 2026Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Surveycited by 3
  30. 2026Evaluating Progress in Graph Foundation Models: A Comprehensive Benchmark and New Insightscited by 3
  31. 2026Bridging Input Feature Spaces Towards Graph Foundation Modelscited by 3
  32. 2026The Post-GCN Decade Revisited: Curvature-Stratified Evaluation of Relational Learningcited by 3
  33. 2024MiNT: Multi-Network Training for Transfer Learning on Temporal Graphscited by 2
  34. 2024A Cognac Shot To Forget Bad Memories: Corrective Unlearning for Graph Neural Networkscited by 2
  35. 2025Toward a Graph Foundation Model: Pre-Training Transformers With Random Walkscited by 2
  36. 2025GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learningcited by 2
  37. 2026Relatron: Automating Relational Machine Learning over Relational Databasescited by 2
  38. 2025Deep Graph Learning will stall without Network Sciencecited by 1

50 most cited of 86 All 86 (JSON)