mlsanityData to 14 Oct 2026

GFT: Graph Foundation Model with Transferable Tree Vocabulary

arXiv 2411.06070 ↗First posted 9 Nov 2024cs.LG cs.AI cs.SIcited by 85

Not on the map. Why?

Abstract (arXiv)

Inspired by the success of foundation models in applications such as ChatGPT, as graph data has been ubiquitous, one can envision the far-reaching impacts that can be brought by Graph Foundation Models (GFMs) with broader applications in the areas such as scientific research, social network analysis, drug discovery, and e-commerce. Despite the significant progress of pre-trained graph neural networks, there haven't been GFMs that can achieve desired performance on various graph-learning-related tasks. Building GFMs may rely on a vocabulary that encodes transferable patterns shared among different tasks and domains. Unlike image and text, defining such transferable patterns for graphs remains an open question. In this paper, we aim to bridge this gap by rethinking the transferable patterns on graphs as computation trees -- i.e., tree structures derived from the message-passing process. Based on this insight, we propose a cross-task, cross-domain graph foundation model named GFT, short for Graph Foundation model with transferable Tree vocabulary. By treating computation trees as tokens within the transferable vocabulary, GFT improves model generalization and reduces the risk of negative transfer. The theoretical analyses and extensive experimental studies have demonstrated the transferability of computation trees and shown the effectiveness of GFT across diverse tasks and domains in graph learning. The open source code and data are available at https://github.com/Zehong-Wang/GFT.

Key facts. Cited by 85 papers in the map (data through 14 Oct 2026). Builds on 91 of its 116 references that are in the map. First posted on arXiv 9 Nov 2024.

Builds on

91 of 116 references.

References · 91

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2017Graph Attention Networkscited by 8,923
  5. 2017Inductive Representation Learning on Large Graphscited by 6,320
  6. 2017Neural Discrete Representation Learningcited by 4,712
  7. 2018How Powerful are Graph Neural Networks?cited by 4,393
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  9. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  10. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,919
  11. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  12. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
38 more
  1. 2016Variational Graph Auto-Encoderscited by 1,382
  2. 2018Deep Graph Infomaxcited by 1,106
  3. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 881
  4. 2020Geom-GCN: Geometric Graph Convolutional Networkscited by 826
  5. 2021Florence: A New Foundation Model for Computer Visioncited by 804
  6. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  7. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  8. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 652
  9. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  10. 2020Deep Graph Contrastive Representation Learningcited by 539
  11. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  12. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 463
  13. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 448
  14. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
  15. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 329
  16. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 252
  17. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 243
  18. 2020Can Graph Neural Networks Count Substructures?cited by 236
  19. 2018Characterizing and Avoiding Negative Transfercited by 229
  20. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 226
  21. 2020Generalization and Representational Limits of Graph Neural Networkscited by 218
  22. 2017struc2vec: Learning Node Representations from Structural Identitycited by 215
  23. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  24. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 210
  25. 2017Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learningcited by 192
  26. 2021Large-Scale Representation Learning on Graphs via Bootstrappingcited by 186
  27. 2021Is Homophily a Necessity for Graph Neural Networks?cited by 182
  28. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 176
  29. 2023GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networkscited by 167
  30. 2021Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillationcited by 152
  31. 2020Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learningcited by 149
  32. 2023All in One: Multi-task Prompting for Graph Neural Networkscited by 148
  33. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 138
  34. 2020Graphon Neural Networks and the Transferability of Graph Neural Networkscited by 118
  35. 2023GraphText: Graph Reasoning in Text Spacecited by 111
  36. 2023GraphLLM: Boosting Graph Reasoning Ability of Large Language Modelcited by 107
  37. 2023PRODIGY: Enabling In-context Learning Over Graphscited by 98
  38. 2020Transfer Learning of Graph Neural Networks with Ego-graph Information Maximizationcited by 93

50 most cited of 91 All 91 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 30; 2026: 51.

85 papers.

Sort:

By area:

  1. 2025Multi-Domain Graph Foundation Models: Robust Knowledge Transfer via Topology Alignmentcited by 42
  2. 2025GFM-RAG: Graph Foundation Model for Retrieval Augmented Generationcited by 40
  3. 2025Turning Tabular Foundation Models into Graph Foundation Modelscited by 35
  4. 2024Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Treescited by 32
  5. 2025AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answeringcited by 32
  6. 2025LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Modelscited by 22
  7. 2025GraphPFN: A Prior-Data Fitted Graph Foundation Modelcited by 21
  8. 2024Can LLMs Convert Graphs to Text-Attributed Graphs?cited by 19
  9. 2025Edge Prompt Tuning for Graph Neural Networkscited by 18
  10. 2025Beyond Message Passing: Neural Graph Pattern Machinecited by 17
  11. 2024NGQA: A Nutritional Graph Question Answering Benchmark for Personalized Health-aware Nutritional Reasoningcited by 16
  12. 2026EvolveRouter: Co-Evolving Routing and Prompt for Multi-Agent Question Answeringcited by 15
38 more
  1. 2025Towards Effective Federated Graph Foundation Model via Mitigating Knowledge Entanglementcited by 14
  2. 2026Graph is a Substrate Across Data Modalitiescited by 14
  3. 2025GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuningcited by 13
  4. 2025LLM-Empowered Class Imbalanced Graph Prompt Learning for Online Drug Trafficking Detectioncited by 11
  5. 2025Generative Graph Pattern Machinecited by 11
  6. 2025One Prompt Fits All: Universal Graph Adaptation for Pretrained Modelscited by 11
  7. 2026MM-OpenFGL: A Comprehensive Benchmark for Multimodal Federated Graph Learningcited by 11
  8. 2024Training MLPs on Graphs without Supervisioncited by 8
  9. 2025Discrete Tokenization for Multimodal LLMs: A Comprehensive Surveycited by 8
  10. 2025FedBook: A Unified Federated Graph Foundation Codebook with Intra-domain and Inter-domain Knowledge Modelingcited by 8
  11. 2025Towards Graph Foundation Models: A Transferability Perspectivecited by 7
  12. 2025Controllable Graph Generation with Diffusion Models via Inference-Time Tree Search Guidancecited by 7
  13. 2026GLEN-Bench: A Graph-Language based Benchmark for Nutritional Healthcited by 7
  14. 2026Toward Effective Multimodal Graph Foundation Model: A Divide-and-Conquer Based Approachcited by 7
  15. 2026Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimizationcited by 7
  16. 2025Semantic Refinement with LLMs for Graph Representationscited by 6
  17. 2026Molecular Representations in Implicit Functional Space via Hyper-Networkscited by 5
  18. 2026MUG: Meta-path-aware Universal Heterogeneous Graph Pre-Trainingcited by 5
  19. 2026Multi-Domain Riemannian Graph Gluing for Building Graph Foundation Modelscited by 5
  20. 2026W2T: LoRA Weights Already Know What They Can Docited by 5
  21. 2025Towards Foundation Model on Temporal Knowledge Graph Reasoningcited by 3
  22. 2025Graph Prompting for Graph Learning Models: Recent Advances and Future Directionscited by 3
  23. 2025Towards Text-free Graph Foundation Models: Rethinking Multi-Domain Graph Contrastive Learningcited by 3
  24. 2025Towards Effective, Stealthy, and Persistent Backdoor Attacks Targeting Graph Foundation Modelscited by 3
  25. 2026Temporal Graph Pattern Machinecited by 3
  26. 2026Policy4OOD: A Knowledge-Guided World Model for Policy Intervention Simulation against the Opioid Overdose Crisiscited by 3
  27. 2026Evaluating Progress in Graph Foundation Models: A Comprehensive Benchmark and New Insightscited by 3
  28. 2026Modality-free Graph In-context Alignmentcited by 3
  29. 2026Bridging Input Feature Spaces Towards Graph Foundation Modelscited by 3
  30. 2025Toward a Graph Foundation Model: Pre-Training Transformers With Random Walkscited by 2
  31. 2025Graph is a Natural Regularization: Revisiting Vector Quantization for Graph Representation Learningcited by 2
  32. 2025GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learningcited by 2
  33. 2026Rethinking Federated Graph Foundation Models: A Graph-Language Alignment-based Approachcited by 2
  34. 2026BHyGNN+: Unsupervised Representation Learning for Heterophilic Hypergraphscited by 2
  35. 2025Relation-Aware Graph Foundation Modelcited by 1
  36. 2026MAGPrompt: Message-Adaptive Graph Prompt Tuning for Graph Neural Networkscited by 1
  37. 2026Generalizing GNNs with Tokenized Mixture of Expertscited by 1
  38. 2026OPBench: A Graph Benchmark to Combat the Opioid Crisiscited by 1

50 most cited of 85 All 85 (JSON)