mlsanityData to 14 Oct 2026

Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insights

arXiv 2406.10727 ↗First posted 15 Jun 2024cs.LGcited by 44

Not on the map. Why?

Abstract (arXiv)

Given the ubiquity of graph data and its applications in diverse domains, building a Graph Foundation Model (GFM) that can work well across different graphs and tasks with a unified backbone has recently garnered significant interests. A major obstacle to achieving this goal stems from the fact that graphs from different domains often exhibit diverse node features. Inspired by multi-modal models that align different modalities with natural language, the text has recently been adopted to provide a unified feature space for diverse graphs. Despite the great potential of these text-space GFMs, current research in this field is hampered by two problems. First, the absence of a comprehensive benchmark with unified problem settings hinders a clear understanding of the comparative effectiveness and practical value of different text-space GFMs. Second, there is a lack of sufficient datasets to thoroughly explore the methods' full potential and verify their effectiveness across diverse settings. To address these issues, we conduct a comprehensive benchmark providing novel text-space datasets and comprehensive evaluation under unified problem settings. Empirical results provide new insights and inspire future research directions. Our code and data are publicly available from urlhttps://github.com/CurryTang/TSGFM.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Builds on 73 of its 96 references that are in the map. First posted on arXiv 15 Jun 2024.

Builds on

73 of 96 references.

References · 73

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2023Segment Anythingcited by 8,234
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2020Scaling Laws for Neural Language Modelscited by 6,586
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  10. 2023Mistral 7Bcited by 3,164
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  12. 2022Emergent Abilities of Large Language Modelscited by 2,512
38 more
  1. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  2. 2019Simplifying Graph Convolutional Networkscited by 1,477
  3. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,138
  4. 2018Deep Graph Infomaxcited by 1,106
  5. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 976
  6. 2019Graph Neural Networks for Social Recommendationcited by 714
  7. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  8. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 463
  9. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 448
  10. 2023A critical look at the evaluation of GNNs under heterophily: Are we really making progress?cited by 343
  11. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  12. 2021Graph Self-Supervised Learning: A Surveycited by 289
  13. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 274
  14. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 243
  15. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 226
  16. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 223
  17. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 211
  18. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 192
  19. 2021Large-Scale Representation Learning on Graphs via Bootstrappingcited by 186
  20. 2024LLaGA: Large Language and Graph Assistantcited by 181
  21. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 176
  22. 2023GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networkscited by 167
  23. 2022Deep Bidirectional Language-Knowledge Graph Pretrainingcited by 156
  24. 2022Learning on Large-scale Text-attributed Graphs via Variational Inferencecited by 155
  25. 2020Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learningcited by 149
  26. 2021GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graphcited by 143
  27. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 138
  28. 2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 133
  29. 2023Investigating the Catastrophic Forgetting in Multimodal Large Language Modelscited by 119
  30. 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 118
  31. 2023GraphText: Graph Reasoning in Text Spacecited by 111
  32. 2023GraphLLM: Boosting Graph Reasoning Ability of Large Language Modelcited by 107
  33. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 103
  34. 2023PRODIGY: Enabling In-context Learning Over Graphscited by 98
  35. 2020Overcoming Catastrophic Forgetting in Graph Neural Networkscited by 95
  36. 2020Transfer Learning of Graph Neural Networks with Ego-graph Information Maximizationcited by 93
  37. 2023Towards Foundation Models for Knowledge Graph Reasoningcited by 86
  38. 2023Attending to Graph Transformerscited by 83

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 25; 2026: 16.

44 papers.

Sort:

By area:

  1. 2025Position: Graph Learning Will Lose Relevance Due To Poor Benchmarkscited by 53
  2. 2024Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Treescited by 32
  3. 2025LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Modelscited by 22
  4. 2025GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuningcited by 13
  5. 2024One Model for One Graph: A New Perspective for Pretraining with Cross-domain Graphscited by 12
  6. 2025Can TabPFN Compete with GNNs for Node Classification via Graph Tabularization?cited by 10
  7. 2026Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priorscited by 10
  8. 2025Homophily Enhanced Graph Domain Adaptationcited by 9
  9. 2025Model Generalization on Text Attribute Graphs: Principles with Large Language Modelscited by 8
  10. 2025Towards A Universal Graph Structural Encodercited by 7
  11. 2025KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphscited by 6
  12. 2025Graph-KV: Breaking Sequence via Injecting Structural Biases into Large Language Modelscited by 6
32 more
  1. 2025Semantic Refinement with LLMs for Graph Representationscited by 6
  2. 2025View Space: Learning Representation across Arbitrary Graphscited by 5
  3. 2025Unveiling Mode Connectivity in Graph Neural Networkscited by 3
  4. 2025Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphscited by 3
  5. 2025Towards Text-free Graph Foundation Models: Rethinking Multi-Domain Graph Contrastive Learningcited by 3
  6. 2025Text Meets Topology: Rethinking Out-of-distribution Detection in Text-Rich Networkscited by 3
  7. 2026Graph Domain Adaptation via Homophily-Agnostic Reconstructing Structurecited by 3
  8. 2026Modality-free Graph In-context Alignmentcited by 3
  9. 2025GLIP-OOD: Zero-Shot Graph OOD Detection with Graph Foundation Modelcited by 2
  10. 2025Language Models as Messengers: Enhancing Message Passing in Heterophilic Graph Learningcited by 2
  11. 2025A Scalable Pretraining Framework for Link Prediction with Efficient Adaptationcited by 2
  12. 2025Graph is a Natural Regularization: Revisiting Vector Quantization for Graph Representation Learningcited by 2
  13. 2026Node4All: Learning Node Representation Beyond Datasetscited by 2
  14. 2024SaVe-TAG: LLM-based Interpolation for Long-Tailed Text-Attributed Graphscited by 1
  15. 2025Relation-Aware Graph Foundation Modelcited by 1
  16. 2025A Systematic Study of Model Extraction Attacks on Graph Foundation Modelscited by 1
  17. 2026Learning Hierarchical Knowledge in Text-Rich Networks with Taxonomy-Informed Representation Learningcited by 1
  18. 2026Enhancing LLMs for Graph Tasks via Graph-aware LoRA Generationcited by 1
  19. 2026Adaptive Recurrent Message Passing for Test Time Computing on Graphscited by 1
  20. 2025H2GFM: Towards unifying Homogeneity and Heterogeneity on Text-Attributed Graphscited by 0
  21. 2025GSTBench: A Benchmark Study on the Transferability of Graph Self-Supervised Learningcited by 0
  22. 2025Struc-EMB: The Potential of Structure-Aware Encoding in Language Embeddingscited by 0
  23. 2025When LLM Agents Meet Graph Optimization: An Automated Data Quality Improvement Approachcited by 0
  24. 2026On the Safety of Graph Representation Learningcited by 0
  25. 2026S2Aligner: Pair-Efficient and Transferable Pre-Training for Sparse Text-Attributed Graphscited by 0
  26. 2026GraspLLM: Towards Zero-Shot Generalization on Text-Attributed Graphs with LLMscited by 0
  27. 2026LLM Features Can Hurt GNNs: Concatenation Interference on Homophilous Graph Benchmarkscited by 0
  28. 2026OpenRTAG: A Comprehensive Benchmark for Robust Text-Attributed Graph Learning under Data Quality Degradationcited by 0
  29. 2026Semi-Supervised Text-Attributed Graph Distillationcited by 0
  30. 2026Chimaera: A Mixture-of-Graph-Experts Architecture for Cross-Task and Cross-Dataset Graph Learningcited by 0
  31. 2026Preserving Geometric Integrity in Graph Prompting via Measure-Constrained Optimal Transportcited by 0
  32. 2026Graph Memory: Spectral Associative Memory via Dirichlet Energycited by 0