mlsanityData to 7 Oct 2026

Learning on Large-scale Text-attributed Graphs via Variational Inference

arXiv 2210.14709 ↗First posted 26 Oct 2022cs.LGcited by 153

Not on the map. Why?

Abstract (arXiv)

This paper studies learning on text-attributed graphs (TAGs), where each node is associated with a text description. An ideal solution for such a problem would be integrating both the text and graph structure information with large language models and graph neural networks (GNNs). However, the problem becomes very challenging when graphs are large due to the high computational complexity brought by training large language models and GNNs together. In this paper, we propose an efficient and effective solution to learning on large text-attributed graphs by fusing graph structure and language learning with a variational Expectation-Maximization (EM) framework, called GLEM. Instead of simultaneously training large language models and GNNs on big graphs, GLEM proposes to alternatively update the two modules in the E-step and M-step. Such a procedure allows training the two modules separately while simultaneously allowing the two modules to interact and mutually enhance each other. Extensive experiments on multiple data sets demonstrate the efficiency and effectiveness of the proposed approach.

Key facts. Cited by 153 papers in the map (data through 7 Oct 2026). Builds on 29 of its 37 references that are in the map. First posted on arXiv 26 Oct 2022.

Builds on

29 of 37 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2017Graph Attention Networkscited by 8,854
  6. 2017Inductive Representation Learning on Large Graphscited by 6,284
  7. 2018How Powerful are Graph Neural Networks?cited by 4,355
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  10. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  11. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  12. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
17 more
  1. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
  2. 2018Link Prediction Based on Graph Neural Networkscited by 829
  3. 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 305
  4. 2019Inductive Relation Prediction by Subgraph Reasoningcited by 237
  5. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 209
  6. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  7. 2021GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graphcited by 143
  8. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 137
  9. 2019Fine-grained Fact Verification with Kernel Graph Attention Networkcited by 105
  10. 2019GMNN: Graph Markov Neural Networkscited by 82
  11. 2022Graph Attention Multi-Layer Perceptroncited by 82
  12. 2020Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networkscited by 77
  13. 2019Text Level Graph Neural Network for Text Classificationcited by 60
  14. 2021TextGNN: Improving Text Encoder via Graph Neural Network in Sponsored Searchcited by 54
  15. 2021Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced trainingcited by 54
  16. 2020Tensor Graph Convolutional Networks for Text Classificationcited by 49
  17. 2021AdsGNN: Behavior-Graph Augmented Relevance Modeling in Sponsored Searchcited by 38

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 27; 2024: 52; 2025: 43; 2026: 31.

153 papers.

Sort:

By area:

  1. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  2. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 239
  3. 2023Language is All a Graph Needscited by 219
  4. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  5. 2023Exphormer: Sparse Transformers for Graphscited by 166
  6. 2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 132
  7. 2024GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoningcited by 128
  8. 2024Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphscited by 116
  9. 2023GraphText: Graph Reasoning in Text Spacecited by 110
  10. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
  11. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 84
  12. 2023SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learningcited by 73
38 more
  1. 2024Can GNN be Good Adapter for LLMs?cited by 71
  2. 2024A Survey of Large Language Models for Graphscited by 71
  3. 2025SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptationcited by 61
  4. 2024GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphscited by 58
  5. 2024Efficient Tuning and Inference for Large Language Models on Textual Graphscited by 54
  6. 2024Can we Soft Prompt LLMs for Graph Learning Tasks?cited by 52
  7. 2025UniGraph2: Learning a Unified Embedding Space to Bind Multimodal Graphscited by 47
  8. 2024Text-space Graph Foundation Models: Comprehensive Benchmarks and New Insightscited by 44
  9. 2023Disentangled Representation Learning with Large Language Models for Text-Attributed Graphscited by 36
  10. 2024GLBench: A Comprehensive Benchmark for Graph with Large Language Modelscited by 36
  11. 2023Continual Learning on Dynamic Graphs via Parameter Isolationcited by 35
  12. 2024ResearchTown: Simulator of Human Research Communitycited by 35
  13. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  14. 2023ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddingscited by 32
  15. 2023IGB: Addressing The Gaps In Labeling, Features, Heterogeneity, and Size of Public Graph Datasets for Deep Learning Researchcited by 29
  16. 2023Train Your Own GNN Teacher: Graph-Aware Distillation on Textual Graphscited by 29
  17. 2023GRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphscited by 28
  18. 2023Multimodal Graph Learning for Generative Taskscited by 27
  19. 2023Large Language Models as Topological Structure Enhancers for Text-Attributed Graphscited by 27
  20. 2025When Do LLMs Help With Node Classification? A Comprehensive Analysiscited by 21
  21. 2025LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Modelscited by 21
  22. 2024Distilling Large Language Models for Text-Attributed Graph Learningcited by 20
  23. 2024When Graph meets Multimodal: Benchmarking and Meditating on Multimodal Attributed Graphs Learningcited by 20
  24. 2024A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applicationscited by 17
  25. 2024Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniquescited by 16
  26. 2024Bridging Local Details and Global Context in Text-Attributed Graphscited by 15
  27. 2024UniGLM: Training One Unified Language Model for Text-Attributed Graph Embeddingcited by 14
  28. 2024Revisiting Random Walks for Learning on Graphscited by 13
  29. 2024Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision?cited by 13
  30. 2023TouchUp-G: Improving Feature Representation through Graph-Centric Finetuningcited by 12
  31. 2023TransGNN: Harnessing the Collaborative Power of Transformers and Graph Neural Networks for Recommender Systemscited by 11
  32. 2024Learning on Graphs with Large Language Models(LLMs): A Deep Dive into Model Robustnesscited by 11
  33. 2024Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphscited by 11
  34. 2024Graph Learning in the Era of LLMs: A Survey from the Perspective of Data, Models, and Taskscited by 10
  35. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  36. 2024TEG-DB: A Comprehensive Dataset and Benchmark of Textual-Edge Graphscited by 9
  37. 2025Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Modelscited by 9
  38. 2025Align-GRAG: Anchor and Rationale Guided Dual Alignment for Graph Retrieval-Augmented Generationcited by 9

50 most cited of 153 All 153 (JSON)