mlsanityData to 7 Oct 2026

Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques

arXiv 2402.05952 ↗First posted 4 Feb 2024cs.LG cs.AI cs.CLcited by 16

Not on the map. Why?

Abstract (arXiv)

The integration of Large Language Models (LLMs) with Graph Representation Learning (GRL) marks a significant evolution in analyzing complex data structures. This collaboration harnesses the sophisticated linguistic capabilities of LLMs to improve the contextual understanding and adaptability of graph models, thereby broadening the scope and potential of GRL. Despite a growing body of research dedicated to integrating LLMs into the graph domain, a comprehensive review that deeply analyzes the core components and operations within these models is notably lacking. Our survey fills this gap by proposing a novel taxonomy that breaks down these models into primary components and operation techniques from a novel technical perspective. We further dissect recent literature into two primary components including knowledge extractors and organizers, and two operation techniques including integration and training stratigies, shedding light on effective model design and training strategies. Additionally, we identify and explore potential future research avenues in this nascent yet underexplored field, proposing paths for continued progress.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 48 of its 55 references that are in the map. First posted on arXiv 4 Feb 2024.

Builds on

48 of 55 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Graph Attention Networkscited by 8,854
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  7. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  10. 2023A Survey of Large Language Modelscited by 2,537
  11. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  12. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
36 more
  1. 2014DeepWalk: Online Learning of Social Representationscited by 1,543
  2. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  3. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  4. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  5. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  6. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  7. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 302
  8. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  9. 2017A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applicationscited by 260
  10. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  11. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 239
  12. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 223
  13. 2023Language is All a Graph Needscited by 219
  14. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  15. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  16. 2023GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networkscited by 166
  17. 2022Multi-modal Molecule Structure-text Model for Text-based Retrieval and Editingcited by 165
  18. 2022Learning on Large-scale Text-attributed Graphs via Variational Inferencecited by 153
  19. 2022A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Languagecited by 129
  20. 2023GraphText: Graph Reasoning in Text Spacecited by 110
  21. 2022Transformer for Graphs: An Overview from Architecture Perspectivecited by 107
  22. 2023Graph Foundation Models: Concepts, Opportunities and Challengescited by 101
  23. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 84
  24. 2023SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learningcited by 73
  25. 2023Label-free Node Classification on Graphs with Large Language Models (LLMS)cited by 72
  26. 2023Patton: Language Model Pretraining on Text-Rich Networkscited by 49
  27. 2023Graph-Aware Language Model Pre-Training on a Large Graph Corpus Can Help Multiple Graph Applicationscited by 33
  28. 2023ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddingscited by 32
  29. 2023Train Your Own GNN Teacher: Graph-Aware Distillation on Textual Graphscited by 29
  30. 2023Large Language Models as Topological Structure Enhancers for Text-Attributed Graphscited by 27
  31. 2023Beyond Text: A Deep Dive into Large Language Models' Ability on Understanding Graph Datacited by 25
  32. 2023Prompt-based Node Feature Extractor for Few-shot Learning on Text-Attributed Graphscited by 23
  33. 2023Leveraging Large Language Models for Node Generation in Few-Shot Learning on Text-Attributed Graphscited by 23
  34. 2020Graph-based Modeling of Online Communities for Fake News Detectioncited by 21
  35. 2023Pretraining Language Models with Text-Attributed Heterogeneous Graphscited by 15
  36. 2023Prompt Tuning on Graph-augmented Low-resource Text Classificationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 11; 2026: 1.

16 papers.

Sort:

By area:

  1. 2024Graph Retrieval-Augmented Generation: A Surveycited by 320
  2. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  3. 2025Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Surveycited by 18
  4. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  5. 2024SimGRAG: Leveraging Similar Subgraphs for Knowledge Graphs Driven Retrieval-Augmented Generationcited by 6
  6. 2025DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillationcited by 5
  7. 2025Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applicationscited by 5
  8. 2025LLMTM: Benchmarking and Optimizing LLMs for Temporal Motif Analysis in Dynamic Graphscited by 4
  9. 2025Court of LLMs: Evidence-Augmented Generation via Multi-LLM Collaboration for Text-Attributed Graph Anomaly Detectioncited by 3
  10. 2024Subgraph Retrieval Enhanced by Graph-Text Alignment for Commonsense Question Answeringcited by 2
  11. 2025KG-BiLM: Knowledge Graph Embedding via Bidirectional Language Modelscited by 1
  12. 2025Multilevel Analysis of Cryptocurrency News using RAG Approach with Fine-Tuned Mistral Large Language Modelcited by 1
4 more
  1. 2025Multi-dimensional Data Analysis and Applications Basing on LLM Agents and Knowledge Graph Interactionscited by 1
  2. 2025Evaluating Effects of Augmented SELFIES for Molecular Understanding Using QK-LSTMcited by 0
  3. 2025Dual Refinement Cycle Learning: Unsupervised Text Classification of Mamba and Community Detection on Text Attributed Graphcited by 0
  4. 2026AdaSTORM: Scaling LLM Reasoning on Dynamic Graphs via Adaptive Spatio-Temporal Multi-Agent Collaborationcited by 0