mlsanityData to 7 Oct 2026

Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications

arXiv 2510.21131 ↗First posted 24 Oct 2025cs.CL cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have achieved remarkable success in natural language processing through strong semantic understanding and generation. However, their black-box nature limits structured and multi-hop reasoning. In contrast, Text-Attributed Graphs (TAGs) provide explicit relational structures enriched with textual context, yet often lack semantic depth. Recent research shows that combining LLMs and TAGs yields complementary benefits: enhancing TAG representation learning and improving the reasoning and interpretability of LLMs. This survey provides the first systematic review of LLM--TAG integration from an orchestration perspective. We introduce a novel taxonomy covering two fundamental directions: LLM for TAG, where LLMs enrich graph-based tasks, and TAG for LLM, where structured graphs improve LLM reasoning. We categorize orchestration strategies into sequential, parallel, and multi-module frameworks, and discuss advances in TAG-specific pretraining, prompting, and parameter-efficient fine-tuning. Beyond methodology, we summarize empirical insights, curate available datasets, and highlight diverse applications across recommendation systems, biomedical analysis, and knowledge-intensive question answering. Finally, we outline open challenges and promising research directions, aiming to guide future work at the intersection of language and graph learning.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 164 of its 227 references that are in the map. First posted on arXiv 24 Oct 2025.

Builds on

164 of 227 references.

References · 164

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2024The Llama 3 Herd of Modelscited by 14,192
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  10. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  11. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  12. 2017Graph Attention Networkscited by 8,854
38 more
  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2017Inductive Representation Learning on Large Graphscited by 6,284
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2018How Powerful are Graph Neural Networks?cited by 4,355
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  11. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  12. 2017Billion-scale similarity search with GPUscited by 2,477
  13. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  14. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  15. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  16. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  17. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  18. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  19. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  20. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
  21. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  22. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  23. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  24. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  25. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  26. 2022Galactica: A Large Language Model for Sciencecited by 736
  27. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  28. 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
  29. 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
  30. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  31. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  32. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  33. 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
  34. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  35. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 400
  36. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 332
  37. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  38. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 326

50 most cited of 164 All 164 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Distilling Large Language Models for Text-Attributed Graph Learning
  2. 2024Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs
  3. 2023Talk like a Graph: Encoding Graphs for Large Language Models
  4. 2026Improving LLM Reasoning with Homophily-aware Structural and Semantic Text-Attributed Graph Compression
  5. 2024GUNDAM: Aligning Large Language Models with Graph Understanding
  6. 2026The Graph Language: How Knowledge Graphs Speak to Large Language Models
  7. 2025Actions Speak Louder than Prompts: A Large-Scale Study of LLMs for Graph Inference
  8. 2025Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation