Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) have achieved remarkable success in natural language processing through strong semantic understanding and generation. However, their black-box nature limits structured and multi-hop reasoning. In contrast, Text-Attributed Graphs (TAGs) provide explicit relational structures enriched with textual context, yet often lack semantic depth. Recent research shows that combining LLMs and TAGs yields complementary benefits: enhancing TAG representation learning and improving the reasoning and interpretability of LLMs. This survey provides the first systematic review of LLM--TAG integration from an orchestration perspective. We introduce a novel taxonomy covering two fundamental directions: LLM for TAG, where LLMs enrich graph-based tasks, and TAG for LLM, where structured graphs improve LLM reasoning. We categorize orchestration strategies into sequential, parallel, and multi-module frameworks, and discuss advances in TAG-specific pretraining, prompting, and parameter-efficient fine-tuning. Beyond methodology, we summarize empirical insights, curate available datasets, and highlight diverse applications across recommendation systems, biomedical analysis, and knowledge-intensive question answering. Finally, we outline open challenges and promising research directions, aiming to guide future work at the intersection of language and graph learning.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 164 of its 227 references that are in the map. First posted on arXiv 24 Oct 2025.
Builds on
164 of 227 references.
References · 164
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2017Graph Attention Networkscited by 8,854
38 more
- 2025Qwen3 Technical Reportcited by 8,195
- 2017Inductive Representation Learning on Large Graphscited by 6,284
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2018How Powerful are Graph Neural Networks?cited by 4,355
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2017Billion-scale similarity search with GPUscited by 2,477
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
- 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
- 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
- 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,404
- 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
- 2020Graph Contrastive Learning with Augmentationscited by 1,203
- 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
- 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
- 2022Galactica: A Large Language Model for Sciencecited by 736
- 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
- 2022Recipe for a General, Powerful, Scalable Graph Transformercited by 702
- 2019DropEdge: Towards Deep Graph Convolutional Networks on Node Classificationcited by 679
- 2023Baichuan 2: Open Large-scale Language Modelscited by 664
- 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
- 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
- 2019Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological Viewcited by 544
- 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
- 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 400
- 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 332
- 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
- 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 326
50 most cited of 164 All 164 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 4.
5 papers.
- 2025Dual Refinement Cycle Learning: Unsupervised Text Classification of Mamba and Community Detection on Text Attributed Graphcited by 0
- 2026Both Topology and Text Matter: Revisiting LLM-guided Out-of-Distribution Detection on Text-attributed Graphscited by 0
- 2026Node-to-Neighborhood Semantic Consistency: Text-Topology Alignment for TAGs Anomaly Detectioncited by 0
- 2026Semi-Supervised Text-Attributed Graph Distillationcited by 0
- 2026LLMs for Social Network Modeling: From Network Generation to Dynamic Processescited by 0
- 2026LLMs for Social Network Modeling: From Network Generation to Dynamic Processes7 Sep 2026
- 2026Node-to-Neighborhood Semantic Consistency: Text-Topology Alignment for TAGs Anomaly Detection29 Jun 2026
- 2026Semi-Supervised Text-Attributed Graph Distillation27 May 2026
- 2026Both Topology and Text Matter: Revisiting LLM-guided Out-of-Distribution Detection on Text-attributed Graphs12 Feb 2026
- 2025Dual Refinement Cycle Learning: Unsupervised Text Classification of Mamba and Community Detection on Text Attributed Graph8 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2024Distilling Large Language Models for Text-Attributed Graph Learning
- 2024Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs
- 2023Talk like a Graph: Encoding Graphs for Large Language Models
- 2026Improving LLM Reasoning with Homophily-aware Structural and Semantic Text-Attributed Graph Compression
- 2024GUNDAM: Aligning Large Language Models with Graph Understanding
- 2026The Graph Language: How Knowledge Graphs Speak to Large Language Models
- 2025Actions Speak Louder than Prompts: A Large-Scale Study of LLMs for Graph Inference
- 2025Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation