mlsanityData to 7 Oct 2026

Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs

arXiv 2310.18152 ↗First posted 27 Oct 2023cs.CL cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

Text-attributed graphs (TAGs) are prevalent on the web and research over TAGs such as citation networks, e-commerce networks and social networks has attracted considerable attention in the web community. Recently, large language models (LLMs) have demonstrated exceptional capabilities across a wide range of tasks. However, the existing works focus on harnessing the potential of LLMs solely relying on prompts to convey graph structure information to LLMs, thus suffering from insufficient understanding of the complex structural relationships within TAGs. To address this problem, in this paper we present the Disentangled Graph-Text Learner (DGTL) model, which is able to enhance the reasoning and predicting capabilities of LLMs for TAGs. Our proposed DGTL model incorporates graph structure information through tailored disentangled graph neural network (GNN) layers, enabling LLMs to capture the intricate relationships hidden in text-attributed graphs from multiple structural factors. Furthermore, DGTL operates with frozen pre-trained LLMs, reducing computational costs and allowing much more flexibility in combining with different LLM models. Experimental evaluations demonstrate the effectiveness of the proposed DGTL model on achieving superior or comparable performance over state-of-the-art baselines. Additionally, we also demonstrate that our DGTL model can offer natural language explanations for predictions, thereby significantly enhancing model interpretability.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 27 Oct 2023.

Builds on

27 of 38 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2017Graph Attention Networkscited by 8,854
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2017Inductive Representation Learning on Large Graphscited by 6,284
  9. 2018How Powerful are Graph Neural Networks?cited by 4,355
  10. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  11. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  12. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
15 more
  1. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  2. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  3. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  4. 2019Heterogeneous Graph Attention Networkcited by 732
  5. 2023A Survey on Large Language Models for Recommendationcited by 378
  6. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 271
  7. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  8. 2023S3: Social-network Simulation System with Large Language Model-Empowered Agentscited by 228
  9. 2023Language is All a Graph Needscited by 219
  10. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 209
  11. 2020Graph-Bert: Only Attention is Needed for Learning Graph Representationscited by 176
  12. 2022Learning on Large-scale Text-attributed Graphs via Variational Inferencecited by 153
  13. 2023SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learningcited by 73
  14. 2023Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysiscited by 35
  15. 2023Prompt-based Node Feature Extractor for Few-shot Learning on Text-Attributed Graphscited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 19; 2025: 11; 2026: 3.

36 papers.

Sort:

By area:

  1. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 217
  2. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 190
  3. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 84
  4. 2024A Survey of Large Language Models for Graphscited by 71
  5. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignmentcited by 56
  6. 2024Can LLM Graph Reasoning Generalize beyond Pattern Memorization?cited by 46
  7. 2024XRec: Large Language Models for Explainable Recommendationcited by 34
  8. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  9. 2023Large Language Models as Topological Structure Enhancers for Text-Attributed Graphscited by 27
  10. 2024A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applicationscited by 17
  11. 2024GraphTeam: Facilitating Large Language Model-based Graph Analysis via Multi-Agent Collaborationcited by 12
  12. 2024Learning on Graphs with Large Language Models(LLMs): A Deep Dive into Model Robustnesscited by 11
24 more
  1. 2024Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completioncited by 10
  2. 2024Graph Learning in the Era of LLMs: A Survey from the Perspective of Data, Models, and Taskscited by 10
  3. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  4. 2025Align-GRAG: Anchor and Rationale Guided Dual Alignment for Graph Retrieval-Augmented Generationcited by 9
  5. 2025Model Generalization on Text Attribute Graphs: Principles with Large Language Modelscited by 8
  6. 2024A Unified Taxonomy-Guided Instruction Tuning Framework for Entity Set Expansion and Taxonomy Expansioncited by 7
  7. 2025Towards Graph Foundation Models: A Transferability Perspectivecited by 5
  8. 2025Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applicationscited by 5
  9. 2024How to Make LMs Strong Node Classifiers?cited by 4
  10. 2024Verbalized Graph Representation Learning: A Fully Interpretable Graph Model Based on Large Language Models Throughout the Entire Processcited by 3
  11. 2024A Hierarchical Language Model For Interpretable Graph Reasoningcited by 3
  12. 2024Large Language Models Meet Graph Neural Networks: A Perspective of Graph Miningcited by 3
  13. 2025Integrating Structural and Semantic Signals in Text-Attributed Graphs with BiGTexcited by 3
  14. 2025Adversarial Attacks and Defenses on Graph-aware Large Language Models (LLMs)cited by 3
  15. 2025Scale-Free Graph-Language Modelscited by 2
  16. 2025The Knowledge-Reasoning Dissociation: Fundamental Limitations of LLMs in Clinical Natural Language Inferencecited by 2
  17. 2024Towards Graph Prompt Learning: A Survey and Beyondcited by 1
  18. 2024Can Large Language Models Act as Ensembler for Multi-GNNs?cited by 1
  19. 2026Colorful Talks with Graphs: Human-Interpretable Graph Encodings for Large Language Modelscited by 1
  20. 2025Scalability Matters: Overcoming Challenges in InstructGLM with Similarity-Degree-Based Samplingcited by 0
  21. 2025Efficient Text-Attributed Graph Learning through Selective Annotation and Graph Alignmentcited by 0
  22. 2025Unveiling the Vulnerability of Graph-LLMs: An Interpretable Multi-Dimensional Adversarial Attack on TAGscited by 0
  23. 2026Topology of Reasoning: Retrieved Cell Complex-Augmented Generation for Textual Graph Question Answeringcited by 0
  24. 2026TemporalGraphLLM: Temporal Graph Neural Networks with Large Language Models for Dynamic Text-Attributed Graphscited by 0