# Large Language Models as Topological Structure Enhancers for Text-Attributed Graphs

arXiv: https://arxiv.org/abs/2311.14324 · HTML: https://mlsanity.com/paper/2311.14324/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 60 references that are in the map.
- First posted on arXiv 24 Nov 2023.

## Builds on (most cited first, 30 of 40)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks](https://mlsanity.com/paper/1905.07953/) (2019)
- [Pre-trained Models for Natural Language Processing: A Survey](https://mlsanity.com/paper/2003.08271/) (2020)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [FastText.zip: Compressing text classification models](https://mlsanity.com/paper/1612.03651/) (2016)
- [Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs](https://mlsanity.com/paper/2307.03393/) (2023)
- [GraphGPT: Graph Instruction Tuning for Large Language Models](https://mlsanity.com/paper/2310.13023/) (2023)
- [Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning](https://mlsanity.com/paper/1805.10002/) (2018)
- [GreaseLM: Graph REASoning Enhanced Language Models for Question Answering](https://mlsanity.com/paper/2201.08860/) (2022)
- [Deep Bidirectional Language-Knowledge Graph Pretraining](https://mlsanity.com/paper/2210.09338/) (2022)
- [Learning on Large-scale Text-attributed Graphs via Variational Inference](https://mlsanity.com/paper/2210.14709/) (2022)
- [GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph](https://mlsanity.com/paper/2105.02605/) (2021)
- [Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction](https://mlsanity.com/paper/2111.00064/) (2021)

## Built on by (most cited first, 27 of 27)

- [G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering](https://mlsanity.com/paper/2402.07630/) (2024)
- [Graph Machine Learning in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2404.14928/) (2024)
- [Distilling Large Language Models for Text-Attributed Graph Learning](https://mlsanity.com/paper/2402.12022/) (2024)
- [Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques](https://mlsanity.com/paper/2402.05952/) (2024)
- [Exploring the Potential of Large Language Models for Heterophilic Graphs](https://mlsanity.com/paper/2408.14134/) (2024)
- [PriorDrive: Enhancing Online HD Mapping with Unified Vector Priors](https://mlsanity.com/paper/2409.05352/) (2024)
- [A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective](https://mlsanity.com/paper/2403.16137/) (2024)
- [An Automatic Graph Construction Framework based on Large Language Models for Recommendation](https://mlsanity.com/paper/2412.18241/) (2024)
- [LOGIN: A Large Language Model Consulted Graph Neural Network Training Framework](https://mlsanity.com/paper/2405.13902/) (2024)
- [Attention Mechanisms Perspective: Exploring LLM Processing of Graph-Structured Data](https://mlsanity.com/paper/2505.02130/) (2025)
- [LangTopo: Aligning Language Descriptions of Graphs with Tokenized Topological Modeling](https://mlsanity.com/paper/2406.13250/) (2024)
- [Graph Reasoning with Large Language Models via Pseudo-code Prompting](https://mlsanity.com/paper/2409.17906/) (2024)
- [How to Make LMs Strong Node Classifiers?](https://mlsanity.com/paper/2410.02296/) (2024)
- [Robust Anomaly Detection in Network Traffic: Evaluating Machine Learning Models on CICIDS2017](https://mlsanity.com/paper/2506.19877/) (2025)
- [Enhancing Graph Neural Networks with Limited Labeled Data by Actively Distilling Knowledge from Large Language Models](https://mlsanity.com/paper/2407.13989/) (2024)
- [Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphs](https://mlsanity.com/paper/2505.17599/) (2025)
- [Adversarial Attacks and Defenses on Graph-aware Large Language Models (LLMs)](https://mlsanity.com/paper/2508.04894/) (2025)
- [Molecular Graph Representation Learning Integrating Large Language Models with Domain-specific Small Models](https://mlsanity.com/paper/2408.10124/) (2024)
- [Refining Interactions: Enhancing Anisotropy in Graph Neural Networks with Language Semantics](https://mlsanity.com/paper/2504.01429/) (2025)
- [Hyperbolic Contrastive Learning with Model-augmentation for Knowledge-aware Recommendation](https://mlsanity.com/paper/2505.08157/) (2025)
- [Colorful Talks with Graphs: Human-Interpretable Graph Encodings for Large Language Models](https://mlsanity.com/paper/2602.10386/) (2026)
- [Characterizing the Influence of Topology on Graph Learning Tasks](https://mlsanity.com/paper/2404.07493/) (2024)
- [Efficient Text-Attributed Graph Learning through Selective Annotation and Graph Alignment](https://mlsanity.com/paper/2506.07168/) (2025)
- [Enhanced Pre-training of Graph Neural Networks for Million-Scale Heterogeneous Graphs](https://mlsanity.com/paper/2510.12401/) (2025)
- [Topology of Reasoning: Retrieved Cell Complex-Augmented Generation for Textual Graph Question Answering](https://mlsanity.com/paper/2602.19240/) (2026)
- [Semi-Supervised Text-Attributed Graph Distillation](https://mlsanity.com/paper/2607.20477/) (2026)
- [HGP:An on-device personalized agent memory via hybrid graph storage](https://mlsanity.com/paper/2610.10071/) (2026)

## Abstract (arXiv)

The latest advancements in large language models (LLMs) have revolutionized the field of natural language processing (NLP). Inspired by the success of LLMs in NLP tasks, some recent work has begun investigating the potential of applying LLMs in graph learning tasks. However, most of the existing work focuses on utilizing LLMs as powerful node feature augmenters, leaving employing LLMs to enhance graph topological structures an understudied problem. In this work, we explore how to leverage the information retrieval and text generation capabilities of LLMs to refine/enhance the topological structure of text-attributed graphs (TAGs) under the node classification setting. First, we propose using LLMs to help remove unreliable edges and add reliable ones in the TAG. Specifically, we first let the LLM output the semantic similarity between node attributes through delicate prompt designs, and then perform edge deletion and edge addition based on the similarity. Second, we propose using pseudo-labels generated by the LLM to improve graph topology, that is, we introduce the pseudo-label propagation as a regularization to guide the graph neural network (GNN) in learning proper edge weights. Finally, we incorporate the two aforementioned LLM-based methods for graph topological refinement into the process of GNN training, and perform extensive experiments on four real-world datasets. The experimental results demonstrate the effectiveness of LLM-based graph topology refinement (achieving a 0.15%--2.47% performance gain on public benchmarks).