# Can LLMs Convert Graphs to Text-Attributed Graphs?

arXiv: https://arxiv.org/abs/2412.10136 · HTML: https://mlsanity.com/paper/2412.10136/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 43 references that are in the map.
- First posted on arXiv 13 Dec 2024.

## Builds on (most cited first, 25 of 25)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Graph Neural Networks with Learnable Structural and Positional Representations](https://mlsanity.com/paper/2110.07875/) (2021)
- [Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs](https://mlsanity.com/paper/2307.03393/) (2023)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024)
- [One for All: Towards Training One Graph Model for All Classification Tasks](https://mlsanity.com/paper/2310.00149/) (2023)
- [struc2vec: Learning Node Representations from Structural Identity](https://mlsanity.com/paper/1704.03165/) (2017)
- [GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking](https://mlsanity.com/paper/2305.15066/) (2023)
- [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024)
- [Does Synthetic Data Generation of LLMs Help Clinical Text Mining?](https://mlsanity.com/paper/2303.04360/) (2023)
- [GFT: Graph Foundation Model with Transferable Tree Vocabulary](https://mlsanity.com/paper/2411.06070/) (2024)
- [Graph Transfer Learning via Adversarial Domain Adaptation with Graph Convolution](https://mlsanity.com/paper/1909.01541/) (2019)
- [Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models](https://mlsanity.com/paper/2405.13934/) (2024)
- [MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initialization](https://mlsanity.com/paper/2210.00102/) (2022)
- [Subgraph Pooling: Tackling Negative Transfer on Graphs](https://mlsanity.com/paper/2402.08907/) (2024)
- [Diet-ODIN: A Novel Framework for Opioid Misuse Detection with Interpretable Dietary Patterns](https://mlsanity.com/paper/2403.08820/) (2024)
- [Training MLPs on Graphs without Supervision](https://mlsanity.com/paper/2412.03864/) (2024)

## Built on by (most cited first, 19 of 19)

- [Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees](https://mlsanity.com/paper/2412.16441/) (2024)
- [AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answering](https://mlsanity.com/paper/2510.05445/) (2025)
- [Modality-Aware Neuron Pruning for Unlearning in Multimodal Large Language Models](https://mlsanity.com/paper/2502.15910/) (2025)
- [AutoData: A Multi-Agent System for Open Web Data Collection](https://mlsanity.com/paper/2505.15859/) (2025)
- [AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora](https://mlsanity.com/paper/2505.23628/) (2025)
- [Graph is a Substrate Across Data Modalities](https://mlsanity.com/paper/2601.22384/) (2026)
- [LLM-Empowered Class Imbalanced Graph Prompt Learning for Online Drug Trafficking Detection](https://mlsanity.com/paper/2503.01900/) (2025)
- [NG-Router: Graph-Supervised Multi-Agent Collaboration for Nutrition Question Answering](https://mlsanity.com/paper/2510.09854/) (2025)
- [Semantic Refinement with LLMs for Graph Representations](https://mlsanity.com/paper/2512.21106/) (2025)
- [Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphs](https://mlsanity.com/paper/2505.17599/) (2025)
- [Temporal Graph Pattern Machine](https://mlsanity.com/paper/2601.22454/) (2026)
- [BHyGNN+: Unsupervised Representation Learning for Heterophilic Hypergraphs](https://mlsanity.com/paper/2602.14919/) (2026)
- [OPBench: A Graph Benchmark to Combat the Opioid Crisis](https://mlsanity.com/paper/2602.14602/) (2026)
- [Beyond One-Size-Fits-All: Adaptive Subgraph Denoising for Zero-Shot Graph Learning with Large Language Models](https://mlsanity.com/paper/2603.02938/) (2026)
- [Let Relations Speak: An End-to-End LLM-GNN Soft Prompt Framework for Fraud Detection](https://mlsanity.com/paper/2605.28524/) (2026)
- [DuConTE: Dual-Granularity Text Encoder with Topology-Constrained Attention for Text-attributed Graphs](https://mlsanity.com/paper/2604.17411/) (2026)
- [Where LLM Annotators Fail: Label-Free Learning on Graphs with LLMs](https://mlsanity.com/paper/2605.27913/) (2026)
- [TAG-DLM: Diffusion Language Models for Text-Attributed Graph Learning](https://mlsanity.com/paper/2606.31166/) (2026)
- [Semi-Supervised Text-Attributed Graph Distillation](https://mlsanity.com/paper/2607.20477/) (2026)

## Abstract (arXiv)

Graphs are ubiquitous structures found in numerous real-world applications, such as drug discovery, recommender systems, and social network analysis. To model graph-structured data, graph neural networks (GNNs) have become a popular tool. However, existing GNN architectures encounter challenges in cross-graph learning where multiple graphs have different feature spaces. To address this, recent approaches introduce text-attributed graphs (TAGs), where each node is associated with a textual description, which can be projected into a unified feature space using textual encoders. While promising, this method relies heavily on the availability of text-attributed graph data, which is difficult to obtain in practice. To bridge this gap, we propose a novel method named Topology-Aware Node description Synthesis (TANS), leveraging large language models (LLMs) to convert existing graphs into text-attributed graphs. The key idea is to integrate topological information into LLMs to explain how graph topology influences node semantics. We evaluate our TANS on text-rich, text-limited, and text-free graphs, demonstrating its applicability. Notably, on text-free graphs, our method significantly outperforms existing approaches that manually design node features, showcasing the potential of LLMs for preprocessing graph-structured data in the absence of textual information. The code and data are available at https://github.com/Zehong-Wang/TANS.