# Each Graph is a New Language: Graph Learning with LLMs

arXiv: https://arxiv.org/abs/2501.11478 · HTML: https://mlsanity.com/paper/2501.11478/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jan 2025.

## Built on by (most cited first, 11 of 11)

- [You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structures](https://mlsanity.com/paper/2508.06105/) (2025)
- [LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models](https://mlsanity.com/paper/2503.03313/) (2025)
- [Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigation](https://mlsanity.com/paper/2502.18771/) (2025)
- [Knapsack Optimization-based Schema Linking for LLM-based Text-to-SQL Generation](https://mlsanity.com/paper/2502.12911/) (2025)
- [Toward Graph-Tokenizing Large Language Models with Reconstructive Graph Instruction Tuning](https://mlsanity.com/paper/2603.01385/) (2026)
- [Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigation](https://mlsanity.com/paper/2510.08962/) (2025)
- [LoSemB: Logic-Guided Semantic Bridging for Inductive Tool Retrieval](https://mlsanity.com/paper/2508.07690/) (2025)
- [Integrating Graphs, Large Language Models, and Agents: Reasoning and Retrieval](https://mlsanity.com/paper/2604.15951/) (2026)
- [MultiFluxAI Enhancing Platform Engineering with Advanced Agent-Orchestrated Retrieval Systems](https://mlsanity.com/paper/2508.21307/) (2025)
- [Bridging the Divide: End-to-End Sequence-Graph Learning](https://mlsanity.com/paper/2510.25126/) (2025)
- [Odin: Oriented Dual-module Integration for Text-rich Network Representation Learning](https://mlsanity.com/paper/2511.21416/) (2025)

## Abstract (arXiv)

Recent efforts leverage Large Language Models (LLMs) for modeling text-attributed graph structures in node classification tasks. These approaches describe graph structures for LLMs to understand or aggregate LLM-generated textual attribute embeddings through graph structure. However, these approaches face two main limitations in modeling graph structures with LLMs. (i) Graph descriptions become verbose in describing high-order graph structure. (ii) Textual attributes alone do not contain adequate graph structure information. It is challenging to model graph structure concisely and adequately with LLMs. LLMs lack built-in mechanisms to model graph structures directly. They also struggle with complex long-range dependencies between high-order nodes and target nodes. Inspired by the observation that LLMs pre-trained on one language can achieve exceptional performance on another with minimal additional training, we propose Graph-Defined Language for Large Language Model (GDL4LLM). This novel framework enables LLMs to transfer their powerful language understanding capabilities to graph-structured data. GDL4LLM translates graphs into a graph language corpus instead of graph descriptions and pre-trains LLMs on this corpus to adequately understand graph structures. During fine-tuning, this corpus describes the structural information of target nodes concisely with only a few tokens. By treating graphs as a new language, GDL4LLM enables LLMs to model graph structures adequately and concisely for node classification tasks. Extensive experiments on three real-world datasets demonstrate that GDL4LLM outperforms description-based and textual attribute embeddings-based baselines by efficiently modeling different orders of graph structure with LLMs.