# LLM as Graph Kernel: Rethinking Message Passing on Text-Rich Graphs

arXiv: https://arxiv.org/abs/2603.14937 · HTML: https://mlsanity.com/paper/2603.14937/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 24 of its 51 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 24 of 24)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Adapting Language Models to Compress Contexts](https://mlsanity.com/paper/2305.14788/) (2023)
- [Scaling Large Language Model-based Multi-Agent Collaboration](https://mlsanity.com/paper/2406.07155/) (2024)
- [Language is All a Graph Needs](https://mlsanity.com/paper/2308.07134/) (2023)
- [LinkBERT: Pretraining Language Models with Document Links](https://mlsanity.com/paper/2203.15827/) (2022)
- [In-context Autoencoder for Context Compression in a Large Language Model](https://mlsanity.com/paper/2307.06945/) (2023)
- [Talk like a Graph: Encoding Graphs for Large Language Models](https://mlsanity.com/paper/2310.04560/) (2023)
- [Learning on Large-scale Text-attributed Graphs via Variational Inference](https://mlsanity.com/paper/2210.14709/) (2022)
- [GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning](https://mlsanity.com/paper/2405.20139/) (2024)
- [Long Context Compression with Activation Beacon](https://mlsanity.com/paper/2401.03462/) (2024)
- [Towards Foundation Models for Knowledge Graph Reasoning](https://mlsanity.com/paper/2310.04562/) (2023)
- [Latent Collaboration in Multi-Agent Systems](https://mlsanity.com/paper/2511.20639/) (2025)
- [ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning](https://mlsanity.com/paper/2104.07644/) (2021)
- [GOFA: A Generative One-For-All Model for Joint Graph Language Modeling](https://mlsanity.com/paper/2407.09709/) (2024)
- [Architectures of Topological Deep Learning: A Survey of Message-Passing Topological Neural Networks](https://mlsanity.com/paper/2304.10031/) (2023)
- [Large Scale Product Graph Construction for Recommendation in E-commerce](https://mlsanity.com/paper/2010.05525/) (2020)
- [LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models](https://mlsanity.com/paper/2503.03313/) (2025)
- [FocusLLM: Precise Understanding of Long Context by Dynamic Condensing](https://mlsanity.com/paper/2408.11745/) (2024)
- [Graph Learning in the Era of LLMs: A Survey from the Perspective of Data, Models, and Tasks](https://mlsanity.com/paper/2412.12456/) (2024)
- [Align-GRAG: Anchor and Rationale Guided Dual Alignment for Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.16237/) (2025)
- [GL-Fusion: Rethinking the Combination of Graph Neural Network and Large Language model](https://mlsanity.com/paper/2412.06849/) (2024)
- [LangTopo: Aligning Language Descriptions of Graphs with Tokenized Topological Modeling](https://mlsanity.com/paper/2406.13250/) (2024)
- [Hierarchical Compression of Text-Rich Graphs via Large Language Models](https://mlsanity.com/paper/2406.11884/) (2024)

## Built on by (most cited first, 1 of 1)

- [SLM-Conditioned Hierarchical Relation Routing for Labeled Property Graph Learning](https://mlsanity.com/paper/2608.26132/) (2026)

## Abstract (arXiv)

Text-rich graphs, which integrate complex structural dependencies with abundant textual information, are ubiquitous yet remain challenging for existing learning paradigms. Conventional methods and even LLM-hybrids compress rich text into static embeddings or summaries before structural reasoning, creating an information bottleneck and detaching updates from the raw content. We argue that in text-rich graphs, the text is not merely a node attribute but the primary medium through which structural relationships are manifested. We introduce RAMP, a Raw-text Anchored Message Passing approach that moves beyond using LLMs as mere feature extractors and instead recasts the LLM itself as a graph-native aggregation operator. RAMP exploits the text-rich nature of the graph via a novel dual-representation scheme: it anchors inference on each node's raw text during each iteration while propagating dynamically optimized messages from neighbors. It further handles both discriminative and generative tasks under a single unified generative formulation. Extensive experiments show that RAMP effectively bridges the gap between graph propagation and deep text reasoning, achieving competitive performance and offering new insights into the role of LLMs as graph kernels for general-purpose graph learning.