mlsanityData to 14 Oct 2026

Rethinking Message Passing as Retrieval for Text-Attributed Graph Learning

arXiv 2608.26732 ↗First posted 27 Aug 2026cs.LG cs.AIcited by 0

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Graph neural networks (GNNs) are typically conceptualized as message-passing neural networks, yet it remains unclear why neighborhood aggregation reliably outperforms node-wise multilayer perceptrons (MLPs). Despite its empirical success, this paradigm can be computationally expensive and sensitive to imperfect graph structures. In this work, we present a retrieval-augmented view of GNNs: each layer makes predictions by applying an MLP to a node representation together with a permutation-invariant summary of retrieved graph context. Motivated by this perspective, we propose RTA, a simple MLP-based framework that replaces structural message passing with label-aware retrieval and propagation. We provide theoretical insights that (i) connect retrieval-based aggregation to softmax-attention message passing, and (ii) establish the robustness of retrieved-context supervision to mis-retrieved outliers. Experiments on multiple text-attributed graph benchmarks show that RTA matches or even outperforms strong GNN and graph LLM baselines while improving efficiency and robustness across diverse scenarios.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 51 of its 63 references that are in the map. First posted on arXiv 27 Aug 2026.

Builds on

51 of 63 references.

References · 51

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2017Graph Attention Networkscited by 8,923
  7. 2017Inductive Representation Learning on Large Graphscited by 6,320
  8. 2018How Powerful are Graph Neural Networks?cited by 4,393
  9. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  10. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  11. 2017Billion-scale similarity search with GPUscited by 2,497
  12. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
38 more
  1. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  2. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  3. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  4. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  5. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,139
  6. 2018Deep Graph Infomaxcited by 1,106
  7. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  8. 2019Graph Neural Networks for Social Recommendationcited by 714
  9. 2020On the Bottleneck of Graph Neural Networks and its Practical Implicationscited by 631
  10. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  11. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
  12. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 448
  13. 2020Graph Structure Learning for Robust Graph Neural Networkscited by 371
  14. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 281
  15. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 274
  16. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 243
  17. 2025From RAG to Memory: Non-Parametric Continual Learning for Large Language Modelscited by 231
  18. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 226
  19. 2024G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answeringcited by 223
  20. 2023NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classificationcited by 210
  21. 2024Corrective Retrieval Augmented Generationcited by 197
  22. 2024LLaGA: Large Language and Graph Assistantcited by 181
  23. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  24. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 166
  25. 2022Learning on Large-scale Text-attributed Graphs via Variational Inferencecited by 155
  26. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 138
  27. 2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 133
  28. 2020Node Similarity Preserving Graph Convolutional Networkscited by 112
  29. 2022What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoderscited by 82
  30. 2024Making Text Embedders Few-Shot Learnerscited by 79
  31. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 77
  32. 2022Does GNN Pretraining Help Molecular Representation?cited by 60
  33. 2024GOFA: A Generative One-For-All Model for Joint Graph Language Modelingcited by 57
  34. 2023LLatrieval: LLM-Verified Retrieval for Verifiable Generationcited by 52
  35. 2020Adversarial Attack on Large Scale Graphcited by 39
  36. 2023ReLM: Leveraging Language Models for Enhanced Chemical Reaction Predictioncited by 24
  37. 2022Deformable Graph Transformercited by 7
  38. 2024How to Make LMs Strong Node Classifiers?cited by 4

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024RAGraph: A General Retrieval-Augmented Graph Learning Framework
  2. 2025Are LLMs Better GNN Helpers? Rethinking Robust Graph Learning under Deficiencies with Iterative Refinement
  3. 2026TAG-DLM: Diffusion Language Models for Text-Attributed Graph Learning
  4. 2026Beyond the Golden Teacher: Enhancing Graph Learning through LLM-GNN Co-teaching
  5. 2026LLM as Graph Kernel: Rethinking Message Passing on Text-Rich Graphs
  6. 2022SA-MLP: Distilling Graph Knowledge from GNNs into Structure-Aware MLP
  7. 2026RAG-GFM: Overcoming In-Memory Bottlenecks in Graph Foundation Models via Retrieval-Augmented Generation
  8. 2026Can LLMs Fool Graph Learning? Exploring Universal Adversarial Attacks on Text-Attributed Graphs