mlsanityData to 14 Oct 2026

Model Generalization on Text Attribute Graphs: Principles with Large Language Models

arXiv 2502.11836 ↗First posted 17 Feb 2025cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have recently been introduced to graph learning, aiming to extend their zero-shot generalization success to tasks where labeled graph data is scarce. Among these applications, inference over text-attributed graphs (TAGs) presents unique challenges: existing methods struggle with LLMs' limited context length for processing large node neighborhoods and the misalignment between node embeddings and the LLM token space. To address these issues, we establish two key principles for ensuring generalization and derive the framework LLM-BP accordingly: (1) Unifying the attribute space with task-adaptive embeddings, where we leverage LLM-based encoders and task-aware prompting to enhance generalization of the text attribute embeddings; (2) Developing a generalizable graph information aggregation mechanism, for which we adopt belief propagation with LLM-estimated parameters that adapt across graphs. Evaluations on 11 real-world TAG benchmarks demonstrate that LLM-BP significantly outperforms existing approaches, achieving 8.10% improvement with task-conditional embeddings and an additional 1.71% gain from adaptive aggregation. The code and task-adaptive embeddings are publicly available.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 79 of its 108 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

79 of 108 references.

References · 79

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  8. 2017Graph Attention Networkscited by 8,923
  9. 2020Scaling Laws for Neural Language Modelscited by 6,586
  10. 2017Inductive Representation Learning on Large Graphscited by 6,320
  11. 2024GPT-4o System Cardcited by 4,588
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
38 more
  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  2. 2023Mistral 7Bcited by 3,164
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  4. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  6. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  7. 2018Deep Graph Infomaxcited by 1,106
  8. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  9. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  10. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  11. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  12. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  13. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 448
  14. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 431
  15. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
  16. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 329
  17. 2021Graph Self-Supervised Learning: A Surveycited by 289
  18. 2023Exploring the Potential of Large Language Models (LLMs) in Learning on Graphscited by 274
  19. 2020Wiki-CS: A Wikipedia-Based Benchmark for Graph Neural Networkscited by 260
  20. 2023One for All: Towards Training One Graph Model for All Classification Taskscited by 243
  21. 2023GraphGPT: Graph Instruction Tuning for Large Language Modelscited by 226
  22. 2024LLaGA: Large Language and Graph Assistantcited by 181
  23. 2022Learning on Large-scale Text-attributed Graphs via Variational Inferencecited by 155
  24. 2021GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graphcited by 143
  25. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 138
  26. 2023Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learningcited by 133
  27. 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 118
  28. 2024All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretrainingcited by 86
  29. 2024ZeroG: Investigating Cross-dataset Zero-shot Transferability in Graphscited by 85
  30. 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 84
  31. 2024Making Text Embedders Few-Shot Learnerscited by 79
  32. 2013Loopy Belief Propagation for Approximate Inference: An Empirical Studycited by 76
  33. 2023SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learningcited by 73
  34. 2023Label-free Node Classification on Graphs with Large Language Models (LLMS)cited by 72
  35. 2024Can GNN be Good Adapter for LLMs?cited by 71
  36. 2023GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learningcited by 70
  37. 2024GraphTranslator: Aligning Graph Model to Large Language Model for Open-ended Taskscited by 63
  38. 2024LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddingscited by 61

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 2.

8 papers.

  1. 2025Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Modelscited by 6
  2. 2025Graph-KV: Breaking Sequence via Injecting Structural Biases into Large Language Modelscited by 6
  3. 2026GNN-as-Judge: Unleashing the Power of LLMs for Graph Learning with GNN Feedbackcited by 4
  4. 2025Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphscited by 3
  5. 2025When Structure Doesn't Help: LLMs Do Not Read Text-Attributed Graphs as Effectively as We Expectedcited by 3
  6. 2025GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learningcited by 2
  7. 2025GRIP: In-Parameter Graph Reasoning through Fine-Tuning Large Language Modelscited by 1
  8. 2026PromptGNN-sim: Deep Fusion and Alignment of GNN and LLMs for Text-Attributed Graph Learningcited by 0