mlsanityData to 14 Oct 2026

GRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphs

arXiv 2310.15109 ↗First posted 23 Oct 2023cs.CLcited by 28

Not on the map. Why?

Abstract (arXiv)

Self-supervised representation learning on text-attributed graphs, which aims to create expressive and generalizable representations for various downstream tasks, has received increasing research attention lately. However, existing methods either struggle to capture the full extent of structural context information or rely on task-specific training labels, which largely hampers their effectiveness and generalizability in practice. To solve the problem of self-supervised representation learning on text-attributed graphs, we develop a novel Graph-Centric Language model -- GRENADE. Specifically, GRENADE exploits the synergistic effect of both pre-trained language model and graph neural network by optimizing with two specialized self-supervised learning algorithms: graph-centric contrastive learning and graph-centric knowledge alignment. The proposed graph-centric self-supervised learning algorithms effectively help GRENADE to capture informative textual semantics as well as structural context information on text-attributed graphs. Through extensive experiments, GRENADE shows its superiority over state-of-the-art methods. Implementation is available at urlhttps://github.com/bigheiniu/GRENADE.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 30 of its 42 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

30 of 42 references.

References · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  8. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  9. 2017Graph Attention Networkscited by 8,923
  10. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  11. 2017Inductive Representation Learning on Large Graphscited by 6,320
  12. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
18 more
  1. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  2. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  3. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
  4. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
  5. 2020A Survey on Contrastive Self-supervised Learningcited by 554
  6. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  7. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 211
  8. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  9. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  10. 2022Learning on Large-scale Text-attributed Graphs via Variational Inferencecited by 155
  11. 2021GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graphcited by 143
  12. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 138
  13. 2022NAGphormer: A Tokenized Graph Transformer for Node Classification in Large Graphscited by 137
  14. 2021Augmentation-Free Self-Supervised Learning on Graphscited by 94
  15. 2022Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddingscited by 75
  16. 2021TextGNN: Improving Text Encoder via Graph Neural Network in Sponsored Searchcited by 54
  17. 2023Train Your Own GNN Teacher: Graph-Aware Distillation on Textual Graphscited by 29
  18. 2021Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networkscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 12; 2025: 7; 2026: 6.

28 papers.

Sort:

By area:

  1. 2024LightRAG: Simple and Fast Retrieval-Augmented Generationcited by 315
  2. 2023Large Language Models on Graphs: A Comprehensive Surveycited by 192
  3. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 85
  4. 2024A Survey of Large Language Models for Graphscited by 72
  5. 2024Can LLM Graph Reasoning Generalize beyond Pattern Memorization?cited by 47
  6. 2024Heterogeneous Contrastive Learning for Foundation Models and Beyondcited by 27
  7. 2025LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Modelscited by 22
  8. 2024Let's Ask GNN: Empowering Large Language Model for Graph In-Context Learningcited by 21
  9. 2025When Do LLMs Help With Node Classification? A Comprehensive Analysiscited by 21
  10. 2024Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision?cited by 13
  11. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  12. 2025Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applicationscited by 6
16 more
  1. 2023Efficient End-to-end Language Model Fine-tuning on Graphscited by 5
  2. 2024TAGA: Text-Attributed Graph Self-Supervised Learning by Synergizing Graph and Text Mutual Transformationscited by 5
  3. 2024Multi-Scale Heterogeneous Text-Attributed Graph Datasets From Diverse Domainscited by 5
  4. 2024From Anchors to Answers: A Novel Node Tokenizer for Integrating Graph Structure into Large Language Modelscited by 4
  5. 2026GNN-as-Judge: Unleashing the Power of LLMs for Graph Learning with GNN Feedbackcited by 4
  6. 2024Large Language Models Meet Graph Neural Networks: A Perspective of Graph Miningcited by 3
  7. 2025Integrating Structural and Semantic Signals in Text-Attributed Graphs with BiGTexcited by 3
  8. 2025Learning Noise-Resilient and Transferable Graph-Text Alignment via Dynamic Quality Assessmentcited by 3
  9. 2024Path-LLM: A Shortest-Path-based LLM Learning for Unified Graph Representationcited by 2
  10. 2025Bridging Thoughts and Words: Graph-Based Intent-Semantic Joint Learning for Fake News Detectioncited by 2
  11. 2025DualResearch: Entropy-Gated Dual-Graph Retrieval for Answer Reconstructioncited by 1
  12. 2026Learning Hierarchical Knowledge in Text-Rich Networks with Taxonomy-Informed Representation Learningcited by 1
  13. 2026S2Aligner: Pair-Efficient and Transferable Pre-Training for Sparse Text-Attributed Graphscited by 0
  14. 2026Beyond the Golden Teacher: Enhancing Graph Learning through LLM-GNN Co-teachingcited by 0
  15. 2026GraspLLM: Towards Zero-Shot Generalization on Text-Attributed Graphs with LLMscited by 0
  16. 2026TaLK: Text-attributed Graph Dataset Distillation via Coupling Language Model with Graph-Aware Kernelcited by 0