mlsanityData to 7 Oct 2026

Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs

arXiv 2407.15431 ↗First posted 22 Jul 2024cs.SI cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

The text-attributed graph (TAG) is one kind of important real-world graph-structured data with each node associated with raw texts. For TAGs, traditional few-shot node classification methods directly conduct training on the pre-processed node features and do not consider the raw texts. The performance is highly dependent on the choice of the feature pre-processing method. In this paper, we propose P2TAG, a framework designed for few-shot node classification on TAGs with graph pre-training and prompting. P2TAG first pre-trains the language model (LM) and graph neural network (GNN) on TAGs with self-supervised loss. To fully utilize the ability of language models, we adapt the masked language modeling objective for our framework. The pre-trained model is then used for the few-shot node classification with a mixed prompt method, which simultaneously considers both text and graph information. We conduct experiments on six real-world TAGs, including paper citation networks and product co-purchasing networks. Experimental results demonstrate that our proposed framework outperforms existing graph few-shot learning methods on these datasets with +18.98% ~ +35.98% improvements.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 41 of its 57 references that are in the map. First posted on arXiv 22 Jul 2024.

Builds on

41 of 57 references.

References · 41

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017Graph Attention Networkscited by 8,854
  3. 2017Inductive Representation Learning on Large Graphscited by 6,284
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  8. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  9. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  10. 2019Simplifying Graph Convolutional Networkscited by 1,470
  11. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  12. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
29 more
  1. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  2. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  3. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  4. 2019GraphSAINT: Graph Sampling Based Inductive Learning Methodcited by 618
  5. 2019Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networkscited by 540
  6. 2020Deep Graph Contrastive Representation Learningcited by 535
  7. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 461
  8. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  9. 2020SIGN: Scalable Inception Graph Neural Networkscited by 283
  10. 2022LinkBERT: Pretraining Language Models with Document Linkscited by 209
  11. 2022Learning on Large-scale Text-attributed Graphs via Variational Inferencecited by 153
  12. 2023All in One: Multi-task Prompting for Graph Neural Networkscited by 147
  13. 2021GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graphcited by 143
  14. 2021From Canonical Correlation Analysis to Self-supervised Graph Neural Networkscited by 138
  15. 2021Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Predictioncited by 137
  16. 2019Graph Wavelet Neural Networkcited by 131
  17. 2023GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learnercited by 131
  18. 2019Meta-GNN: On Few-shot Node Classification in Graph Meta-learningcited by 105
  19. 2020Graph Prototypical Networks for Few-shot Learning on Attributed Networkscited by 90
  20. 2020Graph Meta Learning via Local Subgraphscited by 89
  21. 2020Contrastive Self-supervised Learning for Graph Classificationcited by 68
  22. 2022Rethinking and Scaling Up Graph Contrastive Learning: An Extremely Efficient Approach with Group Discriminationcited by 67
  23. 2019Graph Few-shot Learning via Knowledge Transfercited by 63
  24. 2023Augmenting Low-Resource Text Classification with Graph-Grounded Pre-training and Promptingcited by 62
  25. 2021TextGNN: Improving Text Encoder via Graph Neural Network in Sponsored Searchcited by 54
  26. 2023Does Graph Distillation See Like Vision Dataset Counterpart?cited by 45
  27. 2022Task-Adaptive Few-shot Node Classificationcited by 43
  28. 2021AdsGNN: Behavior-Graph Augmented Relevance Modeling in Sponsored Searchcited by 38
  29. 2023Prompt-based Node Feature Extractor for Few-shot Learning on Text-Attributed Graphscited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 2.

11 papers.

  1. 2024Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision?cited by 13
  2. 2025Exploring Graph Learning Tasks with Pure LLMs: A Comprehensive Benchmark and Investigationcited by 11
  3. 2025One Prompt Fits All: Universal Graph Adaptation for Pretrained Modelscited by 11
  4. 2024Multi-Scale Heterogeneous Text-Attributed Graph Datasets From Diverse Domainscited by 5
  5. 2025Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applicationscited by 5
  6. 2025Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphscited by 3
  7. 2025SSTAG: Structure-Aware Self-Supervised Learning Method for Text-Attributed Graphscited by 3
  8. 2024Hound: Hunting Supervision Signals for Few and Zero Shot Node Classification on Text-attributed Graphcited by 2
  9. 2025Exploiting Text Semantics for Few and Zero Shot Node Classification on Text-attributed Graphcited by 1
  10. 2026Both Topology and Text Matter: Revisiting LLM-guided Out-of-Distribution Detection on Text-attributed Graphscited by 0
  11. 2026DuConTE: Dual-Granularity Text Encoder with Topology-Constrained Attention for Text-attributed Graphscited by 0