# Prompt-based Node Feature Extractor for Few-shot Learning on Text-Attributed Graphs

arXiv: https://arxiv.org/abs/2309.02848 · HTML: https://mlsanity.com/paper/2309.02848/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2023.

## Built on by (most cited first, 23 of 23)

- [Large Language Models on Graphs: A Comprehensive Survey](https://mlsanity.com/paper/2312.02783/) (2023)
- [A Survey of Graph Meets Large Language Model: Progress and Future Directions](https://mlsanity.com/paper/2311.12399/) (2023)
- [A Survey of Large Language Models for Graphs](https://mlsanity.com/paper/2405.08011/) (2024)
- [Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs](https://mlsanity.com/paper/2310.18152/) (2023)
- [When Graph meets Multimodal: Benchmarking and Meditating on Multimodal Attributed Graphs Learning](https://mlsanity.com/paper/2410.09132/) (2024)
- [LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models](https://mlsanity.com/paper/2503.03313/) (2025)
- [A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications](https://mlsanity.com/paper/2404.14809/) (2024)
- [Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models](https://mlsanity.com/paper/2409.19667/) (2024)
- [Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques](https://mlsanity.com/paper/2402.05952/) (2024)
- [Large Language Model Meets Graph Neural Network in Knowledge Distillation](https://mlsanity.com/paper/2402.05894/) (2024)
- [Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs](https://mlsanity.com/paper/2407.15431/) (2024)
- [GCoder: Improving Large Language Model for Generalized Graph Problem Solving](https://mlsanity.com/paper/2410.19084/) (2024)
- [A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspective](https://mlsanity.com/paper/2403.16137/) (2024)
- [A Comprehensive Data-centric Overview of Federated Graph Learning](https://mlsanity.com/paper/2507.16541/) (2025)
- [Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications](https://mlsanity.com/paper/2510.21131/) (2025)
- [Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphs](https://mlsanity.com/paper/2505.17599/) (2025)
- [Adversarial Attacks and Defenses on Graph-aware Large Language Models (LLMs)](https://mlsanity.com/paper/2508.04894/) (2025)
- [Hound: Hunting Supervision Signals for Few and Zero Shot Node Classification on Text-attributed Graph](https://mlsanity.com/paper/2409.00727/) (2024)
- [LLM-Powered Text-Attributed Graph Anomaly Detection via Retrieval-Augmented Reasoning](https://mlsanity.com/paper/2511.17584/) (2025)
- [Towards Graph Prompt Learning: A Survey and Beyond](https://mlsanity.com/paper/2408.14520/) (2024)
- [Exploiting Text Semantics for Few and Zero Shot Node Classification on Text-attributed Graph](https://mlsanity.com/paper/2505.08168/) (2025)
- [Scalability Matters: Overcoming Challenges in InstructGLM with Similarity-Degree-Based Sampling](https://mlsanity.com/paper/2505.03799/) (2025)
- [Meta-Semantics Augmented Few-Shot Relational Learning](https://mlsanity.com/paper/2505.05684/) (2025)

## Abstract (arXiv)

Text-attributed Graphs (TAGs) are commonly found in the real world, such as social networks and citation networks, and consist of nodes represented by textual descriptions. Currently, mainstream machine learning methods on TAGs involve a two-stage modeling approach: (1) unsupervised node feature extraction with pre-trained language models (PLMs); and (2) supervised learning using Graph Neural Networks (GNNs). However, we observe that these representations, which have undergone large-scale pre-training, do not significantly improve performance with a limited amount of training samples. The main issue is that existing methods have not effectively integrated information from the graph and downstream tasks simultaneously. In this paper, we propose a novel framework called G-Prompt, which combines a graph adapter and task-specific prompts to extract node features. First, G-Prompt introduces a learnable GNN layer (i.e., adaptor) at the end of PLMs, which is fine-tuned to better capture the masked tokens considering graph neighborhood information. After the adapter is trained, G-Prompt incorporates task-specific prompts to obtain interpretable node representations for the downstream task. Our experiment results demonstrate that our proposed method outperforms current state-of-the-art (SOTA) methods on few-shot node classification. More importantly, in zero-shot settings, the G-Prompt embeddings can not only provide better task interpretability than vanilla PLMs but also achieve comparable performance with fully-supervised baselines.