# All in One: Multi-task Prompting for Graph Neural Networks

arXiv: https://arxiv.org/abs/2307.01504 · HTML: https://mlsanity.com/paper/2307.01504/

## Key facts

- Cited by 148 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 40 references that are in the map.
- First posted on arXiv 4 Jul 2023.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [Graph Transformer Networks](https://mlsanity.com/paper/1911.06455/) (2019)
- [Graph Contrastive Learning with Adaptive Augmentation](https://mlsanity.com/paper/2010.14945/) (2020)
- [Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification](https://mlsanity.com/paper/2009.03509/) (2020)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [GraphMAE: Self-Supervised Masked Graph Autoencoders](https://mlsanity.com/paper/2205.10803/) (2022)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Factual Probing Is [MASK]: Learning vs. Learning to Recall](https://mlsanity.com/paper/2104.05240/) (2021)
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://mlsanity.com/paper/2010.15980/) (2020)
- [Rethinking Graph Neural Networks for Anomaly Detection](https://mlsanity.com/paper/2205.15508/) (2022)
- [SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation](https://mlsanity.com/paper/2202.03104/) (2022)
- [Self-supervised Learning on Graphs: Deep Insights and New Direction](https://mlsanity.com/paper/2006.10141/) (2020)
- [AFEC: Active Forgetting of Negative Transfer in Continual Learning](https://mlsanity.com/paper/2110.12187/) (2021)
- [Predicting Path Failure In Time-Evolving Graphs](https://mlsanity.com/paper/1905.03994/) (2019)
- [Multi-level Graph Convolutional Networks for Cross-platform Anchor Link Prediction](https://mlsanity.com/paper/2006.01963/) (2020)
- [Self-supervised Hypergraph Representation Learning for Sociological Analysis](https://mlsanity.com/paper/2212.11440/) (2022)
- [BrainNet: Epileptic Wave Detection from SEEG with Hierarchical Graph Diffusion Learning](https://mlsanity.com/paper/2306.13101/) (2023)
- [Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2204.07955/) (2022)
- [Wiener Graph Deconvolutional Network Improves Graph Self-Supervised Learning](https://mlsanity.com/paper/2206.12933/) (2022)
- [A Multi-Strategy based Pre-Training Method for Cold-Start Recommendation](https://mlsanity.com/paper/2112.02275/) (2021)

## Built on by (most cited first, 30 of 148)

- [One for All: Towards Training One Graph Model for All Classification Tasks](https://mlsanity.com/paper/2310.00149/) (2023)
- [Graph Foundation Models: Concepts, Opportunities and Challenges](https://mlsanity.com/paper/2310.11829/) (2023)
- [Position: Graph Foundation Models are Already Here](https://mlsanity.com/paper/2402.02216/) (2024)
- [All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining](https://mlsanity.com/paper/2402.09834/) (2024)
- [A Survey of Graph Meets Large Language Model: Progress and Future Directions](https://mlsanity.com/paper/2311.12399/) (2023)
- [ZeroG: Investigating Cross-dataset Zero-shot Transferability in Graphs](https://mlsanity.com/paper/2402.11235/) (2024)
- [GFT: Graph Foundation Model with Transferable Tree Vocabulary](https://mlsanity.com/paper/2411.06070/) (2024)
- [OpenGraph: Towards Open Graph Foundation Models](https://mlsanity.com/paper/2403.01121/) (2024)
- [GraphTranslator: Aligning Graph Model to Large Language Model for Open-ended Tasks](https://mlsanity.com/paper/2402.07197/) (2024)
- [Graph Prompt Learning: A Comprehensive Survey and Beyond](https://mlsanity.com/paper/2311.16534/) (2023)
- [LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings](https://mlsanity.com/paper/2408.14512/) (2024)
- [GraphCLIP: Enhancing Transferability in Graph Foundation Models for Text-Attributed Graphs](https://mlsanity.com/paper/2410.10329/) (2024)
- [GOFA: A Generative One-For-All Model for Joint Graph Language Modeling](https://mlsanity.com/paper/2407.09709/) (2024)
- [Multi-Domain Graph Foundation Models: Robust Knowledge Transfer via Topology Alignment](https://mlsanity.com/paper/2502.02017/) (2025)
- [Deep Graph Anomaly Detection: A Survey and New Perspectives](https://mlsanity.com/paper/2409.09957/) (2024)
- [Generalized Graph Prompt: Toward a Unification of Pre-Training and Downstream Tasks on Graphs](https://mlsanity.com/paper/2311.15317/) (2023)
- [GraphControl: Adding Conditional Control to Universal Graph Pre-trained Models for Graph Domain Transfer Learning](https://mlsanity.com/paper/2310.07365/) (2023)
- [Graph Machine Learning in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2404.14928/) (2024)
- [Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees](https://mlsanity.com/paper/2412.16441/) (2024)
- [ProG: A Graph Prompt Learning Benchmark](https://mlsanity.com/paper/2406.05346/) (2024)
- [Zero-shot Generalist Graph Anomaly Detection with Unified Neighborhood Prompts](https://mlsanity.com/paper/2410.14886/) (2024)
- [SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases](https://mlsanity.com/paper/2308.13212/) (2023)
- [A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning](https://mlsanity.com/paper/2410.12288/) (2024)
- [Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models](https://mlsanity.com/paper/2402.01749/) (2024)
- [AnomalyGFM: Graph Foundation Model for Zero/Few-shot Anomaly Detection](https://mlsanity.com/paper/2502.09254/) (2025)
- [Data-centric Graph Learning: A Survey](https://mlsanity.com/paper/2310.04987/) (2023)
- [GraphLoRA: Structure-Aware Contrastive Low-Rank Adaptation for Cross-Graph Transfer Learning](https://mlsanity.com/paper/2409.16670/) (2024)
- [Graph-oriented Instruction Tuning of Large Language Models for Generic Graph Mining](https://mlsanity.com/paper/2403.04780/) (2024)
- [Edge Prompt Tuning for Graph Neural Networks](https://mlsanity.com/paper/2503.00750/) (2025)
- [Subgraph Pooling: Tackling Negative Transfer on Graphs](https://mlsanity.com/paper/2402.08907/) (2024)

## Abstract (arXiv)

Recently, ''pre-training and fine-tuning'' has been adopted as a standard workflow for many graph tasks since it can take general graph knowledge to relieve the lack of graph annotations from each application. However, graph tasks with node level, edge level, and graph level are far diversified, making the pre-training pretext often incompatible with these multiple tasks. This gap may even cause a ''negative transfer'' to the specific application, leading to poor results. Inspired by the prompt learning in natural language processing (NLP), which has presented significant effectiveness in leveraging prior knowledge for various NLP tasks, we study the prompting topic for graphs with the motivation of filling the gap between pre-trained models and various graph tasks. In this paper, we propose a novel multi-task prompting method for graph models. Specifically, we first unify the format of graph prompts and language prompts with the prompt token, token structure, and inserting pattern. In this way, the prompting idea from NLP can be seamlessly introduced to the graph area. Then, to further narrow the gap between various graph tasks and state-of-the-art pre-training strategies, we further study the task space of various graph applications and reformulate downstream problems to the graph-level task. Afterward, we introduce meta-learning to efficiently learn a better initialization for the multi-task prompt of graphs so that our prompting framework can be more reliable and general for different tasks. We conduct extensive experiments, results from which demonstrate the superiority of our method.