# All in One and One for All: A Simple yet Effective Method towards Cross-domain Graph Pretraining

arXiv: https://arxiv.org/abs/2402.09834 · HTML: https://mlsanity.com/paper/2402.09834/

## Key facts

- Cited by 86 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 68 references that are in the map.
- First posted on arXiv 15 Feb 2024.

## Builds on (most cited first, 30 of 36)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Variational Graph Auto-Encoders](https://mlsanity.com/paper/1611.07308/) (2016)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Pitfalls of Graph Neural Network Evaluation](https://mlsanity.com/paper/1811.05868/) (2018)
- [Geom-GCN: Geometric Graph Convolutional Networks](https://mlsanity.com/paper/2002.05287/) (2020)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [GraphMAE: Self-Supervised Masked Graph Autoencoders](https://mlsanity.com/paper/2205.10803/) (2022)
- [Beyond Low-frequency Information in Graph Convolutional Networks](https://mlsanity.com/paper/2101.00797/) (2021)
- [Graph Self-Supervised Learning: A Survey](https://mlsanity.com/paper/2103.00111/) (2021)
- [One for All: Towards Training One Graph Model for All Classification Tasks](https://mlsanity.com/paper/2310.00149/) (2023)
- [Rethinking Graph Neural Networks for Anomaly Detection](https://mlsanity.com/paper/2205.15508/) (2022)
- [SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation](https://mlsanity.com/paper/2202.03104/) (2022)
- [GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks](https://mlsanity.com/paper/2302.08043/) (2023)
- [All in One: Multi-task Prompting for Graph Neural Networks](https://mlsanity.com/paper/2307.01504/) (2023)
- [GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner](https://mlsanity.com/paper/2304.04779/) (2023)
- [Multi-Stage Self-Supervised Learning for Graph Convolutional Networks on Graphs with Few Labels](https://mlsanity.com/paper/1902.11038/) (2019)
- [Large Language Models Can Learn Temporal Reasoning](https://mlsanity.com/paper/2401.06853/) (2024)
- [Understanding and Improving Visual Prompting: A Label-Mapping Perspective](https://mlsanity.com/paper/2211.11635/) (2022)
- [Graph Prompt Learning: A Comprehensive Survey and Beyond](https://mlsanity.com/paper/2311.16534/) (2023)
- [GADBench: Revisiting and Benchmarking Supervised Graph Anomaly Detection](https://mlsanity.com/paper/2306.12251/) (2023)
- [SEGNO: Generalizing Equivariant Graph Neural Networks with Physical Inductive Biases](https://mlsanity.com/paper/2308.13212/) (2023)
- [Semi-supervised Domain Adaptation in Graph Transfer Learning](https://mlsanity.com/paper/2309.10773/) (2023)

## Built on by (most cited first, 30 of 86)

- [GFT: Graph Foundation Model with Transferable Tree Vocabulary](https://mlsanity.com/paper/2411.06070/) (2024)
- [SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptation](https://mlsanity.com/paper/2502.05424/) (2025)
- [AnyGraph: Graph Foundation Model in the Wild](https://mlsanity.com/paper/2408.10700/) (2024)
- [ARC: A Generalist Graph Anomaly Detector with In-Context Learning](https://mlsanity.com/paper/2405.16771/) (2024)
- [Multi-Domain Graph Foundation Models: Robust Knowledge Transfer via Topology Alignment](https://mlsanity.com/paper/2502.02017/) (2025)
- [GLBench: A Comprehensive Benchmark for Graph with Large Language Models](https://mlsanity.com/paper/2407.07457/) (2024)
- [RiemannGFM: Learning a Graph Foundation Model from Riemannian Geometry](https://mlsanity.com/paper/2502.03251/) (2025)
- [Turning Tabular Foundation Models into Graph Foundation Models](https://mlsanity.com/paper/2508.20906/) (2025)
- [Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees](https://mlsanity.com/paper/2412.16441/) (2024)
- [ProG: A Graph Prompt Learning Benchmark](https://mlsanity.com/paper/2406.05346/) (2024)
- [Zero-shot Generalist Graph Anomaly Detection with Unified Neighborhood Prompts](https://mlsanity.com/paper/2410.14886/) (2024)
- [AnomalyGFM: Graph Foundation Model for Zero/Few-shot Anomaly Detection](https://mlsanity.com/paper/2502.09254/) (2025)
- [LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models](https://mlsanity.com/paper/2503.03313/) (2025)
- [GraphPFN: A Prior-Data Fitted Graph Foundation Model](https://mlsanity.com/paper/2509.21489/) (2025)
- [GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Data](https://mlsanity.com/paper/2409.14500/) (2024)
- [A Survey of Deep Graph Learning under Distribution Shifts: from Graph Out-of-Distribution Generalization to Adaptation](https://mlsanity.com/paper/2410.19265/) (2024)
- [GraphFM: A generalist graph transformer that learns transferable representations across diverse domains](https://mlsanity.com/paper/2407.11907/) (2024)
- [GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning](https://mlsanity.com/paper/2511.05592/) (2025)
- [One Model for One Graph: A New Perspective for Pretraining with Cross-domain Graphs](https://mlsanity.com/paper/2412.00315/) (2024)
- [One Prompt Fits All: Universal Graph Adaptation for Pretrained Models](https://mlsanity.com/paper/2509.22416/) (2025)
- [Does Graph Prompt Work? A Data Operation Perspective with Theoretical Analysis](https://mlsanity.com/paper/2410.01635/) (2024)
- [Can TabPFN Compete with GNNs for Node Classification via Graph Tabularization?](https://mlsanity.com/paper/2512.08798/) (2025)
- [Cross-Domain Graph Data Scaling: A Showcase with Diffusion Models](https://mlsanity.com/paper/2406.01899/) (2024)
- [Model Generalization on Text Attribute Graphs: Principles with Large Language Models](https://mlsanity.com/paper/2502.11836/) (2025)
- [GCAL: Adapting Graph Models to Evolving Domain Shifts](https://mlsanity.com/paper/2505.16860/) (2025)
- [From Few-Shot to Zero-Shot: Towards Generalist Graph Anomaly Detection](https://mlsanity.com/paper/2602.18793/) (2026)
- [Towards A Universal Graph Structural Encoder](https://mlsanity.com/paper/2504.10917/) (2025)
- [Towards Foundation Models on Graphs: An Analysis on Cross-Dataset Transfer of Pretrained GNNs](https://mlsanity.com/paper/2412.17609/) (2024)
- [Hypergraph Foundation Model](https://mlsanity.com/paper/2503.01203/) (2025)
- [DP-DGAD: A Generalist Dynamic Graph Anomaly Detector with Dynamic Prototypes](https://mlsanity.com/paper/2508.00664/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) have revolutionized the fields of computer vision (CV) and natural language processing (NLP). One of the most notable advancements of LLMs is that a single model is trained on vast and diverse datasets spanning multiple domains -- a paradigm we term `All in One'. This methodology empowers LLMs with super generalization capabilities, facilitating an encompassing comprehension of varied data distributions. Leveraging these capabilities, a single LLM demonstrates remarkable versatility across a variety of domains -- a paradigm we term `One for All'. However, applying this idea to the graph field remains a formidable challenge, with cross-domain pretraining often resulting in negative transfer. This issue is particularly important in few-shot learning scenarios, where the paucity of training data necessitates the incorporation of external knowledge sources. In response to this challenge, we propose a novel approach called Graph COordinators for PrEtraining (GCOPE), that harnesses the underlying commonalities across diverse graph datasets to enhance few-shot learning. Our novel methodology involves a unification framework that amalgamates disparate graph datasets during the pretraining phase to distill and transfer meaningful knowledge to target tasks. Extensive experiments across multiple graph datasets demonstrate the superior efficacy of our approach. By successfully leveraging the synergistic potential of multiple graph datasets for pretraining, our work stands as a pioneering contribution to the realm of graph foundational model.