# GraphSculptor: Sculpting Pre-training Coreset for Graph Self-supervised Learning

arXiv: https://arxiv.org/abs/2605.01310 · HTML: https://mlsanity.com/paper/2605.01310/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Builds on 26 of its 33 references that are in the map.
- First posted on arXiv 2 May 2026.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [TUDataset: A collection of benchmark datasets for learning with graphs](https://mlsanity.com/paper/2007.08663/) (2020)
- [Recipe for a General, Powerful, Scalable Graph Transformer](https://mlsanity.com/paper/2205.12454/) (2022)
- [GraphMAE: Self-Supervised Masked Graph Autoencoders](https://mlsanity.com/paper/2205.10803/) (2022)
- [SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation](https://mlsanity.com/paper/2202.03104/) (2022)
- [Graph Condensation for Graph Neural Networks](https://mlsanity.com/paper/2110.07580/) (2021)
- [GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner](https://mlsanity.com/paper/2304.04779/) (2023)
- [Condensing Graphs via One-Step Gradient Matching](https://mlsanity.com/paper/2206.07746/) (2022)
- [BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations](https://mlsanity.com/paper/2310.07276/) (2023)
- [What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoders](https://mlsanity.com/paper/2205.10053/) (2022)
- [Rethinking Tokenizer and Decoder in Masked Graph Modeling for Molecules](https://mlsanity.com/paper/2310.14753/) (2023)
- [Does Graph Distillation See Like Vision Dataset Counterpart?](https://mlsanity.com/paper/2310.09192/) (2023)
- [A Survey of Pretraining on Graphs: Taxonomy, Methods, and Applications](https://mlsanity.com/paper/2202.07893/) (2022)
- [Rethinking Graph Masked Autoencoders through Alignment and Uniformity](https://mlsanity.com/paper/2402.07225/) (2024)
- [GiGaMAE: Generalizable Graph Masked Autoencoder via Collaborative Latent Space Reconstruction](https://mlsanity.com/paper/2308.09663/) (2023)
- [Better with Less: A Data-Active Perspective on Pre-Training Graph Neural Networks](https://mlsanity.com/paper/2311.01038/) (2023)
- [Mirage: Model-Agnostic Graph Distillation for Graph Classification](https://mlsanity.com/paper/2310.09486/) (2023)
- [Beyond Efficiency: Molecular Data Pruning for Enhanced Generalization](https://mlsanity.com/paper/2409.01081/) (2024)
- [GDeR: Safeguarding Efficiency, Balancing, and Robustness via Prototypical Graph Pruning](https://mlsanity.com/paper/2410.13761/) (2024)
- [Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders](https://mlsanity.com/paper/2404.15806/) (2024)
- [Graph Positional Autoencoders as Self-supervised Learners](https://mlsanity.com/paper/2505.23345/) (2025)
- [Hi-GMAE: Hierarchical Graph Masked Autoencoders](https://mlsanity.com/paper/2405.10642/) (2024)

## Abstract (arXiv)

Graph self-supervised learning typically relies on large-scale unlabeled datasets, heavily inflating computational costs. However, empirical evidence suggests that these datasets contain substantial redundancy-our analysis reveals that uniformly subsampling 50% of graphs retains over 96% of downstream performance. To exploit this redundancy, we introduce GraphSculptor for pre-training coreset construction. Unlike methods dependent on additional training-time signals or limited solely to topological statistics, GraphSculptor provides a label-free solution that constructs coresets via two complementary perspectives: intrinsic structure and contextual semantics. Concretely, structural diversity is quantified using intrinsic graph statistics, yielding a structural feature vector for each graph, while semantic diversity is captured by utilizing a pre-trained language model to encode descriptions generated via graph-to-text. GraphSculptor integrates these signals into a unified metric space and performs cluster-aware selection to preserve joint structural-semantic diversity. We further derive a theoretical bound on the loss gap between coreset and full-data pre-training, offering theoretical motivation for our selection formulation. Extensive experiments demonstrate that GraphSculptor effectively sculpts the dataset: a 10% coreset achieves 99.6% of full-data performance while reducing pre-training time by nearly 90%, offering a scalable solution for data-efficient graph pre-training.