mlsanityData to 7 Oct 2026

Decoupling Weighing and Selecting for Integrating Multiple Graph Pre-training Tasks

arXiv 2403.01400 ↗First posted 3 Mar 2024cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Recent years have witnessed the great success of graph pre-training for graph representation learning. With hundreds of graph pre-training tasks proposed, integrating knowledge acquired from multiple pre-training tasks has become a popular research topic. In this paper, we identify two important collaborative processes for this topic: (1) select: how to select an optimal task combination from a given task pool based on their compatibility, and (2) weigh: how to weigh the selected tasks based on their importance. While there currently has been a lot of work focused on weighing, comparatively little effort has been devoted to selecting. This paper proposes a novel instance-level framework for integrating multiple graph pre-training tasks, Weigh And Select (WAS), where the two collaborative processes, weighing and selecting, are combined by decoupled siamese networks. Specifically, it first adaptively learns an optimal combination of tasks for each instance from a given task pool, based on which a customized instance-level task weighing strategy is learned. Extensive experiments on 16 graph datasets across node-level and graph-level downstream tasks have demonstrated that by combining a few simple but classical tasks, WAS can achieve comparable performance to other leading counterparts. The code is available at https://github.com/TianyuFan0504/WAS.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 53 of its 77 references that are in the map. First posted on arXiv 3 Mar 2024.

Builds on

53 of 77 references.

References · 53

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Graph Attention Networkscited by 8,854
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2018How Powerful are Graph Neural Networks?cited by 4,355
  6. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  7. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  8. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  9. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  10. 2018Deep Graph Infomaxcited by 1,101
  11. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  12. 2018Pitfalls of Graph Neural Network Evaluationcited by 994
38 more
  1. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
  2. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  3. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  4. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  5. 2020Deep Graph Contrastive Representation Learningcited by 535
  6. 2020Graph Contrastive Learning with Adaptive Augmentationcited by 519
  7. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  8. 2019InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximizationcited by 441
  9. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  10. 2022Prompt-aligned Gradient for Prompt Tuningcited by 339
  11. 2020GPT-GNN: Generative Pre-Training of Graph Neural Networkscited by 302
  12. 2021Graph Self-Supervised Learning: A Surveycited by 289
  13. 2021Graph Contrastive Learning Automatedcited by 287
  14. 2019Adversarial Attacks on Graph Neural Networks via Meta Learningcited by 261
  15. 2021Pre-training Molecular Graph Representation with 3D Geometrycited by 248
  16. 2020GEOM: Energy-annotated molecular conformations for property prediction and molecular generationcited by 238
  17. 2017Multi-task Self-Supervised Visual Learningcited by 200
  18. 2021Adversarial Graph Augmentation to Improve Graph Contrastive Learningcited by 200
  19. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  20. 2022SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentationcited by 171
  21. 2021Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillationcited by 151
  22. 2021Self-supervised Learning on Graphs: Contrastive, Generative,or Predictivecited by 136
  23. 2021Self-supervised Graph-level Representation Learning with Local and Global Structurecited by 125
  24. 2020When Does Self-Supervision Help Graph Convolutional Networks?cited by 121
  25. 2020Understanding and Improving Information Transfer in Multi-Task Learningcited by 113
  26. 2020Self-supervised Learning on Graphs: Deep Insights and New Directioncited by 108
  27. 2022HyperPrompt: Prompt-based Task-Conditioning of Transformerscited by 67
  28. 2022Let Invariant Rationale Discovery Inspire Graph Contrastive Learningcited by 57
  29. 2017Cross-Domain Self-supervised Multi-task Feature Learning using Synthetic Imagerycited by 56
  30. 2021Automated Self-Supervised Learning for Graphscited by 50
  31. 2020Contrastive and Generative Graph Convolutional Networks for Graph-based Semi-Supervised Learningcited by 43
  32. 2020Self-Supervised Graph Representation Learning via Global Context Predictioncited by 38
  33. 2022Multi-task Self-supervised Graph Neural Networks Enable Stronger Task Generalizationcited by 35
  34. 2021Adaptive Transfer Learning on Graph Neural Networkscited by 31
  35. 2023Quantifying the Knowledge in GNNs for Reliable Distillation into MLPscited by 25
  36. 2023Extracting Low-/High- Frequency Knowledge from Graph Neural Networks and Injecting it into MLPs: An Effective GNN-to-MLP Distillation Frameworkcited by 24
  37. 2020Iterative Graph Self-Distillationcited by 21
  38. 2021Gumbel-Attention for Multi-modal Machine Translationcited by 20

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 4.

7 papers.

  1. 2024A Survey on Self-Supervised Graph Foundation Models: Knowledge-Based Perspectivecited by 9
  2. 2024HG-Adapter: Improving Pre-Trained Heterogeneous Graph Neural Networks with Dual Adapterscited by 2
  3. 2026PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documentscited by 1
  4. 2024Learning to Model Graph Structural Information on MLPs via Graph Structure Self-Contrastingcited by 0
  5. 2026Feedback Control for Multi-Objective Graph Self-Supervisioncited by 0
  6. 2026A Unified Perspective for Learning Graph Representations Across Multi-Level Abstractionscited by 0
  7. 2026CalibHyper: Chance-Corrected Relational Hypergraphs for Few-Shot Molecular Property Predictioncited by 0