mlsanityData to 7 Oct 2026

Boosting Multitask Learning on Graphs through Higher-Order Task Affinities

arXiv 2306.14009 ↗First posted 24 Jun 2023cs.LG cs.SIcited by 15

Not on the map. Why?

Abstract (arXiv)

Predicting node labels on a given graph is a widely studied problem with many applications, including community detection and molecular graph prediction. This paper considers predicting multiple node labeling functions on graphs simultaneously and revisits this problem from a multitask learning perspective. For a concrete example, consider overlapping community detection: each community membership is a binary node classification task. Due to complex overlapping patterns, we find that negative transfer is prevalent when we apply naive multitask learning to multiple community detection, as task relationships are highly nonlinear across different node labeling. To address the challenge, we develop an algorithm to cluster tasks into groups based on a higher-order task affinity measure. We then fit a multitask model on each task group, resulting in a boosting procedure on top of the baseline model. We estimate the higher-order task affinity measure between two tasks as the prediction loss of one task in the presence of another task and a random subset of other tasks. Then, we use spectral clustering on the affinity score matrix to identify task grouping. We design several speedup techniques to compute the higher-order affinity scores efficiently and show that they can predict negative transfers more accurately than pairwise task affinities. We validate our procedure using various community detection and molecular graph prediction data sets, showing favorable results compared with existing methods. Lastly, we provide a theoretical analysis to show that under a planted block model of tasks on graphs, our affinity scores can provably separate tasks into groups.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 36 of its 69 references that are in the map. First posted on arXiv 24 Jun 2023.

Builds on

36 of 69 references.

References · 36

  1. 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,148
  2. 2016node2vec: Scalable Feature Learning for Networkscited by 1,913
  3. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  4. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
  5. 2019Strategies for Pre-training Graph Neural Networkscited by 964
  6. 2017A Survey on Multi-Task Learningcited by 838
  7. 2020TUDataset: A collection of benchmark datasets for learning with graphscited by 718
  8. 2020GCC: Graph Contrastive Coding for Graph Neural Network Pre-Trainingcited by 461
  9. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
  10. 2017Community Detection and Stochastic Block Modelscited by 317
  11. 2020SIGN: Scalable Inception Graph Neural Networkscited by 283
  12. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
24 more
  1. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  2. 2020Scaling Graph Neural Networks with Approximate PageRankcited by 174
  3. 2020Graph Clustering with Graph Neural Networkscited by 163
  4. 2020Exploring and Predicting Transferability across NLP Taskscited by 126
  5. 2020Understanding and Improving Information Transfer in Multi-Task Learningcited by 113
  6. 2016Higher-order organization of complex networkscited by 109
  7. 2022The CLRS Algorithmic Reasoning Benchmarkcited by 103
  8. 2020Transfer Learning of Graph Neural Networks with Ego-graph Information Maximizationcited by 93
  9. 2020Scalable Graph Neural Networks via Bidirectional Propagationcited by 87
  10. 2022Graph Attention Multi-Layer Perceptroncited by 82
  11. 2019Graphonomy: Universal Human Parsing via Graph Transfer Learningcited by 81
  12. 2019Multi-Task Feature Learning for Knowledge Graph Enhanced Recommendationcited by 70
  13. 2016Deep Multi-task Representation Learning: A Tensor Factorisation Approachcited by 69
  14. 2018VERSE: Versatile Graph Embeddings from Similarity Measurescited by 54
  15. 2020On the Generalization Effects of Linear Transformations in Data Augmentationcited by 47
  16. 2021Improved Regularization and Robustness for Fine-tuning in Neural Networkscited by 45
  17. 2023Generalization in Graph Neural Networks: Improved PAC-Bayesian Bounds on Graph Diffusioncited by 39
  18. 2021Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learningcited by 34
  19. 2022Robust Fine-Tuning of Deep Neural Networks with Hessian-based Generalization Guaranteescited by 34
  20. 2020Precise High-Dimensional Asymptotics for Quantifying Heterogeneous Transferscited by 26
  21. 2023Identification of Negative Transfers in Multitask Learning Using Surrogate Modelscited by 20
  22. 2016Intrinsic Geometric Information Transfer Learning on Multiple Graph-Structured Datasetscited by 18
  23. 2022Saliency-Regularized Deep Multi-Task Learningcited by 5
  24. 2019Learning Universal Graph Neural Network Embeddings With Aid Of Transfer Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 6; 2026: 4.

15 papers.

Sort:

By area:

  1. 2024Scalable Multitask Learning Using Gradient-based Estimation of Task Affinitycited by 20
  2. 2023Graph Neural Networks for Road Safety Modeling: Datasets and Evaluations for Accident Analysiscited by 13
  3. 2024Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approachcited by 13
  4. 2025Linear-Time Demonstration Selection for In-Context Learning via Gradient Estimationcited by 8
  5. 2025Scalable Multi-Objective and Meta Reinforcement Learning via Gradient Estimationcited by 6
  6. 2025Causal Representation Learning from Multimodal Clinical Records under Non-Random Modality Missingnesscited by 5
  7. 2024Learning Tree-Structured Composition of Data Augmentationcited by 1
  8. 2024Hierarchical Structure Sharing Empowers Multi-task Heterogeneous GNNs for Customer Expansioncited by 1
  9. 2026Empowering Power Outage Prediction with Spatially Aware Hybrid Graph Neural Networks and Contrastive Learningcited by 1
  10. 2025When Does Global Attention Help? A Unified Empirical Study on Atomistic Graph Learningcited by 0
  11. 2025Efficiently Learning Branching Networks for Multitask Algorithmic Reasoningcited by 0
  12. 2025Learning Multimodal Embeddings for Traffic Accident Prediction and Causal Estimationcited by 0
3 more
  1. 2026Learning Dynamic Representations and Policies from Multimodal Clinical Time-Series with Informative Missingnesscited by 0
  2. 2026MoPLEx: Estimating Plackett-Luce Mixture Models for Multi-Objective Alignmentcited by 0
  3. 2026Long-Context Demonstration Selection Using State Space Modelscited by 0