NTKMTL: Mitigating Task Imbalance in Multi-Task Learning from Neural Tangent Kernel Perspective
Not on the map. Why?
Abstract (arXiv)
Multi-Task Learning (MTL) enables a single model to learn multiple tasks simultaneously, leveraging knowledge transfer among tasks for enhanced generalization, and has been widely applied across various domains. However, task imbalance remains a major challenge in MTL. Although balancing the convergence speeds of different tasks is an effective approach to address this issue, it is highly challenging to accurately characterize the training dynamics and convergence speeds of multiple tasks within the complex MTL system. To this end, we attempt to analyze the training dynamics in MTL by leveraging Neural Tangent Kernel (NTK) theory and propose a new MTL method, NTKMTL. Specifically, we introduce an extended NTK matrix for MTL and adopt spectral analysis to balance the convergence speeds of multiple tasks, thereby mitigating task imbalance. Based on the approximation via shared representation, we further propose NTKMTL-SR, achieving training efficiency while maintaining competitive performance. Extensive experiments demonstrate that our methods achieve state-of-the-art performance across a wide range of benchmarks, including both multi-task supervised learning and multi-task reinforcement learning. Source code is available at https://github.com/jianke0604/NTKMTL.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 50 of its 63 references that are in the map. First posted on arXiv 21 Oct 2025.
Builds on
50 of 63 references.
References · 50
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2014Deep Learning Face Attributes in the Wildcited by 3,679
- 2017Neural Message Passing for Quantum Chemistrycited by 3,460
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
- 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
- 2018On the Spectral Bias of Neural Networkscited by 1,321
- 2020Gradient Surgery for Multi-Task Learningcited by 1,210
- 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
- 2017A Survey on Multi-Task Learningcited by 838
38 more
- 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
- 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
- 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
- 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
- 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
- 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
- 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
- 2018End-to-End Multi-Task Learning with Attentioncited by 533
- 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
- 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
- 2022Multi-Task Learning as a Bargaining Gamecited by 209
- 2020Deep Learning-Based Human Pose Estimation: A Surveycited by 205
- 2020Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropoutcited by 199
- 2019On the Inductive Bias of Neural Tangent Kernelscited by 177
- 2021The Principles of Deep Learning Theorycited by 169
- 2020Frequency Bias in Neural Networks for Input of Non-Uniform Densitycited by 140
- 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 124
- 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 100
- 2020Self-Supervised Learning for Domain Adaptation on Point-Cloudscited by 99
- 2017Adversarial Multi-task Learning for Text Classificationcited by 91
- 2021RotoGrad: Gradient Homogenization in Multitask Learningcited by 82
- 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
- 2023FAMO: Fast Adaptive Multitask Optimizationcited by 74
- 2023Independent Component Alignment for Multi-Task Learningcited by 69
- 2021Multi-Task Learning in Natural Language Processing: An Overviewcited by 61
- 2020Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Datacited by 50
- 2024Smooth Tchebycheff Scalarization for Multi-Objective Optimizationcited by 50
- 2019MultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learningcited by 45
- 2020Auxiliary Learning by Implicit Differentiationcited by 45
- 2024Fair Resource Allocation in Multi-Task Learningcited by 34
- 2023Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithmscited by 33
- 2022AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learningcited by 31
- 2023Dual-Balancing for Multi-Task Learningcited by 27
- 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 15
- 2024GO4Align: Group Optimization for Multi-Task Alignmentcited by 11
- 2024Grounding and Enhancing Grid-based Models for Neural Fieldscited by 8
- 2022On the Interpretability of Regularisation for Neural Networks Through Model Gradient Similaritycited by 4
- 2017Self-Paced Multitask Learning with Shared Knowledgecited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.