mlsanityData to 14 Oct 2026

Conflict-Averse Gradient Descent for Multi-task Learning

arXiv 2110.14048 ↗First posted 26 Oct 2021cs.LG cs.AIcited by 459

Not on the map. Why?

Cited by 38 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The goal of multi-task learning is to enable more efficient learning than single task learning by sharing model structures for a diverse set of tasks. A standard multi-task learning objective is to minimize the average loss across all tasks. While straightforward, using this objective often results in much worse final performance for each task than learning them independently. A major challenge in optimizing a multi-task model is the conflicting gradients, where gradients of different task objectives are not well aligned so that following the average gradient direction can be detrimental to specific tasks' performance. Previous work has proposed several heuristics to manipulate the task gradients for mitigating this problem. But most of them lack convergence guarantee and/or could converge to any Pareto-stationary point. In this paper, we introduce Conflict-Averse Gradient descent (CAGrad) which minimizes the average loss function, while leveraging the worst local improvement of individual tasks to regularize the algorithm trajectory. CAGrad balances the objectives automatically and still provably converges to a minimum over the average loss. It includes the regular gradient descent (GD) and the multiple gradient descent algorithm (MGDA) in the multi-objective optimization (MOO) literature as special cases. On a series of challenging multi-task supervised learning and reinforcement learning tasks, CAGrad achieves improved performance over prior state-of-the-art multi-objective gradient manipulation methods.

Key facts. Cited by 459 papers in the map (data through 14 Oct 2026). Builds on 29 of its 48 references that are in the map. First posted on arXiv 26 Oct 2021.

Builds on

29 of 48 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  5. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  6. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  7. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  9. 2017A Survey on Multi-Task Learningcited by 840
  10. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  11. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  12. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
17 more
  1. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  2. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  3. 2018End-to-End Multi-Task Learning with Attentioncited by 534
  4. 2016Cross-stitch Networks for Multi-task Learningcited by 467
  5. 2015Policy Distillationcited by 440
  6. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
  7. 2019Pareto Multi-Task Learningcited by 224
  8. 2020Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropoutcited by 201
  9. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 178
  10. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 174
  11. 2019Attentive Single-Tasking of Multiple Taskscited by 148
  12. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 143
  13. 2017Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learningcited by 126
  14. 2016A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Taskscited by 62
  15. 2020Certified Monotonic Neural Networkscited by 51
  16. 2017Divide-and-Conquer Reinforcement Learningcited by 48
  17. 2020Auxiliary Task Reweighting for Minimum-data Learningcited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 13; 2023: 55; 2024: 102; 2025: 119; 2026: 168.

459 papers.

Sort:

By area:

  1. 2023AdaMerging: Adaptive Model Merging for Multi-Task Learningcited by 252
  2. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 243
  3. 2024Twin-Merging: Dynamic Integration of Modular Expertise in Model Mergingcited by 116
  4. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 115
  5. 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 101
  6. 2024Representation Surgery for Multi-Task Model Mergingcited by 96
  7. 2023Diffusion Models for Reinforcement Learning: A Surveycited by 91
  8. 2023FAMO: Fast Adaptive Multitask Optimizationcited by 78
  9. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  10. 2023Independent Component Alignment for Multi-Task Learningcited by 70
  11. 2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 67
  12. 2024A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trendscited by 60
38 more
  1. 2024Erasing Undesirable Influence in Diffusion Modelscited by 57
  2. 2025Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspectivecited by 56
  3. 2024Smooth Tchebycheff Scalarization for Multi-Objective Optimizationcited by 54
  4. 2024MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmeticcited by 53
  5. 2023Recon: Reducing Conflicting Gradients from the Root for Multi-Task Learningcited by 51
  6. 2023Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspectivecited by 51
  7. 2024Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmeticcited by 50
  8. 2024ConFIG: Towards Conflict-free Training of Physics Informed Neural Networkscited by 46
  9. 2023TaskExpert: Dynamically Assembling Multi-Task Representations with Memorial Mixture-of-Expertscited by 45
  10. 2024MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learningcited by 44
  11. 2022One Model is All You Need: Multi-Task Learning Enables Simultaneous Histology Image Segmentation and Classificationcited by 38
  12. 2023HumanBench: Towards General Human-centric Perception with Projector Assisted Pretrainingcited by 38
  13. 2025MotionGPT3: Human Motion as a Second Modalitycited by 37
  14. 2023Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidancecited by 36
  15. 2024Multimodal Instruction Tuning with Conditional Mixture of LoRAcited by 36
  16. 2024Fair Resource Allocation in Multi-Task Learningcited by 35
  17. 2023Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithmscited by 34
  18. 2023Addressing Negative Transfer in Diffusion Modelscited by 33
  19. 2022AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learningcited by 31
  20. 2022Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task modelscited by 29
  21. 2023ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learningcited by 28
  22. 2023Dual-Balancing for Multi-Task Learningcited by 27
  23. 2022LibMTL: A Python Library for Multi-Task Learningcited by 25
  24. 2024Revisiting Weight Averaging for Model Mergingcited by 25
  25. 2024Robust Multi-Task Learning with Excess Riskscited by 24
  26. 2022Universal Representations: A Unified Look at Multiple Task and Domain Learningcited by 23
  27. 2024MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learningcited by 23
  28. 2025MoE-Loco: Mixture of Experts for Multitask Locomotioncited by 23
  29. 2025CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Mergingcited by 23
  30. 2025Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learnerscited by 23
  31. 2023HarmonyDream: Task Harmonization Inside World Modelscited by 22
  32. 2024MTMamba: Enhancing Multi-Task Dense Scene Understanding by Mamba-Based Decoderscited by 22
  33. 2025Time-o1: Time-Series Forecasting Needs Transformed Label Alignmentcited by 22
  34. 2023Trust Region-Based Safe Distributional Reinforcement Learning for Multiple Constraintscited by 21
  35. 2024E2E-MFD: Towards End-to-End Synchronous Multimodal Fusion Detectioncited by 21
  36. 2024Balance Reward and Safety Optimization for Safe Reinforcement Learning: A Perspective of Gradient Manipulationcited by 21
  37. 2024Dual Cone Gradient Descent for Training Physics-Informed Neural Networkscited by 21
  38. 2024Multi-Task Reinforcement Learning for Quadrotorscited by 20

50 most cited of 459 All 459 (JSON)