mlsanityData to 7 Oct 2026

Auto-Lambda: Disentangling Dynamic Task Relationships

arXiv 2202.03091 ↗First posted 7 Feb 2022cs.LG cs.AI cs.CVcited by 73

Not on the map. Why?

Abstract (arXiv)

Understanding the structure of multiple related tasks allows for multi-task learning to improve the generalisation ability of one or all of them. However, it usually requires training each pairwise combination of tasks together in order to capture task relationships, at an extremely high computational cost. In this work, we learn task relationships via an automated weighting framework, named Auto-Lambda. Unlike previous methods where task relationships are assumed to be fixed, Auto-Lambda is a gradient-based meta learning framework which explores continuous, dynamic task relationships via task-specific weightings, and can optimise any choice of combination of tasks through the formulation of a meta-loss; where the validation loss automatically influences task weightings throughout training. We apply the proposed framework to both multi-task and auxiliary learning problems in computer vision and robotics, and show that Auto-Lambda achieves state-of-the-art performance, even when compared to optimisation strategies designed specifically for each problem and data domain. Finally, we observe that Auto-Lambda can discover interesting learning behaviors, leading to new insights in multi-task learning. Code is available at https://github.com/lorenmt/auto-lambda.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Builds on 51 of its 60 references that are in the map. First posted on arXiv 7 Feb 2022.

Builds on

51 of 60 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  5. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  6. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  7. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  8. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  9. 2015Prioritized Experience Replaycited by 1,070
  10. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  11. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  12. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
38 more
  1. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  2. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  3. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  4. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  5. 2018End-to-End Multi-Task Learning with Attentioncited by 533
  6. 2016Cross-stitch Networks for Multi-task Learningcited by 466
  7. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  8. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
  9. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
  10. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 357
  11. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  12. 2019Pareto Multi-Task Learningcited by 224
  13. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
  14. 2022Multi-Task Learning as a Bargaining Gamecited by 209
  15. 2016UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memorycited by 200
  16. 2020Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropoutcited by 199
  17. 2019AdaShare: Learning What To Share For Efficient Deep Multi-Task Learningcited by 159
  18. 2020MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learningcited by 157
  19. 2019Attentive Single-Tasking of Multiple Taskscited by 148
  20. 2017Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learningcited by 125
  21. 2020Robust Learning Through Cross-Task Consistencycited by 105
  22. 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 98
  23. 2019Self-Supervised Generalisation with Meta Auxiliary Learningcited by 88
  24. 2019Representation Similarity Analysis for Efficient Task taxonomy & Transfer Learningcited by 86
  25. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  26. 2020Meta-Learning with Adaptive Hyperparameterscited by 79
  27. 2017Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Taskcited by 66
  28. 2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 65
  29. 2021Part-aware Panoptic Segmentationcited by 56
  30. 2020MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learningcited by 50
  31. 2021Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptationcited by 50
  32. 2020Auxiliary Learning by Implicit Differentiationcited by 45
  33. 2021Multi-Objective Meta Learningcited by 39
  34. 2020Adaptive Task Sampling for Meta-Learningcited by 29
  35. 2020Probabilistic Active Meta-Learningcited by 23
  36. 2021Robustness via Cross-Domain Ensemblescited by 23
  37. 2021MultiTask-CenterNet (MCN): Efficient and Diverse Multitask Learning using an Anchor Free Approachcited by 17
  38. 2017Deep Asymmetric Multi-task Feature Learningcited by 15

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 17; 2024: 28; 2025: 13; 2026: 8.

73 papers.

Sort:

By area:

  1. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  2. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 276
  3. 2023Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulationcited by 135
  4. 2024ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulationcited by 106
  5. 2024Representation Surgery for Multi-Task Model Mergingcited by 95
  6. 2022Instruction-driven history-aware policies for robotic manipulationscited by 93
  7. 2023FAMO: Fast Adaptive Multitask Optimizationcited by 74
  8. 2023PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulationcited by 56
  9. 2024Smooth Tchebycheff Scalarization for Multi-Objective Optimizationcited by 50
  10. 2024Towards Generalizable Vision-Language Robotic Manipulation: A Benchmark and LLM-guided 3D Policycited by 37
  11. 2022Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task modelscited by 29
  12. 2023ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learningcited by 28
38 more
  1. 2023Prismer: A Vision-Language Model with Multi-Task Expertscited by 28
  2. 2023Dual-Balancing for Multi-Task Learningcited by 27
  3. 2024SUGAR: Pre-training 3D Visual Representations for Roboticscited by 26
  4. 2025CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Mergingcited by 22
  5. 2023Identification of Negative Transfers in Multitask Learning Using Surrogate Modelscited by 20
  6. 2024Scalable Multitask Learning Using Gradient-based Estimation of Task Affinitycited by 20
  7. 2024DiffusionMTL: Learning Multi-Task Denoising Diffusion Model from Partially Annotated Datacited by 18
  8. 2023A Universal Semantic-Geometric Representation for Robotic Manipulationcited by 16
  9. 2024Few for Many: Tchebycheff Set Scalarization for Many-Objective Optimizationcited by 15
  10. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 15
  11. 2022Instruction-Following Agents with Multimodal Transformercited by 13
  12. 2024Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferencescited by 13
  13. 2023Auxiliary Learning as an Asymmetric Bargaining Gamecited by 12
  14. 2023Image Quality-aware Diagnosis via Meta-knowledge Co-embeddingcited by 12
  15. 2026Multi-Task GRPO: Reliable LLM Reasoning Across Taskscited by 12
  16. 2024GO4Align: Group Optimization for Multi-Task Alignmentcited by 11
  17. 2025Task Arithmetic in Trust Region: A Training-Free Model Merging Approach to Navigate Knowledge Conflictscited by 11
  18. 2023Improvable Gap Balancing for Multi-Task Learningcited by 9
  19. 2024Efficient Pareto Manifold Learning with Low-Rank Structurecited by 9
  20. 2023Self-supervised Hypergraphs for Learning Multiple World Interpretationscited by 8
  21. 2024Semantic Contextualization of Face Forgery: A New Definition, Dataset, and Detection Methodcited by 8
  22. 2022Curriculum-based Asymmetric Multi-task Reinforcement Learningcited by 7
  23. 2022Do Text-to-Text Multi-Task Learners Suffer from Task Conflict?cited by 6
  24. 2023QMP: Q-switch Mixture of Policies for Multi-Task Behavior Sharingcited by 6
  25. 2024A Behavior-Aware Approach for Deep Reinforcement Learning in Non-stationary Environments without Known Change Pointscited by 5
  26. 2024Redundancy-aware Action Spaces for Robot Learningcited by 5
  27. 2025Bi-Level Optimization for Self-Supervised AI-Generated Face Detectioncited by 4
  28. 2023Learning Task-preferred Inference Routes for Gradient De-conflict in Multi-output DNNscited by 3
  29. 2024Cross-Task Affinity Learning for Multitask Dense Scene Predictionscited by 3
  30. 2024Towards Principled Task Grouping for Multi-Task Learningcited by 3
  31. 2025Revisit the Imbalance Optimization in Multi-task Learning: An Experimental Analysiscited by 3
  32. 2022An Ensemble Method to Automatically Grade Diabetic Retinopathy with Optical Coherence Tomography Angiography Imagescited by 2
  33. 2023Task Weighting in Meta-learning with Trajectory Optimisationcited by 2
  34. 2023Examining Common Paradigms in Multi-Task Learningcited by 2
  35. 2024Joint multi-task learning improves weakly-supervised biomarker prediction in computational pathologycited by 2
  36. 2024Region-aware Distribution Contrast: A Novel Approach to Multi-Task Partially Supervised Learningcited by 2
  37. 2024Can Optimization Trajectories Explain Multi-Task Transfer?cited by 2
  38. 2025Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learningcited by 2

50 most cited of 73 All 73 (JSON)