mlsanityData to 14 Oct 2026

AutoMTL: A Programming Framework for Automating Efficient Multi-Task Learning

arXiv 2110.13076 ↗First posted 25 Oct 2021cs.LG cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

Multi-task learning (MTL) jointly learns a set of tasks by sharing parameters among tasks. It is a promising approach for reducing storage costs while improving task accuracy for many computer vision tasks. The effective adoption of MTL faces two main challenges. The first challenge is to determine what parameters to share across tasks to optimize for both memory efficiency and task accuracy. The second challenge is to automatically apply MTL algorithms to an arbitrary CNN backbone without requiring time-consuming manual re-implementation and significant domain expertise. This paper addresses the challenges by developing the first programming framework AutoMTL that automates efficient MTL model development for vision tasks. AutoMTL takes as inputs an arbitrary backbone convolutional neural network (CNN) and a set of tasks to learn, and automatically produces a multi-task model that achieves high accuracy and small memory footprint simultaneously. Experiments on three popular MTL benchmarks (CityScapes, NYUv2, Tiny-Taskonomy) demonstrate the effectiveness of AutoMTL over state-of-the-art approaches as well as the generalizability of AutoMTL across CNNs. AutoMTL is open-sourced and available at https://github.com/zhanglijun95/AutoMTL.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 44 of its 66 references that are in the map. First posted on arXiv 25 Oct 2021.

Builds on

44 of 66 references.

References · 44

  1. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  3. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  6. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  7. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  8. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  9. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  10. 2017A Survey on Multi-Task Learningcited by 840
  11. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  12. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
32 more
  1. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
  2. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  3. 2018End-to-End Multi-Task Learning with Attentioncited by 534
  4. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  5. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  6. 2016Cross-stitch Networks for Multi-task Learningcited by 467
  7. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 375
  8. 2019Pareto Multi-Task Learningcited by 224
  9. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 222
  10. 2016UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memorycited by 200
  11. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  12. 2019AdaShare: Learning What To Share For Efficient Deep Multi-Task Learningcited by 160
  13. 2019Attentive Single-Tasking of Multiple Taskscited by 148
  14. 2018NDDR-CNN: Layerwise Feature Fusing in Multi-Task CNNs by Neural Discriminative Dimensionality Reductioncited by 127
  15. 2020Learning to Branch for Multi-Task Learningcited by 115
  16. 2016HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognitioncited by 104
  17. 2019Branched Multi-Task Networks: Deciding What Layers To Sharecited by 76
  18. 2013Indoor Semantic Segmentation using depth informationcited by 56
  19. 2015Cross-domain Image Retrieval with a Dual Attribute-aware Ranking Networkcited by 53
  20. 2018Real-Time Joint Semantic Segmentation and Depth Estimation Using Asymmetric Annotationscited by 51
  21. 2020Knowledge Distillation for Multi-task Learningcited by 51
  22. 2020Automated Search for Resource-Efficient Branched Multi-Task Networkscited by 51
  23. 2020MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learningcited by 50
  24. 2019MultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learningcited by 45
  25. 2019Stochastic Filter Groups for Multi-Task CNNs: Learning Specialist and Generalist Convolution Kernelscited by 42
  26. 2019Regularizing Deep Multi-Task Networks using Orthogonal Gradientscited by 36
  27. 2020Dynamic Task Weighting Methods for Multi-task Networks in Autonomous Driving Systemscited by 34
  28. 2017BlitzNet: A Real-Time Deep Network for Scene Understandingcited by 29
  29. 2018Evolutionary Architecture Search For Deep Multitask Networkscited by 23
  30. 2018You Only Search Once: Single Shot Neural Architecture Search via Direct Sparse Optimizationcited by 23
  31. 2019Deep Elastic Networks with Model Selection for Multi-Task Learningcited by 20
  32. 2016Deep Cross Residual Learning for Multitask Visual Recognitioncited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 6; 2025: 2; 2026: 1.

17 papers.

Sort:

By area:

  1. 2023TaskExpert: Dynamically Assembling Multi-Task Representations with Memorial Mixture-of-Expertscited by 45
  2. 2024DINO-Foresight: Looking into the Future with DINOcited by 45
  3. 2024DiffusionMTL: Learning Multi-Task Denoising Diffusion Model from Partially Annotated Datacited by 18
  4. 2023When Multi-Task Learning Meets Partial Supervision: A Computer Vision Reviewcited by 6
  5. 2024Joint-Task Regularization for Partially Labeled Multi-Task Learningcited by 6
  6. 2024Automated Multi-Task Learning for Joint Disease Prediction on Electronic Health Recordscited by 5
  7. 2023Split-Ensemble: Efficient OOD-aware Ensemble via Task and Model Splittingcited by 4
  8. 2024Region-aware Distribution Contrast: A Novel Approach to Multi-Task Partially Supervised Learningcited by 2
  9. 2024AdapMTL: Adaptive Pruning Framework for Multitask Learning Modelcited by 2
  10. 2023Attacking All Tasks at Once Using Adversarial Examples in Multi-Task Learningcited by 1
  11. 2023Parameter-efficient Multi-Task and Multi-Domain Learning using Factorized Tensor Networkscited by 1
  12. 2023Joint 2D-3D Multi-Task Learning on Cityscapes-3D: 3D Detection, Segmentation, and Depth Estimationcited by 0
5 more
  1. 2023AutoTaskFormer: Searching Vision Transformers for Multi-task Learningcited by 0
  2. 2023Label Budget Allocation in Multi-Task Learningcited by 0
  3. 2025Uncertainty Modeling for Multi-Objective RTA Interception with Distillation Accelerationcited by 0
  4. 2025Task-oriented Learnable Diffusion Timesteps for Universal Few-shot Learning of Dense Taskscited by 0
  5. 2026MTLSI-Net: A Linear Semantic Interaction Network for Parameter-Efficient Multi-Task Dense Predictioncited by 0