# Automated Search for Resource-Efficient Branched Multi-Task Networks

arXiv: https://arxiv.org/abs/2008.10292 · HTML: https://mlsanity.com/paper/2008.10292/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2020.

## Built on by (most cited first, 30 of 51)

- [FAMO: Fast Adaptive Multitask Optimization](https://mlsanity.com/paper/2306.03792/) (2023)
- [Exploring Relational Context for Multi-Task Dense Prediction](https://mlsanity.com/paper/2104.13874/) (2021)
- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [Multi-Task Dense Prediction via Mixture of Low-Rank Experts](https://mlsanity.com/paper/2403.17749/) (2024)
- [Recon: Reducing Conflicting Gradients from the Root for Multi-Task Learning](https://mlsanity.com/paper/2302.11289/) (2023)
- [Learning Multiple Dense Prediction Tasks from Partially Annotated Data](https://mlsanity.com/paper/2111.14893/) (2021)
- [Multi-Task Learning with Multi-Query Transformer for Dense Prediction](https://mlsanity.com/paper/2205.14354/) (2022)
- [Learning to Relate Depth and Semantics for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2105.07830/) (2021)
- [Universal Representations: A Unified Look at Multiple Task and Domain Learning](https://mlsanity.com/paper/2204.02744/) (2022)
- [InvPT++: Inverted Pyramid Multi-Task Transformer for Visual Scene Understanding](https://mlsanity.com/paper/2306.04842/) (2023)
- [Controllable Dynamic Multi-Task Architectures](https://mlsanity.com/paper/2203.14949/) (2022)
- [AutoMTL: A Programming Framework for Automating Efficient Multi-Task Learning](https://mlsanity.com/paper/2110.13076/) (2021)
- [FedHCA2: Towards Hetero-Client Federated Multi-Task Learning](https://mlsanity.com/paper/2311.13250/) (2023)
- [Selective Task Group Updates for Multi-Task Optimization](https://mlsanity.com/paper/2502.11986/) (2025)
- [PromptonomyViT: Multi-Task Prompt Learning Improves Video Transformers using Synthetic Scene Data](https://mlsanity.com/paper/2212.04821/) (2022)
- [Prompt Guided Transformer for Multi-Task Dense Prediction](https://mlsanity.com/paper/2307.15362/) (2023)
- [BridgeNet: Comprehensive and Effective Feature Interactions via Bridge Feature for Multi-task Dense Predictions](https://mlsanity.com/paper/2312.13514/) (2023)
- [Multi-task Learning with 3D-Aware Regularization](https://mlsanity.com/paper/2310.00986/) (2023)
- [Efficient Controllable Multi-Task Architectures](https://mlsanity.com/paper/2308.11744/) (2023)
- [Entire Chain Uplift Modeling with Context-Enhanced Learning for Intelligent Marketing](https://mlsanity.com/paper/2402.03379/) (2024)
- [Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learning](https://mlsanity.com/paper/2410.14633/) (2024)
- [A Tree-Structured Multi-Task Model Recommender](https://mlsanity.com/paper/2203.05092/) (2022)
- [Efficient Multi-task Uncertainties for Joint Semantic Segmentation and Monocular Depth Estimation](https://mlsanity.com/paper/2402.10580/) (2024)
- [Joint-Task Regularization for Partially Labeled Multi-Task Learning](https://mlsanity.com/paper/2404.01976/) (2024)
- [Toward Edge-Efficient Dense Predictions with Synergistic Multi-Task Neural Architecture Search](https://mlsanity.com/paper/2210.01384/) (2022)
- [Composite Learning for Robust and Effective Dense Predictions](https://mlsanity.com/paper/2210.07239/) (2022)
- [Resolving Token-Space Gradient Conflicts: Token Space Manipulation for Transformer-Based Multi-Task Learning](https://mlsanity.com/paper/2507.07485/) (2025)
- [Multi-Task Classification of Sewer Pipe Defects and Properties using a Cross-Task Graph Neural Network Decoder](https://mlsanity.com/paper/2111.07846/) (2021)
- [Data exploitation: multi-task learning of object detection and semantic segmentation on partially annotated data](https://mlsanity.com/paper/2311.04040/) (2023)
- [Revisit the Imbalance Optimization in Multi-task Learning: An Experimental Analysis](https://mlsanity.com/paper/2509.23915/) (2025)

## Abstract (arXiv)

The multi-modal nature of many vision problems calls for neural network architectures that can perform multiple tasks concurrently. Typically, such architectures have been handcrafted in the literature. However, given the size and complexity of the problem, this manual architecture exploration likely exceeds human design abilities. In this paper, we propose a principled approach, rooted in differentiable neural architecture search, to automatically define branching (tree-like) structures in the encoding stage of a multi-task neural network. To allow flexibility within resource-constrained environments, we introduce a proxyless, resource-aware loss that dynamically controls the model size. Evaluations across a variety of dense prediction tasks show that our approach consistently finds high-performing branching structures within limited resource budgets.