# Latent Multi-task Architecture Learning

arXiv: https://arxiv.org/abs/1705.08142 · HTML: https://mlsanity.com/paper/1705.08142/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2017.

## Built on by (most cited first, 29 of 29)

- [A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods](https://mlsanity.com/paper/2204.03508/) (2022)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [HoME: Hierarchy of Multi-Gate Experts for Multi-Task Learning at Kuaishou](https://mlsanity.com/paper/2408.05430/) (2024)
- [Losses over Labels: Weakly Supervised Learning via Direct Loss Construction](https://mlsanity.com/paper/2212.06921/) (2022)
- [Automatic Expert Selection for Multi-Scenario and Multi-Task Search](https://mlsanity.com/paper/2205.14321/) (2022)
- [Dynamic Neural Network for Multi-Task Learning Searching across Diverse Network Topologies](https://mlsanity.com/paper/2303.06856/) (2023)
- [Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer](https://mlsanity.com/paper/2105.01844/) (2021)
- [Let the Model Decide its Curriculum for Multitask Learning](https://mlsanity.com/paper/2205.09898/) (2022)
- [GDOD: Effective Gradient Descent using Orthogonal Decomposition for Multi-Task Learning](https://mlsanity.com/paper/2301.13465/) (2023)
- [Multi-task Learning for Real-time Autonomous Driving Leveraging Task-adaptive Attention Generator](https://mlsanity.com/paper/2403.03468/) (2024)
- [Mixture of Virtual-Kernel Experts for Multi-Objective User Profile Modeling](https://mlsanity.com/paper/2106.07356/) (2021)
- [RMT-Net: Reject-aware Multi-Task Network for Modeling Missing-not-at-random Data in Financial Credit Scoring](https://mlsanity.com/paper/2206.00568/) (2022)
- [Instance-Level Task Parameters: A Robust Multi-task Weighting Framework](https://mlsanity.com/paper/2106.06129/) (2021)
- [THOS: A Benchmark Dataset for Targeted Hate and Offensive Speech](https://mlsanity.com/paper/2311.06446/) (2023)
- [Modeling Task Relationships in Multi-variate Soft Sensor with Balanced Mixture-of-Experts](https://mlsanity.com/paper/2305.16360/) (2023)
- [Learning Multi-Tasks with Inconsistent Labels by using Auxiliary Big Task](https://mlsanity.com/paper/2201.02305/) (2022)
- [Adaptive Integrated Layered Attention (AILA)](https://mlsanity.com/paper/2503.22742/) (2025)
- [Multi-Faceted Hierarchical Multi-Task Learning for a Large Number of Tasks with Multi-dimensional Relations](https://mlsanity.com/paper/2110.13365/) (2021)
- [Challenges and Opportunities of Using Transformer-Based Multi-Task Learning in NLP Through ML Lifecycle: A Survey](https://mlsanity.com/paper/2308.08234/) (2023)
- [A Framework to Implement 1+N Multi-task Fine-tuning Pattern in LLMs Using the CGC-LORA Algorithm](https://mlsanity.com/paper/2402.01684/) (2024)
- [Advancing Robust Underwater Acoustic Target Recognition through Multi-task Learning and Multi-Gate Mixture-of-Experts](https://mlsanity.com/paper/2411.02787/) (2024)
- [Exploring Data Aggregation and Transformations to Generalize across Visual Domains](https://mlsanity.com/paper/2108.09208/) (2021)
- [Transfer Learning as an Essential Tool for Digital Twins in Renewable Energy Systems](https://mlsanity.com/paper/2203.05026/) (2022)
- [Design Perspectives of Multitask Deep Learning Models and Applications](https://mlsanity.com/paper/2209.13444/) (2022)
- [TMoE-P: Towards the Pareto Optimum for Multivariate Soft Sensors](https://mlsanity.com/paper/2302.10477/) (2023)
- [Microstructure quality control of steels using deep learning](https://mlsanity.com/paper/2306.00797/) (2023)
- [DEPHN: Different Expression Parallel Heterogeneous Network using virtual gradient optimization for Multi-task Learning](https://mlsanity.com/paper/2307.12519/) (2023)
- [CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community](https://mlsanity.com/paper/2407.06485/) (2024)
- [From Boundaries to Semantics: Prompt-Guided Multi-Task Learning for Petrographic Thin-section Segmentation](https://mlsanity.com/paper/2604.14805/) (2026)

## Abstract (arXiv)

Multi-task learning (MTL) allows deep neural networks to learn from related tasks by sharing parameters with other networks. In practice, however, MTL involves searching an enormous space of possible parameter sharing architectures to find (a) the layers or subspaces that benefit from sharing, (b) the appropriate amount of sharing, and (c) the appropriate relative weights of the different task losses. Recent work has addressed each of the above problems in isolation. In this work we present an approach that learns a latent multi-task architecture that jointly addresses (a)--(c). We present experiments on synthetic data and data from OntoNotes 5.0, including four different tasks and seven different domains. Our extension consistently outperforms previous approaches to learning latent architectures for multi-task problems and achieves up to 15% average error reductions over common approaches to MTL.