# Trace Norm Regularised Deep Multi-Task Learning

arXiv: https://arxiv.org/abs/1606.04038 · HTML: https://mlsanity.com/paper/1606.04038/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2016.

## Built on by (most cited first, 30 of 50)

- [Efficiently Identifying Task Groupings for Multi-Task Learning](https://mlsanity.com/paper/2109.04617/) (2021)
- [Invariant Information Bottleneck for Domain Generalization](https://mlsanity.com/paper/2106.06333/) (2021)
- [Multi-Task Dense Prediction via Mixture of Low-Rank Experts](https://mlsanity.com/paper/2403.17749/) (2024)
- [Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction](https://mlsanity.com/paper/2301.10034/) (2023)
- [One Model is All You Need: Multi-Task Learning Enables Simultaneous Histology Image Segmentation and Classification](https://mlsanity.com/paper/2203.00077/) (2022)
- [Annotating Columns with Pre-trained Language Models](https://mlsanity.com/paper/2104.01785/) (2021)
- [Deep Multi-Task Learning for Joint Localization, Perception, and Prediction](https://mlsanity.com/paper/2101.06720/) (2021)
- [Revisit Multimodal Meta-Learning through the Lens of Multi-Task Learning](https://mlsanity.com/paper/2110.14202/) (2021)
- [Egocentric Video Task Translation](https://mlsanity.com/paper/2212.06301/) (2022)
- [Emotion and Intent Joint Understanding in Multimodal Conversation: A Benchmarking Dataset](https://mlsanity.com/paper/2407.02751/) (2024)
- [Skill Expansion and Composition in Parameter Space](https://mlsanity.com/paper/2502.05932/) (2025)
- [Uni-Retrieval: A Multi-Style Retrieval Framework for STEM's Education](https://mlsanity.com/paper/2502.05863/) (2025)
- [Losses over Labels: Weakly Supervised Learning via Direct Loss Construction](https://mlsanity.com/paper/2212.06921/) (2022)
- [Beyond RGB: Scene-Property Synthesis with Neural Radiance Fields](https://mlsanity.com/paper/2206.04669/) (2022)
- [Gradient Shaping for Multi-Constraint Safe Reinforcement Learning](https://mlsanity.com/paper/2312.15127/) (2023)
- [Reinforcement Guided Multi-Task Learning Framework for Low-Resource Stereotype Detection](https://mlsanity.com/paper/2203.14349/) (2022)
- [Efficient Controllable Multi-Task Architectures](https://mlsanity.com/paper/2308.11744/) (2023)
- [Unsupervised Modality-Transferable Video Highlight Detection with Representation Activation Sequence Learning](https://mlsanity.com/paper/2403.09401/) (2024)
- [When Multi-Task Learning Meets Partial Supervision: A Computer Vision Review](https://mlsanity.com/paper/2307.14382/) (2023)
- [Analytical Uncertainty-Based Loss Weighting in Multi-Task Learning](https://mlsanity.com/paper/2408.07985/) (2024)
- [Saliency-Regularized Deep Multi-Task Learning](https://mlsanity.com/paper/2207.01117/) (2022)
- [YOLOR-Based Multi-Task Learning](https://mlsanity.com/paper/2309.16921/) (2023)
- [Multi-task Heterogeneous Graph Learning on Electronic Health Records](https://mlsanity.com/paper/2408.07569/) (2024)
- [A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques](https://mlsanity.com/paper/2302.08571/) (2023)
- [A Novel Computational and Modeling Foundation for Automatic Coherence Assessment](https://mlsanity.com/paper/2310.00598/) (2023)
- [Tasks Structure Regularization in Multi-Task Learning for Improving Facial Attribute Prediction](https://mlsanity.com/paper/2108.04353/) (2021)
- [PartAL: Efficient Partial Active Learning in Multi-Task Visual Settings](https://mlsanity.com/paper/2211.11546/) (2022)
- [ThanoRA: Task Heterogeneity-Aware Multi-Task Low-Rank Adaptation](https://mlsanity.com/paper/2505.18640/) (2025)
- [Learning Twofold Heterogeneous Multi-Task by Sharing Similar Convolution Kernel Pairs](https://mlsanity.com/paper/2101.12431/) (2021)
- [Towards a Better Microcredit Decision](https://mlsanity.com/paper/2209.07574/) (2022)

## Abstract (arXiv)

We propose a framework for training multiple neural networks simultaneously. The parameters from all models are regularised by the tensor trace norm, so that each neural network is encouraged to reuse others' parameters if possible -- this is the main motivation behind multi-task learning. In contrast to many deep multi-task learning models, we do not predefine a parameter sharing strategy by specifying which layers have tied parameters. Instead, our framework considers sharing for all shareable layers, and the sharing strategy is learned in a data-driven way.