# Auxiliary Tasks in Multi-task Learning

arXiv: https://arxiv.org/abs/1805.06334 · HTML: https://mlsanity.com/paper/1805.06334/

## Key facts

- Cited by 108 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2018.

## Built on by (most cited first, 30 of 108)

- [Multimodal Motion Prediction with Stacked Transformers](https://mlsanity.com/paper/2103.11624/) (2021)
- [SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks](https://mlsanity.com/paper/2102.05034/) (2021)
- [MobTCast: Leveraging Auxiliary Trajectory Forecasting for Human Mobility Prediction](https://mlsanity.com/paper/2110.01401/) (2021)
- [Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection](https://mlsanity.com/paper/2403.01786/) (2024)
- [MetaBalance: Improving Multi-Task Recommendations via Adapting Gradient Magnitudes of Auxiliary Tasks](https://mlsanity.com/paper/2203.06801/) (2022)
- [Factored Adaptation for Non-Stationary Reinforcement Learning](https://mlsanity.com/paper/2203.16582/) (2022)
- [A Simple yet Effective Network based on Vision Transformer for Camouflaged Object and Salient Object Detection](https://mlsanity.com/paper/2402.18922/) (2024)
- [Auxiliary Tasks Benefit 3D Skeleton-based Human Motion Prediction](https://mlsanity.com/paper/2308.08942/) (2023)
- [Exploring the Relationship between Alignment and Cross-lingual Transfer in Multilingual Transformers](https://mlsanity.com/paper/2306.02790/) (2023)
- [XVO: Generalized Visual Odometry via Cross-Modal Self-Training](https://mlsanity.com/paper/2309.16772/) (2023)
- [MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models](https://mlsanity.com/paper/2303.13009/) (2023)
- [Sensitivity-Constrained Fourier Neural Operators for Forward and Inverse Problems in Parametric Differential Equations](https://mlsanity.com/paper/2505.08740/) (2025)
- [A Multi-modal Fusion Framework Based on Multi-task Correlation Learning for Cancer Prognosis Prediction](https://mlsanity.com/paper/2201.10353/) (2022)
- [Generating Evidential BEV Maps in Continuous Driving Space](https://mlsanity.com/paper/2302.02928/) (2023)
- [Uncertainty-Aware Explainable Recommendation with Large Language Models](https://mlsanity.com/paper/2402.03366/) (2024)
- [Machine-learned 3D Building Vectorization from Satellite Imagery](https://mlsanity.com/paper/2104.06485/) (2021)
- [Targeted-BEHRT: Deep learning for observational causal inference on longitudinal electronic health records](https://mlsanity.com/paper/2202.03487/) (2022)
- [LightsOut: Diffusion-based Outpainting for Enhanced Lens Flare Removal](https://mlsanity.com/paper/2510.15868/) (2025)
- [Recent Advances in Embedding Methods for Multi-Object Tracking: A Survey](https://mlsanity.com/paper/2205.10766/) (2022)
- [Cross-Mode Knowledge Adaptation for Bike Sharing Demand Prediction using Domain-Adversarial Graph Neural Networks](https://mlsanity.com/paper/2211.08903/) (2022)
- [Unsupervised Melody-to-Lyric Generation](https://mlsanity.com/paper/2305.19228/) (2023)
- [Roman Numeral Analysis with Graph Neural Networks: Onset-wise Predictions from Note-wise Features](https://mlsanity.com/paper/2307.03544/) (2023)
- [LitMC-BERT: transformer-based multi-label classification of biomedical literature with an application on COVID-19 literature curation](https://mlsanity.com/paper/2204.08649/) (2022)
- [When Multi-Task Learning Meets Partial Supervision: A Computer Vision Review](https://mlsanity.com/paper/2307.14382/) (2023)
- [Multi-Modal Multi-Task (3MT) Road Segmentation](https://mlsanity.com/paper/2308.11983/) (2023)
- [Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment](https://mlsanity.com/paper/2607.13429/) (2026)
- [Domain Generalisation for Object Detection under Covariate and Concept Shift](https://mlsanity.com/paper/2203.05294/) (2022)
- [SemSegDepth: A Combined Model for Semantic Segmentation and Depth Completion](https://mlsanity.com/paper/2209.00381/) (2022)
- [Constructing Time-Series Momentum Portfolios with Deep Multi-Task Learning](https://mlsanity.com/paper/2306.13661/) (2023)
- [Multitask Learning in Minimally Invasive Surgical Vision: A Review](https://mlsanity.com/paper/2401.08256/) (2024)

## Abstract (arXiv)

Multi-task convolutional neural networks (CNNs) have shown impressive results for certain combinations of tasks, such as single-image depth estimation (SIDE) and semantic segmentation. This is achieved by pushing the network towards learning a robust representation that generalizes well to different atomic tasks. We extend this concept by adding auxiliary tasks, which are of minor relevance for the application, to the set of learned tasks. As a kind of additional regularization, they are expected to boost the performance of the ultimately desired main tasks. To study the proposed approach, we picked vision-based road scene understanding (RSU) as an exemplary application. Since multi-task learning requires specialized datasets, particularly when using extensive sets of tasks, we provide a multi-modal dataset for multi-task RSU, called synMT. More than 2.5 · 105 synthetic images, annotated with 21 different labels, were acquired from the video game Grand Theft Auto V (GTA V). Our proposed deep multi-task CNN architecture was trained on various combination of tasks using synMT. The experiments confirmed that auxiliary tasks can indeed boost network performance, both in terms of final results and training time.