# Dynamic Task Weighting Methods for Multi-task Networks in Autonomous Driving Systems

arXiv: https://arxiv.org/abs/2001.02223 · HTML: https://mlsanity.com/paper/2001.02223/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2020.

## Built on by (most cited first, 30 of 34)

- [Deep Model Fusion: A Survey](https://mlsanity.com/paper/2309.15698/) (2023)
- [Surround-view Fisheye Camera Perception for Automated Driving: Overview, Survey and Challenges](https://mlsanity.com/paper/2205.13281/) (2022)
- [Near-field Perception for Low-Speed Vehicle Automation using Surround-view Fisheye Cameras](https://mlsanity.com/paper/2103.17001/) (2021)
- [End-to-end Autonomous Driving with Semantic Depth Cloud Mapping and Multi-agent](https://mlsanity.com/paper/2204.05513/) (2022)
- [Egocentric Video Task Translation](https://mlsanity.com/paper/2212.06301/) (2022)
- [AutoMTL: A Programming Framework for Automating Efficient Multi-Task Learning](https://mlsanity.com/paper/2110.13076/) (2021)
- [Adversarial Attacks on Multi-task Visual Perception for Autonomous Driving](https://mlsanity.com/paper/2107.07449/) (2021)
- [Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction](https://mlsanity.com/paper/2112.02252/) (2021)
- [Training multi-objective/multi-task collocation physics-informed neural network with student/teachers transfer learnings](https://mlsanity.com/paper/2107.11496/) (2021)
- [Optical Flow for Autonomous Driving: Applications, Challenges and Improvements](https://mlsanity.com/paper/2301.04422/) (2023)
- [A Tree-Structured Multi-Task Model Recommender](https://mlsanity.com/paper/2203.05092/) (2022)
- [Sample-Level Weighting for Multi-Task Learning with Auxiliary Tasks](https://mlsanity.com/paper/2306.04519/) (2023)
- [FisheyeSuperPoint: Keypoint Detection and Description Network for Fisheye Images](https://mlsanity.com/paper/2103.00191/) (2021)
- [Instance-Level Task Parameters: A Robust Multi-task Weighting Framework](https://mlsanity.com/paper/2106.06129/) (2021)
- [MuCoMiD: A Multitask Convolutional Learning Framework for miRNA-Disease Association Prediction](https://mlsanity.com/paper/2108.04820/) (2021)
- [Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation](https://mlsanity.com/paper/2110.09674/) (2021)
- [Task Weighting through Gradient Projection for Multitask Learning](https://mlsanity.com/paper/2409.01793/) (2024)
- [Woodscape Fisheye Object Detection for Autonomous Driving -- CVPR 2022 OmniCV Workshop Challenge](https://mlsanity.com/paper/2206.12912/) (2022)
- [A Dynamic Feature Interaction Framework for Multi-task Visual Perception](https://mlsanity.com/paper/2306.05061/) (2023)
- [LiDAR-BEVMTN: Real-Time LiDAR Bird's-Eye View Multi-Task Perception Network for Autonomous Driving](https://mlsanity.com/paper/2307.08850/) (2023)
- [Multi-camera Bird's Eye View Perception for Autonomous Driving](https://mlsanity.com/paper/2309.09080/) (2023)
- [DaF-BEVSeg: Distortion-aware Fisheye Camera based Bird's Eye View Segmentation with Occlusion Reasoning](https://mlsanity.com/paper/2404.06352/) (2024)
- [CerberusDet: Unified Multi-Dataset Object Detection](https://mlsanity.com/paper/2407.12632/) (2024)
- [Flexible-weighted Chamfer Distance: Enhanced Objective Function for Point Cloud Completion](https://mlsanity.com/paper/2505.14218/) (2025)
- ["It's a Match!" -- A Benchmark of Task Affinity Scores for Joint Learning](https://mlsanity.com/paper/2301.02873/) (2023)
- [Emphasizing Unseen Words: New Vocabulary Acquisition for End-to-End Speech Recognition](https://mlsanity.com/paper/2302.09723/) (2023)
- [Attacking All Tasks at Once Using Adversarial Examples in Multi-Task Learning](https://mlsanity.com/paper/2305.12066/) (2023)
- [Robust-Multi-Task Gradient Boosting](https://mlsanity.com/paper/2507.11411/) (2025)
- [Multi-Task Network Pruning and Embedded Optimization for Real-time Deployment in ADAS](https://mlsanity.com/paper/2101.07831/) (2021)
- [WoodScape Motion Segmentation for Autonomous Driving -- CVPR 2023 OmniCV Workshop Challenge](https://mlsanity.com/paper/2401.00910/) (2023)

## Abstract (arXiv)

Deep multi-task networks are of particular interest for autonomous driving systems. They can potentially strike an excellent trade-off between predictive performance, hardware constraints and efficient use of information from multiple types of annotations and modalities. However, training such models is non-trivial and requires balancing learning over all tasks as their respective losses display different scales, ranges and dynamics across training. Multiple task weighting methods that adjust the losses in an adaptive way have been proposed recently on different datasets and combinations of tasks, making it difficult to compare them. In this work, we review and systematically evaluate nine task weighting strategies on common grounds on three automotive datasets (KITTI, Cityscapes and WoodScape). We then propose a novel method combining evolutionary meta-learning and task-based selective backpropagation, for computing task weights leading to reliable network training. Our method outperforms state-of-the-art methods by a significant margin on a two-task application.