mlsanityData to 14 Oct 2026

AutoScale: Linear Scalarization Guided by Multi-Task Optimization Metrics

arXiv 2508.13979 ↗First posted 19 Aug 2025cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Recent multi-task learning studies suggest that linear scalarization, when using well-chosen fixed task weights, can achieve comparable to or even better performance than complex multi-task optimization (MTO) methods. It remains unclear why certain weights yield optimal performance and how to determine these weights without relying on exhaustive hyperparameter search. This paper establishes a direct connection between linear scalarization and MTO methods, revealing through extensive experiments that well-performing scalarization weights exhibit specific trends in key MTO metrics, such as high gradient magnitude similarity. Building on this insight, we introduce AutoScale, a simple yet effective two-phase framework that uses these MTO metrics to guide weight selection for linear scalarization, without expensive weight search. AutoScale consistently shows superior performance with high efficiency across diverse datasets including a new large-scale benchmark.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 38 of its 49 references that are in the map. First posted on arXiv 19 Aug 2025.

Builds on

38 of 49 references.

References · 38

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  3. 2016Pyramid Scene Parsing Networkcited by 2,479
  4. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  5. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  6. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  7. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  8. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  9. 2018End-to-End Multi-Task Learning with Attentioncited by 534
  10. 2016Cross-stitch Networks for Multi-task Learningcited by 467
  11. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  12. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 375
26 more
  1. 2022Multi-Task Learning as a Bargaining Gamecited by 215
  2. 2020Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropoutcited by 201
  3. 2015Instance-aware Semantic Segmentation via Multi-task Network Cascadescited by 121
  4. 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 101
  5. 2023FAMO: Fast Adaptive Multitask Optimizationcited by 78
  6. 2023UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representationcited by 74
  7. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  8. 2023Independent Component Alignment for Multi-Task Learningcited by 70
  9. 2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 67
  10. 2022Do Current Multi-Task Optimization Methods in Deep Learning Even Help?cited by 60
  11. 2023Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspectivecited by 51
  12. 2019MultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learningcited by 45
  13. 2021Multi-Objective Meta Learningcited by 39
  14. 2019Regularizing Deep Multi-Task Networks using Orthogonal Gradientscited by 36
  15. 2023Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidancecited by 36
  16. 2024Fair Resource Allocation in Multi-Task Learningcited by 35
  17. 2023Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithmscited by 34
  18. 2023Dual-Balancing for Multi-Task Learningcited by 27
  19. 2024Robust Multi-Task Learning with Excess Riskscited by 24
  20. 2023Scalarization for Multi-Task and Multi-Domain Learning at Scalecited by 19
  21. 2024Jacobian Descent for Multi-Objective Optimizationcited by 18
  22. 2025Selective Task Group Updates for Multi-Task Optimizationcited by 17
  23. 2017Deep Asymmetric Multi-task Feature Learningcited by 15
  24. 2023FULLER: Unified Multi-modality Multi-task 3D Perception via Multi-level Gradient Calibrationcited by 13
  25. 2024Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learningcited by 11
  26. 2024Adaptive Online Mirror Descent for Tchebycheff Scalarization in Multi-Objective Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025DeltaFlow: An Efficient Multi-frame Scene Flow Estimation Methodcited by 8
  2. 2026CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal Representationscited by 0