# Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond

arXiv: https://arxiv.org/abs/2501.10945 · HTML: https://mlsanity.com/paper/2501.10945/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 143 of its 248 references that are in the map.
- First posted on arXiv 19 Jan 2025.

## Builds on (most cited first, 30 of 143)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [Conflict-Averse Gradient Descent for Multi-task Learning](https://mlsanity.com/paper/2110.14048/) (2021)
- [Fine-Grained Human Feedback Gives Better Rewards for Language Model Training](https://mlsanity.com/paper/2306.01693/) (2023)
- [Which Tasks Should Be Learned Together in Multi-task Learning?](https://mlsanity.com/paper/1905.07553/) (2019)
- [Multi-Task Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2009.09796/) (2020)
- [Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging](https://mlsanity.com/paper/2310.11564/) (2023)
- [PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference](https://mlsanity.com/paper/2406.15513/) (2024)
- [A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions](https://mlsanity.com/paper/2006.02903/) (2020)
- [Efficient Diffusion Training via Min-SNR Weighting Strategy](https://mlsanity.com/paper/2303.09556/) (2023)
- [Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards](https://mlsanity.com/paper/2306.04488/) (2023)

## Built on by (most cited first, 18 of 18)

- [DriveDPO: Policy Learning via Safety DPO For End-to-End Autonomous Driving](https://mlsanity.com/paper/2509.17940/) (2025)
- [PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model](https://mlsanity.com/paper/2505.06274/) (2025)
- [Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets](https://mlsanity.com/paper/2505.12038/) (2025)
- [TurboTrain: Towards Efficient and Balanced Multi-Task Learning for Multi-Agent Perception and Prediction](https://mlsanity.com/paper/2508.04682/) (2025)
- [Advancing General-Purpose Reasoning Models with Modular Gradient Surgery](https://mlsanity.com/paper/2602.02301/) (2026)
- [Non-conflicting Energy Minimization in Reinforcement Learning based Robot Control](https://mlsanity.com/paper/2509.01765/) (2025)
- [Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs](https://mlsanity.com/paper/2509.25414/) (2025)
- [Addressing Gradient Misalignment in Data-Augmented Training for Robust Speech Deepfake Detection](https://mlsanity.com/paper/2509.20682/) (2025)
- [NTKMTL: Mitigating Task Imbalance in Multi-Task Learning from Neural Tangent Kernel Perspective](https://mlsanity.com/paper/2510.18258/) (2025)
- [A Framework for Controllable Multi-objective Learning with Annealed Stein Variational Hypernetworks](https://mlsanity.com/paper/2506.06715/) (2025)
- [SAMO: A Lightweight Sharpness-Aware Approach for Multi-Task Optimization with Joint Global-Local Perturbation](https://mlsanity.com/paper/2507.07883/) (2025)
- [Barrier-enforced multi-objective optimization for direct point and sharp interval forecasting](https://mlsanity.com/paper/2604.18492/) (2026)
- [MGDA-Decoupled: Geometry-Aware Multi-Objective Optimisation for DPO-based LLM Alignment](https://mlsanity.com/paper/2604.20685/) (2026)
- [From Dual Tracking to Clipping: Provably Faster Distributionally Robust Multi-Objective Optimization](https://mlsanity.com/paper/2605.05660/) (2026)
- [SURF: Steering the Scalarization Weight to Uniformly Traverse the Pareto Front](https://mlsanity.com/paper/2605.20619/) (2026)
- [MAdam: Metric-Aware Multi-Objective Adam](https://mlsanity.com/paper/2606.03904/) (2026)
- [Efficient Hessian-Free Methods for Multi-Objective Bilevel Optimization with Nonconvex Lower Level](https://mlsanity.com/paper/2608.12704/) (2026)
- [Geometry-Aware Multi-Objective Optimization: Norm-Constrained Oracles with Entropic Weight Updates](https://mlsanity.com/paper/2610.13773/) (2026)

## Abstract (arXiv)

Many modern deep learning applications require balancing multiple objectives that are often conflicting. Examples include multi-task learning, fairness-aware learning, and the alignment of Large Language Models (LLMs). This leads to multi-objective deep learning, which tries to find optimal trade-offs or Pareto-optimal solutions by adapting mathematical principles from the field of Multi-Objective Optimization (MOO). However, directly applying gradient-based MOO techniques to deep neural networks presents unique challenges, including high computational costs, optimization instability, and the difficulty of effectively incorporating user preferences. This paper provides a comprehensive survey of gradient-based techniques for multi-objective deep learning. We systematically categorize existing algorithms based on their outputs: (i) methods that find a single, well-balanced solution, (ii) methods that generate a finite set of diverse Pareto-optimal solutions, and (iii) methods that learn a continuous Pareto set of solutions. In addition to this taxonomy, the survey covers theoretical analyses, key applications, practical resources, and highlights open challenges and promising directions for future research. A comprehensive list of multi-objective deep learning algorithms is available at https://github.com/Baijiong-Lin/Awesome-Multi-Objective-Deep-Learning.