# Smooth Tchebycheff Scalarization for Multi-Objective Optimization

arXiv: https://arxiv.org/abs/2402.19078 · HTML: https://mlsanity.com/paper/2402.19078/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 93 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Builds on (most cited first, 30 of 46)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Conflict-Averse Gradient Descent for Multi-task Learning](https://mlsanity.com/paper/2110.14048/) (2021)
- [On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems](https://mlsanity.com/paper/1906.00331/) (2019)
- [Pareto Multi-Task Learning](https://mlsanity.com/paper/1912.12854/) (2019)
- [Multi-Task Learning as a Bargaining Game](https://mlsanity.com/paper/2202.01017/) (2022)
- [Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout](https://mlsanity.com/paper/2010.06808/) (2020)
- [Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models](https://mlsanity.com/paper/2010.05874/) (2020)
- [Minimax Pareto Fairness: A Multi Objective Perspective](https://mlsanity.com/paper/2011.01821/) (2020)
- [Attentive Single-Tasking of Multiple Tasks](https://mlsanity.com/paper/1904.08918/) (2019)
- [A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach](https://mlsanity.com/paper/1901.08511/) (2019)
- [Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile](https://mlsanity.com/paper/1807.02629/) (2018)
- [Learning the Pareto Front with Hypernetworks](https://mlsanity.com/paper/2010.04104/) (2020)
- [MARS: Markov Molecular Sampling for Multi-objective Drug Discovery](https://mlsanity.com/paper/2103.10432/) (2021)
- [Multi-Objective GFlowNets](https://mlsanity.com/paper/2210.12765/) (2022)
- [Auto-Lambda: Disentangling Dynamic Task Relationships](https://mlsanity.com/paper/2202.03091/) (2022)
- [An Easy-to-use Real-world Multi-objective Optimization Problem Suite](https://mlsanity.com/paper/2009.12867/) (2020)
- [Independent Component Alignment for Multi-Task Learning](https://mlsanity.com/paper/2305.19000/) (2023)
- [Stochastic Variance Reduction Methods for Saddle-Point Problems](https://mlsanity.com/paper/1605.06398/) (2016)
- [Do Current Multi-Task Optimization Methods in Deep Learning Even Help?](https://mlsanity.com/paper/2209.11379/) (2022)
- [A Single-Loop Smoothed Gradient Descent-Ascent Algorithm for Nonconvex-Concave Min-Max Problems](https://mlsanity.com/paper/2010.15768/) (2020)
- [A Unified Single-loop Alternating Gradient Projection Algorithm for Nonconvex-Concave and Convex-Nonconcave Minimax Problems](https://mlsanity.com/paper/2006.02032/) (2020)
- [Efficient Continuous Pareto Exploration in Multi-Task Learning](https://mlsanity.com/paper/2006.16434/) (2020)
- [Pareto Set Learning for Neural Multi-objective Combinatorial Optimization](https://mlsanity.com/paper/2203.15386/) (2022)
- [Pareto Set Learning for Expensive Multi-Objective Optimization](https://mlsanity.com/paper/2210.08495/) (2022)
- [Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspective](https://mlsanity.com/paper/2308.13985/) (2023)

## Built on by (most cited first, 30 of 54)

- [PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model](https://mlsanity.com/paper/2505.06274/) (2025)
- [Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond](https://mlsanity.com/paper/2501.10945/) (2025)
- [Few for Many: Tchebycheff Set Scalarization for Many-Objective Optimization](https://mlsanity.com/paper/2405.19650/) (2024)
- [Traversing Pareto Optimal Policies: Provably Efficient Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2407.17466/) (2024)
- [Multi-objective Large Language Model Alignment with Hierarchical Experts](https://mlsanity.com/paper/2505.20925/) (2025)
- [Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion](https://mlsanity.com/paper/2406.09770/) (2024)
- [Test-Time Alignment via Hypothesis Reweighting](https://mlsanity.com/paper/2412.08812/) (2024)
- [LibMOON: A Gradient-based MultiObjective OptimizatioN Library in PyTorch](https://mlsanity.com/paper/2409.02969/) (2024)
- [FERERO: A Flexible Framework for Preference-Guided Multi-Objective Learning](https://mlsanity.com/paper/2412.01773/) (2024)
- [Balanced Rate-Distortion Optimization in Learned Image Compression](https://mlsanity.com/paper/2502.20161/) (2025)
- [AReUReDi: Annealed Rectified Updates for Refining Discrete Flows with Multi-Objective Guidance](https://mlsanity.com/paper/2510.00352/) (2025)
- [radarODE-MTL: A Multi-Task Learning Framework with Eccentric Gradient Alignment for Robust Radar-Based ECG Reconstruction](https://mlsanity.com/paper/2410.08656/) (2024)
- [Adaptive Online Mirror Descent for Tchebycheff Scalarization in Multi-Objective Learning](https://mlsanity.com/paper/2410.21764/) (2024)
- [LiRA: Light-Robust Adversary for Model-based Reinforcement Learning in Real World](https://mlsanity.com/paper/2409.19617/) (2024)
- [One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planning](https://mlsanity.com/paper/2509.07945/) (2025)
- [MOBO-OSD: Batch Multi-Objective Bayesian Optimization via Orthogonal Search Directions](https://mlsanity.com/paper/2510.20872/) (2025)
- [UniARM: Towards a Unified Autoregressive Reward Model for Multi-Objective Test-Time Alignment](https://mlsanity.com/paper/2602.09538/) (2026)
- [DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Discrete Diffusion Models](https://mlsanity.com/paper/2604.24357/) (2026)
- [Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learning](https://mlsanity.com/paper/2507.21049/) (2025)
- [FoMEMO: Towards Foundation Models for Expensive Multi-objective Optimization](https://mlsanity.com/paper/2509.03244/) (2025)
- [SPREAD: Sampling-based Pareto front Refinement via Efficient Adaptive Diffusion](https://mlsanity.com/paper/2509.21058/) (2025)
- [NTKMTL: Mitigating Task Imbalance in Multi-Task Learning from Neural Tangent Kernel Perspective](https://mlsanity.com/paper/2510.18258/) (2025)
- [GraphAllocBench: A Flexible Benchmark for Preference-Conditioned Multi-Objective Policy Learning](https://mlsanity.com/paper/2601.20753/) (2026)
- [Uncovering Cross-Objective Interference in Multi-Objective Alignment](https://mlsanity.com/paper/2602.06869/) (2026)
- [How to make the most of your masked language model for protein engineering](https://mlsanity.com/paper/2603.10302/) (2026)
- [A Reward-Free Viewpoint on Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2604.24532/) (2026)
- [Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning](https://mlsanity.com/paper/2606.03962/) (2026)
- [MOS-Attack: A Scalable Multi-objective Adversarial Attack Framework](https://mlsanity.com/paper/2501.07251/) (2025)
- [Learning Pareto manifolds in high dimensions: How can regularization help?](https://mlsanity.com/paper/2503.08849/) (2025)
- [Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution](https://mlsanity.com/paper/2505.21438/) (2025)

## Abstract (arXiv)

Multi-objective optimization problems can be found in many real-world applications, where the objectives often conflict each other and cannot be optimized by a single solution. In the past few decades, numerous methods have been proposed to find Pareto solutions that represent optimal trade-offs among the objectives for a given problem. However, these existing methods could have high computational complexity or may not have good theoretical properties for solving a general differentiable multi-objective optimization problem. In this work, by leveraging the smooth optimization technique, we propose a lightweight and efficient smooth Tchebycheff scalarization approach for gradient-based multi-objective optimization. It has good theoretical properties for finding all Pareto solutions with valid trade-off preferences, while enjoying significantly lower computational complexity compared to other methods. Experimental results on various real-world application problems fully demonstrate the effectiveness of our proposed method.