mlsanityData to 14 Oct 2026

Smooth Tchebycheff Scalarization for Multi-Objective Optimization

arXiv 2402.19078 ↗First posted 29 Feb 2024cs.LG cs.AI cs.NEcited by 54

Not on the map. Why?

Abstract (arXiv)

Multi-objective optimization problems can be found in many real-world applications, where the objectives often conflict each other and cannot be optimized by a single solution. In the past few decades, numerous methods have been proposed to find Pareto solutions that represent optimal trade-offs among the objectives for a given problem. However, these existing methods could have high computational complexity or may not have good theoretical properties for solving a general differentiable multi-objective optimization problem. In this work, by leveraging the smooth optimization technique, we propose a lightweight and efficient smooth Tchebycheff scalarization approach for gradient-based multi-objective optimization. It has good theoretical properties for finding all Pareto solutions with valid trade-off preferences, while enjoying significantly lower computational complexity compared to other methods. Experimental results on various real-world application problems fully demonstrate the effectiveness of our proposed method.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 46 of its 93 references that are in the map. First posted on arXiv 29 Feb 2024.

Builds on

46 of 93 references.

References · 46

  1. 2021Convex optimizationcited by 2,515
  2. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  3. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  4. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  5. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  6. 2018End-to-End Multi-Task Learning with Attentioncited by 534
  7. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  8. 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 270
  9. 2019Pareto Multi-Task Learningcited by 224
  10. 2022Multi-Task Learning as a Bargaining Gamecited by 215
  11. 2020Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropoutcited by 201
  12. 2020Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Modelscited by 169
34 more
  1. 2020Minimax Pareto Fairness: A Multi Objective Perspectivecited by 153
  2. 2019Attentive Single-Tasking of Multiple Taskscited by 148
  3. 2019A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approachcited by 143
  4. 2018Optimistic mirror descent in saddle-point problems: Going the extra (gradient) milecited by 136
  5. 2020Learning the Pareto Front with Hypernetworkscited by 125
  6. 2021MARS: Markov Molecular Sampling for Multi-objective Drug Discoverycited by 125
  7. 2022Multi-Objective GFlowNetscited by 101
  8. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  9. 2020An Easy-to-use Real-world Multi-objective Optimization Problem Suitecited by 71
  10. 2023Independent Component Alignment for Multi-Task Learningcited by 70
  11. 2016Stochastic Variance Reduction Methods for Saddle-Point Problemscited by 60
  12. 2022Do Current Multi-Task Optimization Methods in Deep Learning Even Help?cited by 60
  13. 2020A Single-Loop Smoothed Gradient Descent-Ascent Algorithm for Nonconvex-Concave Min-Max Problemscited by 58
  14. 2020A Unified Single-loop Alternating Gradient Projection Algorithm for Nonconvex-Concave and Convex-Nonconcave Minimax Problemscited by 57
  15. 2020Efficient Continuous Pareto Exploration in Multi-Task Learningcited by 57
  16. 2022Pareto Set Learning for Neural Multi-objective Combinatorial Optimizationcited by 56
  17. 2022Pareto Set Learning for Expensive Multi-Objective Optimizationcited by 51
  18. 2023Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspectivecited by 51
  19. 2019The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learningcited by 46
  20. 2019MultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learningcited by 45
  21. 2021Scalable Pareto Front Approximation for Deep Multi-Objective Learningcited by 45
  22. 2019Differentiable Game Mechanicscited by 43
  23. 2023Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidancecited by 36
  24. 2015On Graduated Optimization for Stochastic Non-Convex Problemscited by 34
  25. 2022MetaBalance: Improving Multi-Task Recommendations via Adapting Gradient Magnitudes of Auxiliary Taskscited by 34
  26. 2023Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithmscited by 34
  27. 2023Dual-Balancing for Multi-Task Learningcited by 27
  28. 2016Optimal Black-Box Reductions Between Optimization Objectivescited by 21
  29. 2023Scalarization for Multi-Task and Multi-Domain Learning at Scalecited by 19
  30. 2016Frank-Wolfe Algorithms for Saddle Point Problemscited by 18
  31. 2020MTAdam: Automatic Balancing of Multiple Training Loss Termscited by 13
  32. 2023Continuation Path Learning for Homotopy Optimizationcited by 12
  33. 2023Improvable Gap Balancing for Multi-Task Learningcited by 9
  34. 2016Homotopy Smoothing for Non-Smooth Problems with Lower Complexity than O(1/ε)cited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 19; 2026: 26.

54 papers.

Sort:

By area:

  1. 2025PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Modelcited by 28
  2. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 18
  3. 2024Few for Many: Tchebycheff Set Scalarization for Many-Objective Optimizationcited by 16
  4. 2024Traversing Pareto Optimal Policies: Provably Efficient Multi-Objective Reinforcement Learningcited by 15
  5. 2025Multi-objective Large Language Model Alignment with Hierarchical Expertscited by 13
  6. 2024Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusioncited by 11
  7. 2024Test-Time Alignment via Hypothesis Reweightingcited by 11
  8. 2024LibMOON: A Gradient-based MultiObjective OptimizatioN Library in PyTorchcited by 10
  9. 2024FERERO: A Flexible Framework for Preference-Guided Multi-Objective Learningcited by 9
  10. 2025Balanced Rate-Distortion Optimization in Learned Image Compressioncited by 5
  11. 2025AReUReDi: Annealed Rectified Updates for Refining Discrete Flows with Multi-Objective Guidancecited by 5
  12. 2024radarODE-MTL: A Multi-Task Learning Framework with Eccentric Gradient Alignment for Robust Radar-Based ECG Reconstructioncited by 4
38 more
  1. 2024Adaptive Online Mirror Descent for Tchebycheff Scalarization in Multi-Objective Learningcited by 4
  2. 2024LiRA: Light-Robust Adversary for Model-based Reinforcement Learning in Real Worldcited by 3
  3. 2025One Model for All Tasks: Leveraging Efficient World Models in Multi-Task Planningcited by 3
  4. 2025MOBO-OSD: Batch Multi-Objective Bayesian Optimization via Orthogonal Search Directionscited by 3
  5. 2026UniARM: Towards a Unified Autoregressive Reward Model for Multi-Objective Test-Time Alignmentcited by 3
  6. 2026DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Discrete Diffusion Modelscited by 3
  7. 2025Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learningcited by 2
  8. 2025FoMEMO: Towards Foundation Models for Expensive Multi-objective Optimizationcited by 2
  9. 2025SPREAD: Sampling-based Pareto front Refinement via Efficient Adaptive Diffusioncited by 2
  10. 2025NTKMTL: Mitigating Task Imbalance in Multi-Task Learning from Neural Tangent Kernel Perspectivecited by 2
  11. 2026GraphAllocBench: A Flexible Benchmark for Preference-Conditioned Multi-Objective Policy Learningcited by 2
  12. 2026Uncovering Cross-Objective Interference in Multi-Objective Alignmentcited by 2
  13. 2026How to make the most of your masked language model for protein engineeringcited by 2
  14. 2026A Reward-Free Viewpoint on Multi-Objective Reinforcement Learningcited by 2
  15. 2026Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learningcited by 2
  16. 2025MOS-Attack: A Scalable Multi-objective Adversarial Attack Frameworkcited by 1
  17. 2025Learning Pareto manifolds in high dimensions: How can regularization help?cited by 1
  18. 2025Measuring Fine-Grained Relatedness in Multitask Learning via Data Attributioncited by 1
  19. 2025A Framework for Controllable Multi-objective Learning with Annealed Stein Variational Hypernetworkscited by 1
  20. 2025EndoARSS: Adapting Spatially-Aware Foundation Model for Efficient Activity Recognition and Semantic Segmentation in Endoscopic Surgerycited by 1
  21. 2026Auto-Configured Networks for Multi-Scale Multi-Output Time-Series Forecastingcited by 1
  22. 2026A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Setscited by 1
  23. 2026A Tale of Two Problems: Multi-Task Bilevel Learning Meets Equality Constrained Multi-Objective Optimizationcited by 1
  24. 2026Controllability in preference-conditioned multi-objective reinforcement learningcited by 1
  25. 2026MOCHA: Multi-Objective Chebyshev Annealing for Agent Skill Optimizationcited by 1
  26. 2026Monroe: A Molecular Foundation Model for In-Context Probabilistic Inferencecited by 1
  27. 2025Scalable Min-Max Optimization via Primal-Dual Exact Pareto Optimizationcited by 0
  28. 2025Standardization of Multi-Objective QUBOscited by 0
  29. 2025Balancing Multi-modal Sensor Learning via Multi-objective Optimizationcited by 0
  30. 2026Quality-Diversity Optimization as Multi-Objective Optimizationcited by 0
  31. 2026Post Hoc Extraction of Pareto Fronts for Continuous Controlcited by 0
  32. 2026Pareto-Optimal Offline Reinforcement Learning via Smooth Tchebycheff Scalarizationcited by 0
  33. 2026Adaptive Smooth Tchebycheff Attention for Multi-Objective Policy Optimizationcited by 0
  34. 2026SURF: Steering the Scalarization Weight to Uniformly Traverse the Pareto Frontcited by 0
  35. 2026CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal Representationscited by 0
  36. 2026MAdam: Metric-Aware Multi-Objective Adamcited by 0
  37. 2026Redesigning Regularization for Effective Policy Smoothingcited by 0
  38. 2026Multi-Objective Exploration and Preference Optimization via Mutual Informationcited by 0

50 most cited of 54 All 54 (JSON)