mlsanityData to 14 Oct 2026

Aligned Multi Objective Optimization

arXiv 2502.14096 ↗First posted 19 Feb 2025cs.LG math.OCcited by 3

Not on the map. Why?

Abstract (arXiv)

To date, the multi-objective optimization literature has mainly focused on conflicting objectives, studying the Pareto front, or requiring users to balance tradeoffs. Yet, in machine learning practice, there are many scenarios where such conflict does not take place. Recent findings from multi-task learning, reinforcement learning, and LLMs training show that diverse related tasks can enhance performance across objectives simultaneously. Despite this evidence, such phenomenon has not been examined from an optimization perspective. This leads to a lack of generic gradient-based methods that can scale to scenarios with a large number of related objectives. To address this gap, we introduce the Aligned Multi-Objective Optimization framework, propose new algorithms for this setting, and provide theoretical guarantees of their superior performance compared to naive approaches.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 19 Feb 2025.

Builds on

40 of 59 references.

References · 40

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2021Convex optimizationcited by 2,515
  5. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  6. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  7. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  8. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  9. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 834
  10. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  11. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  12. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
28 more
  1. 2023Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-trainingcited by 265
  2. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
  3. 2022Multi-Task Learning as a Bargaining Gamecited by 215
  4. 2020ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learningcited by 202
  5. 2013Optimization, Learning, and Games with Predictable Sequencescited by 188
  6. 2019Online Meta-Learningcited by 137
  7. 2020Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergencecited by 137
  8. 2017Multi-fidelity Bayesian Optimisation with Continuous Approximationscited by 121
  9. 2020Learning to Utilize Shaping Rewards: A New Approach of Reward Shapingcited by 114
  10. 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 101
  11. 2018Last-Iterate Convergence: Zero-Sum Games and Constrained Min-Max Optimizationcited by 98
  12. 2019Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuningcited by 93
  13. 2021Fast Policy Extragradient Methods for Competitive Games with Entropy Regularizationcited by 63
  14. 2019Revisiting the Polyak step sizecited by 55
  15. 2018A General Framework for Multi-fidelity Bayesian Optimization with Gaussian Processescited by 52
  16. 2019MultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learningcited by 45
  17. 2019Discovery of Useful Questions as Auxiliary Taskscited by 45
  18. 2022Dynamics of SGD with Stochastic Polyak Stepsizes: Truly Adaptive Variants and Convergence to Exact Solutioncited by 40
  19. 2024Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generationcited by 36
  20. 2024Robust Multi-Task Learning with Excess Riskscited by 24
  21. 2016The Multi-fidelity Multi-armed Banditcited by 22
  22. 2020Deep Multi-Fidelity Active Learning of High-dimensional Outputscited by 22
  23. 2022Batch Multi-Fidelity Active Learning with Budget Constraintscited by 20
  24. 2021Stochastic Polyak Stepsize with a Moving Targetcited by 19
  25. 2023Generalized Polyak Step Size for First Order Optimization with Momentumcited by 15
  26. 2020Accelerating Reinforcement Learning for Reaching using Continuous Curriculum Learningcited by 13
  27. 2024Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learningcited by 11
  28. 2019Dynamic Subgoal-based Exploration via Bayesian Optimizationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Imbalanced Gradients in RL Post-Training of Multi-Task LLMscited by 20
  2. 2026Uncovering Cross-Objective Interference in Multi-Objective Alignmentcited by 2
  3. 2025Simple Optimizers for Convex Aligned Multi-Objective Optimizationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026A Better Spur Should Start From Each Objective
  2. 2025Self-Improvement Towards Pareto Optimality: Mitigating Preference Conflicts in Multi-Objective Alignment
  3. 2024Multi-objective Deep Learning: Taxonomy and Survey of the State of the Art
  4. 2025Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting
  5. 2025REWARD CONSISTENCY: Improving Multi-Objective Alignment from a Data-Centric Perspective
  6. 2024Safe and Balanced: A Framework for Constrained Multi-Objective Reinforcement Learning
  7. 2026Pareto-Conditioned Diffusion Models for Offline Multi-Objective Optimization
  8. 2025Preference-Guided Diffusion for Multi-Objective Offline Optimization