mlsanityData to 14 Oct 2026

Enabling Pareto-Stationarity Exploration in Multi-Objective Reinforcement Learning: A Multi-Objective Weighted-Chebyshev Actor-Critic Approach

arXiv 2507.21397 ↗First posted 29 Jul 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

In many multi-objective reinforcement learning (MORL) applications, being able to systematically explore the Pareto-stationary solutions under multiple non-convex reward objectives with theoretical finite-time sample complexity guarantee is an important and yet under-explored problem. This motivates us to take the first step and fill the important gap in MORL. Specifically, in this paper, we propose a ulineMulti-ulineObjective weighted-ulineCHebyshev ulineActor-critic (MOCHA) algorithm for MORL, which judiciously integrates the weighted-Chebychev (WC) and actor-critic framework to enable Pareto-stationarity exploration systematically with finite-time sample complexity guarantee. Sample complexity result of MOCHA algorithm reveals an interesting dependency on pmin in finding an ε-Pareto-stationary solution, where pmin denotes the minimum entry of a given weight vector p in WC-scarlarization. By carefully choosing learning rates, the sample complexity for each exploration can be tildeO(ε-2). Furthermore, simulation studies on a large KuaiRand offline dataset, show that the performance of MOCHA algorithm significantly outperforms other baseline MORL approaches.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 16 of its 32 references that are in the map. First posted on arXiv 29 Jul 2025.

Builds on

16 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Traversing Pareto Optimal Policies: Provably Efficient Multi-Objective Reinforcement Learning24 Jul
  2. 2024Finite-Time Convergence and Sample Complexity of Actor-Critic Multi-Objective Reinforcement Learning5 May

Other references · 14

  1. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  2. 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 256
  3. 2019Pareto Multi-Task Learningcited by 224
  4. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 194
  5. 2019Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systemscited by 90
  6. 2020Random Hypervolume Scalarizations for Provable Multi-Objective Black Box Optimizationcited by 60
  7. 2019The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learningcited by 46
  8. 2023Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithmscited by 34
  9. 2020Recent Theoretical Advances in Non-Convex Optimizationcited by 30
  10. 2023Two-Stage Constrained Actor-Critic for Short Video Recommendationcited by 16
  11. 2024Few for Many: Tchebycheff Set Scalarization for Many-Objective Optimizationcited by 16
  12. 2023Federated Multi-Objective Learningcited by 12
2 more
  1. 2021Theoretical Guarantees of Fictitious Discount Algorithms for Episodic Reinforcement Learning and Global Convergence of Policy Gradient Methodscited by 7
  2. 2024Multi-Objective LQR with Linear Scalarizationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024C-MORL: Multi-Objective Reinforcement Learning through Efficient Discovery of Pareto Front
  2. 2026PA2D-MORL: Pareto Ascent Directional Decomposition based Multi-Objective Reinforcement Learning
  3. 2024Theoretical Study of Conflict-Avoidant Multi-Objective Reinforcement Learning
  4. 2024UCB-driven Utility Function Search for Multi-objective Reinforcement Learning
  5. 2025Limitations of Scalarisation in MORL: A Comparative Study in Discrete Environments
  6. 2026Deterministic Pareto-Optimal Policy Synthesis for Multi-Objective Reinforcement Learning
  7. 2026A Reward-Free Viewpoint on Multi-Objective Reinforcement Learning
  8. 2025Benchmarking MOEAs for solving continuous multi-objective RL problems