mlsanityData to 14 Oct 2026

Joint Optimization of Multi-Objective Reinforcement Learning with Policy Gradient Based Algorithm

arXiv 2105.14125 ↗First posted 28 May 2021cs.LG cs.AI cs.SYcited by 12

Not on the map. Why?

Abstract (arXiv)

Many engineering problems have multiple objectives, and the overall aim is to optimize a non-linear function of these objectives. In this paper, we formulate the problem of maximizing a non-linear concave function of multiple long-term objectives. A policy-gradient based model-free algorithm is proposed for the problem. To compute an estimate of the gradient, a biased estimator is proposed. The proposed algorithm is shown to achieve convergence to within an ε of the global optima after sampling O(fracM4σ2(1-γ)8ε4) trajectories where γ is the discount factor and M is the number of the agents, thus achieving the same dependence on ε as the policy gradient algorithm for the standard reinforcement learning.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 17 of its 37 references that are in the map. First posted on arXiv 28 May 2021.

Builds on

17 of 37 references.

References · 17

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 520
  3. 2018Is Q-learning Provably Efficient?cited by 487
  4. 2020On the Global Convergence Rates of Softmax Policy Gradient Methodscited by 264
  5. 2018Provably Efficient Maximum Entropy Explorationcited by 242
  6. 2019On the Theory of Policy Gradient Methods: Optimality, Approximation, and Distribution Shiftcited by 177
  7. 2019Neural Policy Gradient Methods: Global Optimality and Rates of Convergencecited by 159
  8. 2019Global Optimality Guarantees For Policy Gradient Methodscited by 150
  9. 2013Risk-sensitive Reinforcement Learningcited by 130
  10. 2019Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policiescited by 124
  11. 2020Variational Policy Gradient Method for Reinforcement Learning with General Utilitiescited by 111
  12. 2018Stochastic Variance-Reduced Policy Gradientcited by 106
5 more
  1. 2021On the Convergence and Sample Efficiency of Variance-Reduced Policy Gradient Methodcited by 70
  2. 2019Sample Efficient Policy Gradient Methods with Recursive Variance Reductioncited by 60
  3. 2019An Improved Convergence Analysis of Stochastic Variance-Reduced Policy Gradientcited by 59
  4. 2020L2B: Learning to Balance the Safety-Efficiency Trade-off in Interactive Crowd-aware Robot Navigationcited by 7
  5. 2019Optimal Server Selection for Straggler Mitigationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 2; 2023: 0; 2024: 1; 2025: 2; 2026: 5.

12 papers.

  1. 2021Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysiscited by 26
  2. 2021Concave Utility Reinforcement Learning with Zero-Constraint Violationscited by 19
  3. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 18
  4. 2022Anchor-Changing Regularized Natural Policy Gradient for Multi-Objective Reinforcement Learningcited by 15
  5. 2024On the Global Optimality of Policy Gradient Methods in General Utility Reinforcement Learningcited by 6
  6. 2026Breaking the Bias Barrier in Concave Multi-Objective Reinforcement Learningcited by 2
  7. 2026Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learningcited by 2
  8. 2022Addressing the issue of stochastic environments and local decision-making in multi-objective reinforcement learningcited by 1
  9. 2026Convex Markov Games and Beyond: New Proof of Existence, Characterization and Learning Algorithms for Nash Equilibriacited by 1
  10. 2025Variance Reduced Policy Gradient Method for Multi-Objective Reinforcement Learningcited by 0
  11. 2026Multi-objective Reinforcement Learning With Augmented States Requires Rewards After Deploymentcited by 0
  12. 2026Bias-Controlled Primal-Dual Natural Actor-Critic: Optimal Rates for Constrained Multi-Objective Average-Reward RLcited by 0