# A Scale-Independent Multi-Objective Reinforcement Learning with Convergence Analysis

arXiv: https://arxiv.org/abs/2302.04179 · HTML: https://mlsanity.com/paper/2302.04179/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 8 Feb 2023.

## Builds on (most cited first, 9 of 9)

- [Maximum a Posteriori Policy Optimisation](https://mlsanity.com/paper/1806.06920/) (2018)
- [A Survey of Multi-Objective Sequential Decision-Making](https://mlsanity.com/paper/1402.0590/) (2014)
- [A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptation](https://mlsanity.com/paper/1908.08342/) (2019)
- [Multi-Objective Deep Reinforcement Learning](https://mlsanity.com/paper/1610.02707/) (2016)
- [Efficient Continuous Pareto Exploration in Multi-Task Learning](https://mlsanity.com/paper/2006.16434/) (2020)
- [A Distributional View on Multi-Objective Policy Optimization](https://mlsanity.com/paper/2005.07513/) (2020)
- [Meta-Learning for Multi-objective Reinforcement Learning](https://mlsanity.com/paper/1811.03376/) (2018)
- [The convergence of the Stochastic Gradient Descent (SGD) : a self-contained proof](https://mlsanity.com/paper/2103.14350/) (2021)
- [Relationship Explainable Multi-objective Optimization Via Vector Value Function Based Reinforcement Learning](https://mlsanity.com/paper/1910.01919/) (2019)

## Abstract (arXiv)

Many sequential decision-making problems need optimization of different objectives which possibly conflict with each other. The conventional way to deal with a multi-task problem is to establish a scalar objective function based on a linear combination of different objectives. However, for the case of having conflicting objectives with different scales, this method needs a trial-and-error approach to properly find proper weights for the combination. As such, in most cases, this approach cannot guarantee an optimal Pareto solution. In this paper, we develop a single-agent scale-independent multi-objective reinforcement learning on the basis of the Advantage Actor-Critic (A2C) algorithm. A convergence analysis is then done for the devised multi-objective algorithm providing a convergence-in-mean guarantee. We then perform some experiments over a multi-task problem to evaluate the performance of the proposed algorithm. Simulation results show the superiority of developed multi-objective A2C approach against the single-objective algorithm.