# Balancing Profit, Risk, and Sustainability for Portfolio Management

arXiv: https://arxiv.org/abs/2207.02134 · HTML: https://mlsanity.com/paper/2207.02134/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 27 references that are in the map.
- First posted on arXiv 6 Jun 2022.

## Builds on (most cited first, 11 of 11)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)
- [A Study on Overfitting in Deep Reinforcement Learning](https://mlsanity.com/paper/1804.06893/) (2018)
- [Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/1712.06567/) (2017)
- [A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem](https://mlsanity.com/paper/1706.10059/) (2017)
- [FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance](https://mlsanity.com/paper/2011.09607/) (2020)
- [Deep Reinforcement Learning using Genetic Algorithm for Parameter Optimization](https://mlsanity.com/paper/1905.04100/) (2019)
- [Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics](https://mlsanity.com/paper/2004.01509/) (2020)
- [Clustering in Recurrent Neural Networks for Micro-Segmentation using Spending Personality](https://mlsanity.com/paper/2109.09425/) (2021)
- [Deep Reinforcement Learning for Stock Portfolio Optimization](https://mlsanity.com/paper/2012.06325/) (2020)

## Built on by (most cited first, 2 of 2)

- [Portfolio Optimization: A Comparative Study](https://mlsanity.com/paper/2307.05048/) (2023)
- [Beyond ESG Scores: Learning Dynamic Constraints for Sequential Portfolio Optimization](https://mlsanity.com/paper/2605.09310/) (2026)

## Abstract (arXiv)

Stock portfolio optimization is the process of continuous reallocation of funds to a selection of stocks. This is a particularly well-suited problem for reinforcement learning, as daily rewards are compounding and objective functions may include more than just profit, e.g., risk and sustainability. We developed a novel utility function with the Sharpe ratio representing risk and the environmental, social, and governance score (ESG) representing sustainability. We show that a state-of-the-art policy gradient method - multi-agent deep deterministic policy gradients (MADDPG) - fails to find the optimum policy due to flat policy gradients and we therefore replaced gradient descent with a genetic algorithm for parameter optimization. We show that our system outperforms MADDPG while improving on deep Q-learning approaches by allowing for continuous action spaces. Crucially, by incorporating risk and sustainability criteria in the utility function, we improve on the state-of-the-art in reinforcement learning for portfolio optimization; risk and sustainability are essential in any modern trading strategy and we propose a system that does not merely report these metrics, but that actively optimizes the portfolio to improve on them.