# Regret-Optimized Portfolio Enhancement through Deep Reinforcement Learning and Future Looking Rewards

arXiv: https://arxiv.org/abs/2502.02619 · HTML: https://mlsanity.com/paper/2502.02619/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 40 references that are in the map.
- First posted on arXiv 4 Feb 2025.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem](https://mlsanity.com/paper/1706.10059/) (2017)
- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [Smart Predict-and-Optimize for Hard Combinatorial Optimization Problems](https://mlsanity.com/paper/1911.10092/) (2019)
- [Curriculum Learning in Deep Neural Networks for Financial Forecasting](https://mlsanity.com/paper/1904.12887/) (2019)
- [Bridging the gap between Markowitz planning and deep reinforcement learning](https://mlsanity.com/paper/2010.09108/) (2020)
- [Detecting and adapting to crisis pattern with context based Deep Reinforcement Learning](https://mlsanity.com/paper/2009.07200/) (2020)
- [Combining Reinforcement Learning and Inverse Reinforcement Learning for Asset Allocation Recommendations](https://mlsanity.com/paper/2201.01874/) (2022)
- [A Modified CTGAN-Plus-Features Based Method for Optimal Asset Allocation](https://mlsanity.com/paper/2302.02269/) (2023)
- [Multi-Objective reward generalization: Improving performance of Deep Reinforcement Learning for applications in single-asset trading](https://mlsanity.com/paper/2203.04579/) (2022)
- [Evaluation of Deep Reinforcement Learning Algorithms for Portfolio Optimisation](https://mlsanity.com/paper/2307.07694/) (2023)

## Built on by (most cited first, 1 of 1)

- [GIFT: LLM-Guided State-Reward Interface for Financial Reinforcement Learning](https://mlsanity.com/paper/2606.08450/) (2026)

## Abstract (arXiv)

This paper introduces a novel agent-based approach for enhancing existing portfolio strategies using Proximal Policy Optimization (PPO). Rather than focusing solely on traditional portfolio construction, our approach aims to improve an already high-performing strategy through dynamic rebalancing driven by PPO and Oracle agents. Our target is to enhance the traditional 60/40 benchmark (60% stocks, 40% bonds) by employing the Regret-based Sharpe reward function. To address the impact of transaction fee frictions and prevent signal loss, we develop a transaction cost scheduler. We introduce a future-looking reward function and employ synthetic data training through a circular block bootstrap method to facilitate the learning of generalizable allocation strategies. We focus on two key evaluation measures: return and maximum drawdown. Given the high stochasticity of financial markets, we train 20 independent agents each period and evaluate their average performance against the benchmark. Our method not only enhances the performance of the existing portfolio strategy through strategic rebalancing but also demonstrates strong results compared to other baselines.