# Robo-Advising: Enhancing Investment with Inverse Optimization and Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2105.09264 · HTML: https://mlsanity.com/paper/2105.09264/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 55 references that are in the map.
- First posted on arXiv 19 May 2021.

## Builds on (most cited first, 8 of 8)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Deep Hedging](https://mlsanity.com/paper/1802.03042/) (2018)
- [Generalized Inverse Optimization through Online Learning](https://mlsanity.com/paper/1810.01920/) (2018)
- [Personalized Robo-Advising: Enhancing Investment through Client Interaction](https://mlsanity.com/paper/1911.01391/) (2019)
- [Large scale continuous-time mean-variance portfolio allocation via reinforcement learning](https://mlsanity.com/paper/1907.11718/) (2019)
- [Dirichlet policies for reinforced factor portfolios](https://mlsanity.com/paper/2011.05381/) (2020)
- [Learning Risk Preferences from Investment Portfolios Using Inverse Optimization](https://mlsanity.com/paper/2010.01687/) (2020)

## Built on by (most cited first, 7 of 7)

- [Recent Advances in Reinforcement Learning in Finance](https://mlsanity.com/paper/2112.04553/) (2021)
- [Privacy Amplification via Shuffling for Linear Contextual Bandits](https://mlsanity.com/paper/2112.06008/) (2021)
- [The Evolution of Reinforcement Learning in Quantitative Finance: A Survey](https://mlsanity.com/paper/2408.10932/) (2024)
- [Privacy Preserving Reinforcement Learning for Population Processes](https://mlsanity.com/paper/2406.17649/) (2024)
- [Intelligent Systematic Investment Agent: an ensemble of deep learning and evolutionary strategies](https://mlsanity.com/paper/2203.13125/) (2022)
- [Portfolio Optimization: A Comparative Study](https://mlsanity.com/paper/2307.05048/) (2023)
- [Inference of Utilities and Time Preference in Sequential Decision-Making](https://mlsanity.com/paper/2405.15975/) (2024)

## Abstract (arXiv)

Machine Learning (ML) has been embraced as a powerful tool by the financial industry, with notable applications spreading in various domains including investment management. In this work, we propose a full-cycle data-driven investment robo-advising framework, consisting of two ML agents. The first agent, an inverse portfolio optimization agent, infers an investor's risk preference and expected return directly from historical allocation data using online inverse optimization. The second agent, a deep reinforcement learning (RL) agent, aggregates the inferred sequence of expected returns to formulate a new multi-period mean-variance portfolio optimization problem that can be solved using deep RL approaches. The proposed investment pipeline is applied on real market data from April 1, 2016 to February 1, 2021 and has shown to consistently outperform the S&P 500 benchmark portfolio that represents the aggregate market optimal allocation. The outperformance may be attributed to the the multi-period planning (versus single-period planning) and the data-driven RL approach (versus classical estimation approach).