mlsanityData to 7 Oct 2026

Recent Advances in Reinforcement Learning in Finance

arXiv 2112.04553 ↗First posted 8 Dec 2021q-fin.MF cs.LG q-fin.CPcited by 102

Not on the map. Why?

Abstract (arXiv)

The rapid changes in the finance industry due to the increasing amount of data have revolutionized the techniques on data processing and data analysis and brought new theoretical and computational challenges. In contrast to classical stochastic control theory and other analytical approaches for solving financial decision-making problems that heavily reply on model assumptions, new developments from reinforcement learning (RL) are able to make full use of the large amount of financial data with fewer model assumptions and to improve decisions in complex financial environments. This survey paper aims to review the recent developments and use of RL approaches in finance. We give an introduction to Markov decision processes, which is the setting for many of the commonly used RL approaches. Various algorithms are then introduced with a focus on value and policy based methods that do not require any model assumptions. Connections are made with neural networks to extend the framework to encompass deep RL algorithms. Our survey concludes by discussing the application of these RL algorithms in a variety of decision-making problems in finance, including optimal execution, portfolio optimization, option pricing and hedging, market making, smart order routing, and robo-advising.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). Builds on 111 of its 278 references that are in the map. First posted on arXiv 8 Dec 2021.

Builds on

111 of 278 references.

References · 111

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2015Trust Region Policy Optimizationcited by 2,503
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  8. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  9. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  10. 2017Constrained Policy Optimizationcited by 928
  11. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  12. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
38 more
  1. 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
  2. 2018Is Q-learning Provably Efficient?cited by 485
  3. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  4. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 285
  5. 2019A Theory of Regularized Markov Decision Processescited by 279
  6. 2020On the Global Convergence Rates of Softmax Policy Gradient Methodscited by 258
  7. 2013(More) Efficient Reinforcement Learning via Posterior Samplingcited by 233
  8. 2018A Finite Time Analysis of Temporal Difference Learning With Linear Function Approximationcited by 225
  9. 2019A Theoretical Analysis of Deep Q-Learningcited by 212
  10. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 211
  11. 2018An Introduction to Deep Reinforcement Learningcited by 201
  12. 2019Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Boundcited by 199
  13. 2013Maxout Networkscited by 186
  14. 2016Sample Efficient Actor-Critic with Experience Replaycited by 185
  15. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 175
  16. 2020Fast Global Convergence of Natural Policy Gradient Methods with Entropy Regularizationcited by 170
  17. 2019A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptationcited by 166
  18. 2015Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approachcited by 162
  19. 2019Neural Policy Gradient Methods: Global Optimality and Rates of Convergencecited by 157
  20. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 149
  21. 2020Provably Efficient Safe Exploration via Primal-Dual Policy Optimizationcited by 131
  22. 2013Risk-sensitive Reinforcement Learningcited by 129
  23. 2019Model-Based Reinforcement Learning with a Generative Model is Minimax Optimalcited by 124
  24. 2019Learning with Good Feature Representations in Bandits and in RL with a Generative Modelcited by 124
  25. 2021Provably Efficient Reinforcement Learning with Linear Function Approximation Under Adaptivity Constraintscited by 124
  26. 2017A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problemcited by 123
  27. 2019Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policiescited by 117
  28. 2020Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizabilitycited by 114
  29. 2019Adaptive Trust Region Policy Optimization: Global Convergence and Faster Rates for Regularized MDPscited by 113
  30. 2020Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Modelcited by 108
  31. 2019Quant GANs: Deep Generation of Financial Time Seriescited by 107
  32. 2018Deep Hedgingcited by 106
  33. 2015Sample Complexity of Episodic Fixed-Horizon Reinforcement Learningcited by 105
  34. 2018Stochastic Variance-Reduced Policy Gradientcited by 103
  35. 2021Non-stationary Reinforcement Learning without Prior Knowledge: An Optimal Black-box Approachcited by 98
  36. 2019Batched Multi-armed Bandits Problemcited by 97
  37. 2019Finite-Sample Analysis for SARSA with Linear Function Approximationcited by 85
  38. 2020Sample Efficient Reinforcement Learning with REINFORCEcited by 85

50 most cited of 111 All 111 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 17; 2024: 28; 2025: 31; 2026: 16.

102 papers.

Sort:

By area:

  1. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  2. 2024A Survey of Constraint Formulations in Safe Reinforcement Learningcited by 42
  3. 2024AuctionNet: A Novel Benchmark for Decision-Making in Large-Scale Gamescited by 26
  4. 2025FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Tradingcited by 23
  5. 2023Recent Developments in Machine Learning Methods for Stochastic Control and Gamescited by 19
  6. 2023Dynamic Datasets and Market Environments for Financial Reinforcement Learningcited by 15
  7. 2022Deep Hedging: Continuous Reinforcement Learning for Hedging of General Portfolios across Multiple Risk Aversionscited by 13
  8. 2024VQC-Based Reinforcement Learning with Data Re-uploading: Performance and Trainabilitycited by 12
  9. 2023Fast Policy Learning for Linear Quadratic Control with Entropy Regularizationcited by 11
  10. 2024The Evolution of Reinforcement Learning in Quantitative Finance: A Surveycited by 11
  11. 2022MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimizationcited by 10
  12. 2022Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learningcited by 9
38 more
  1. 2025A Primer on Quantum Machine Learningcited by 9
  2. 2023Risk-sensitive Markov Decision Process and Learning under General Utility Functionscited by 7
  3. 2024Adaptive Sampling of k-Space in Magnetic Resonance for Rapid Pathology Predictioncited by 7
  4. 2023PILLOW: Enhancing Efficient Instruction Fine-tuning via Prompt Matchingcited by 6
  5. 2024No-Regret Reinforcement Learning in Smooth MDPscited by 6
  6. 2024Combining Transformer based Deep Reinforcement Learning with Black-Litterman Model for Portfolio Optimizationcited by 6
  7. 2024Fast Deep Hedging with Second-Order Optimizationcited by 6
  8. 2025Finite-Sample Analysis of Policy Evaluation for Robust Average Reward Reinforcement Learningcited by 6
  9. 2022Deep Reinforcement Learning for Cryptocurrency Trading: Practical Approach to Address Backtest Overfittingcited by 5
  10. 2024Projection by Convolution: Optimal Sample Complexity for Reinforcement Learning in Continuous-Space MDPscited by 5
  11. 2024Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learningcited by 5
  12. 2024Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and qualitycited by 5
  13. 2024Deep Reinforcement Learning and Mean-Variance Strategies for Responsible Portfolio Optimizationcited by 4
  14. 2024Local Linearity: the Key for No-regret Reinforcement Learning in Continuous MDPscited by 4
  15. 2025Wasserstein Adaptive Value Estimation for Actor-Critic Reinforcement Learningcited by 4
  16. 2023Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Marketscited by 3
  17. 2023Contextual Dynamic Pricing with Strategic Buyerscited by 3
  18. 2024Revisiting Experience Replayable Conditionscited by 3
  19. 2024Explainable Post hoc Portfolio Management Financial Policy of a Deep Reinforcement Learning agentcited by 3
  20. 2022Markov Decision Processes under Model Uncertaintycited by 2
  21. 2022Applications of Reinforcement Learning in Finance -- Trading with a Double Deep Q-Networkcited by 2
  22. 2023Systematic Review on Reinforcement Learning in the Field of Fintechcited by 2
  23. 2023Automatic Deduction Path Learning via Reinforcement Learning with Environmental Correctioncited by 2
  24. 2024Formal Ethical Obligations in Reinforcement Learning Agents: Verification and Policy Updatescited by 2
  25. 2024Beyond Simple Sum of Delayed Rewards: Non-Markovian Reward Modeling for Reinforcement Learningcited by 2
  26. 2025A novel multi-agent dynamic portfolio optimization learning system based on hierarchical deep reinforcement learningcited by 2
  27. 2025Reinforcement-Learning Portfolio Allocation with Dynamic Embedding of Market Informationcited by 2
  28. 2025Predicting Liquidity-Aware Bond Yields using Causal GANs and Deep Reinforcement Learning with LLM Evaluationcited by 2
  29. 2025Risk-sensitive Reinforcement Learning Based on Convex Scoring Functionscited by 2
  30. 2025A Provable Approach for End-to-End Safe Reinforcement Learningcited by 2
  31. 2025Distributional Adversarial Attacks and Training in Deep Hedgingcited by 2
  32. 2025Robust Market Making: To Quote, or not To Quotecited by 2
  33. 2025Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Surveycited by 2
  34. 2026Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimizationcited by 2
  35. 2026Learning in Markov Decision Processes with Exogenous Dynamicscited by 2
  36. 2022On Parametric Optimal Execution and Machine Learning Surrogatescited by 1
  37. 2022Risk-Aware Linear Bandits: Theory and Applications in Smart Order Routingcited by 1
  38. 2022Predictive Crypto-Asset Automated Market Making Architecture for Decentralized Finance using Deep Reinforcement Learningcited by 1

50 most cited of 102 All 102 (JSON)