mlsanityData to 14 Oct 2026

FinRL: Deep Reinforcement Learning Framework to Automate Trading in Quantitative Finance

arXiv 2111.09395 ↗First posted 7 Nov 2021q-fin.TR cs.LGcited by 58

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) has been envisioned to have a competitive edge in quantitative finance. However, there is a steep development curve for quantitative traders to obtain an agent that automatically positions to win in the market, namely textitto decide where to trade, at what price and textitwhat quantity, due to the error-prone programming and arduous debugging. In this paper, we present the first open-source framework textitFinRL as a full pipeline to help quantitative traders overcome the steep learning curve. FinRL is featured with simplicity, applicability and extensibility under the key principles, textitfull-stack framework, customization, reproducibility and textithands-on tutoring. Embodied as a three-layer architecture with modular structures, FinRL implements fine-tuned state-of-the-art DRL algorithms and common reward functions, while alleviating the debugging workloads. Thus, we help users pipeline the strategy design at a high turnover rate. At multiple levels of time granularity, FinRL simulates various markets as training environments using historical data and live trading APIs. Being highly extensible, FinRL reserves a set of user-import interfaces and incorporates trading constraints such as market friction, market liquidity and investor's risk-aversion. Moreover, serving as practitioners' stepping stones, typical trading tasks are provided as step-by-step tutorials, e.g., stock trading, portfolio allocation, cryptocurrency trading, etc.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Builds on 18 of its 53 references that are in the map. First posted on arXiv 7 Nov 2021.

Builds on

18 of 53 references.

References · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2017A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problemcited by 128
  6. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 122
  7. 2018Deep Hedgingcited by 108
  8. 2019Deep Reinforcement Learning for Tradingcited by 73
  9. 2018Practical Deep Reinforcement Learning Approach for Stock Tradingcited by 49
  10. 2019Experience Replay Optimizationcited by 48
  11. 2019Get Real: Realism Metrics for Robust Limit Order Book Market Simulationscited by 41
  12. 2019Reinforcement Learning for Market Making in a Multi-agent Dealer Marketcited by 39
6 more
  1. 2016Cryptocurrency Portfolio Management with Deep Reinforcement Learningcited by 32
  2. 2018Financial Trading as a Game: A Deep Reinforcement Learning Approachcited by 22
  3. 2019Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysiscited by 15
  4. 2017TensorLayer: A Versatile Library for Efficient Deep Learning Developmentcited by 6
  5. 2020Risk-Sensitive Reinforcement Learning: a Martingale Approach to Reward Uncertaintycited by 5
  6. 2020Sentiment and Knowledge Based Algorithmic Trading with Deep Reinforcement Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 3; 2023: 14; 2024: 10; 2025: 18; 2026: 11.

58 papers.

Sort:

By area:

  1. 2023FinGPT: Open-Source Financial Large Language Modelscited by 253
  2. 2023FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Designcited by 164
  3. 2024FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Makingcited by 113
  4. 2021Recent Advances in Reinforcement Learning in Financecited by 110
  5. 2023FinGPT: Democratizing Internet-scale Data for Financial Large Language Modelscited by 67
  6. 2024Aligning Large Language Models with Representation Editing: A Control Perspectivecited by 64
  7. 2025AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decaycited by 42
  8. 2025FinTSB: A Comprehensive and Practical Benchmark for Financial Time Series Forecastingcited by 24
  9. 2025FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning for Financial Tradingcited by 23
  10. 2025Can LLM-based Financial Investing Strategies Outperform the Market in Long Run?cited by 20
  11. 2022Efficient Integration of Multi-Order Dynamics and Internal Dynamics in Stock Movement Predictioncited by 18
  12. 2023Dynamic Datasets and Market Environments for Financial Reinforcement Learningcited by 15
38 more
  1. 2025Cognitive Alpha Mining via LLM-Driven Code-Based Evolutioncited by 15
  2. 2023CFGPT: Chinese Financial Assistant with Large Language Modelcited by 11
  3. 2023Bridging the Sim-to-Real Gap from the Information Bottleneck Perspectivecited by 10
  4. 2021FinRL-Meta: A Universe of Near-Real Market Environments for Data-Driven Deep Reinforcement Learning in Quantitative Financecited by 7
  5. 2023Optimizing Trading Strategies in Quantitative Markets using Multi-Agent Reinforcement Learningcited by 7
  6. 2024Can we hop in general? A discussion of benchmark selection and design using the Hopper environmentcited by 7
  7. 2024In-Trajectory Inverse Reinforcement Learning: Learn Incrementally Before An Ongoing Trajectory Terminatescited by 7
  8. 2023Adaptive Liquidity Provision in Uniswap V3 with Deep Reinforcement Learningcited by 6
  9. 2024Combining Transformer based Deep Reinforcement Learning with Black-Litterman Model for Portfolio Optimizationcited by 6
  10. 2024MILLION: A General Multi-Objective Framework with Controllable Risk for Portfolio Managementcited by 6
  11. 2022Deep Reinforcement Learning for Cryptocurrency Trading: Practical Approach to Address Backtest Overfittingcited by 5
  12. 2023XuanCe: A Comprehensive and Unified Deep Reinforcement Learning Librarycited by 5
  13. 2024Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Informationcited by 5
  14. 2025Are Large Language Models Good In-context Learners for Financial Sentiment Analysis?cited by 5
  15. 2024Explainable Post hoc Portfolio Management Financial Policy of a Deep Reinforcement Learning agentcited by 3
  16. 2025Gym4ReaL: A Suite for Benchmarking Real-World Reinforcement Learningcited by 3
  17. 2025Language Model Guided Reinforcement Learning in Quantitative Tradingcited by 3
  18. 2023Deep Reinforcement Learning for ESG financial portfolio managementcited by 2
  19. 2025A novel multi-agent dynamic portfolio optimization learning system based on hierarchical deep reinforcement learningcited by 2
  20. 2025Reinforcement-Learning Portfolio Allocation with Dynamic Embedding of Market Informationcited by 2
  21. 2026PredictionMarketBench: A SWE-bench-Style Framework for Backtesting Trading Agents on Prediction Marketscited by 2
  22. 2023Agent Performing Autonomous Stock Trading under Good and Bad Situationscited by 1
  23. 2023Towards Explainable Strategy Templates using NLP Transformerscited by 1
  24. 2024Reinforcement Learning Pair Trading: A Dynamic Scaling approachcited by 1
  25. 2024Hierarchical Reinforced Trader (HRT): A Bi-Level Approach for Optimizing Stock Selection and Executioncited by 1
  26. 2025MTS: A Deep Reinforcement Learning Portfolio Management Framework with Time-Awareness and Short-Sellingcited by 1
  27. 2022Design Process is a Reinforcement Learning Problemcited by 0
  28. 2023Navigation of micro-robot swarms for targeted delivery using reinforcement learningcited by 0
  29. 2023An Ensemble Method of Deep Reinforcement Learning for Automated Cryptocurrency Tradingcited by 0
  30. 2025Revisiting Ensemble Methods for Stock Trading and Crypto Trading Tasks at ACM ICAIF FinRL Contest 2023-2024cited by 0
  31. 2025MIGT: Memory Instance Gated Transformer Framework for Financial Portfolio Managementcited by 0
  32. 2025On the Mistaken Assumption of Interchangeable Deep Reinforcement Learning Implementationscited by 0
  33. 2025Your Offline Policy is Not Trustworthy: Bilevel Reinforcement Learning for Sequential Portfolio Optimizationcited by 0
  34. 2025When a Reinforcement Learning Agent Encounters Unknown Unknownscited by 0
  35. 2025Directly Learning Stock Trading Strategies Through Profit Guided Loss Functionscited by 0
  36. 2025Hi-DARTS: Hierarchical Dynamically Adapting Reinforcement Trading Systemcited by 0
  37. 2026Counterfactually Safe Reinforcement Learningcited by 0
  38. 2026Regime-Adaptive Continual Learning for Portfolio Managementcited by 0

50 most cited of 58 All 58 (JSON)