PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets

arXiv 2302.00586 ↗First posted 14 Jan 2023q-fin.TR cs.AI cs.LGcited by 5

On the map: Mixture-of-experts → MoE for time series and graphs

Abstract (arXiv)

The financial markets, which involve more than 90 trillion market capitals, attract the attention of innumerable investors around the world. Recently, reinforcement learning in financial markets (FinRL) has emerged as a promising direction to train agents for making profitable investment decisions. However, the evaluation of most FinRL methods only focuses on profit-related measures and ignores many critical axes, which are far from satisfactory for financial practitioners to deploy these methods into real-world financial markets. Therefore, we introduce PRUDEX-Compass, which has 6 axes, i.e., Profitability, Risk-control, Universality, Diversity, rEliability, and eXplainability, with a total of 17 measures for a systematic evaluation. Specifically, i) we propose AlphaMix+ as a strong FinRL baseline, which leverages mixture-of-experts (MoE) and risk-sensitive approaches to make diversified risk-aware investment decisions, ii) we evaluate 8 FinRL methods in 4 long-term real-world datasets of influential financial markets to demonstrate the usage of our PRUDEX-Compass, iii) PRUDEX-Compass together with 4 real-world datasets, standard implementation of 8 FinRL methods and a portfolio management environment is released as public resources to facilitate the design and comparison of new FinRL methods. We hope that PRUDEX-Compass can not only shed light on future FinRL research to prevent untrustworthy results from stagnating FinRL into successful industry deployment but also provide a new challenging algorithm evaluation scenario for the reinforcement learning (RL) community.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in MoE for time series and graphs, one of 12 directions in Mixture-of-experts. Builds on 38 of its 104 references that are in the map. First posted on arXiv 14 Jan 2023.

Builds on

38 of 104 references.

References · 38

2017Proximal Policy Optimization Algorithmscited by 14,441
2017A Unified Approach to Interpreting Model Predictionscited by 6,488
2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
2018Datasheets for Datasetscited by 1,371
2018Model Cards for Model Reportingcited by 1,068
2020Meta-Learning in Neural Networks: A Surveycited by 944
2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
2017Deep Reinforcement Learning that Matterscited by 727
2016Deep Exploration via Bootstrapped DQNcited by 518

26 more

2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 318
2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 240
2019Explainable Machine Learning in Deploymentcited by 210
2020SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learningcited by 150
2020Hyperparameter Ensembles for Robustness and Uncertainty Quantificationcited by 149
2022OpenXAI: Towards a Transparent Evaluation of Model Explanationscited by 125
2017A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problemcited by 123
2020Effective Diversity in Population Based Reinforcement Learningcited by 116
2020FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Financecited by 86
2019Deep Reinforcement Learning for Tradingcited by 73
2020DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correctioncited by 71
2018Diversity-Driven Exploration Strategy for Deep Reinforcement Learningcited by 65
2020Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction Statescited by 55
2019Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attributioncited by 50
2019Measuring the Reliability of Reinforcement Learning Algorithmscited by 49
2018Market Making via Reinforcement Learningcited by 45
2020Evaluating the Performance of Reinforcement Learning Algorithmscited by 35
2021Universal Trading for Order Execution with Oracle Policy Distillationcited by 29
2021CLEVA-Compass: A Continual Learning EValuation Assessment Compass to Promote Research Transparency and Comparabilitycited by 28
2021Reinforcement Learning for Quantitative Tradingcited by 24
2020Deep Stock Trading: A Hierarchical Reinforcement Learning Framework for Portfolio Optimization and Order Executioncited by 20
2021DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunitiescited by 14
2019Automatic Model Monitoring for Data Streamscited by 9

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 4.

5 papers.

2024A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalistcited by 137
2025HedgeAgents: A Balanced-aware Multi-agent Financial Trading Systemcited by 24
2025Mixture of Experts (MoE): A Big Data Perspectivecited by 17
2025QuantAgents: Towards Multi-agent Financial System via Simulated Tradingcited by 8
2025FineFT: Efficient and Risk-Aware Ensemble Reinforcement Learning for Futures Tradingcited by 1

Similar, not cited

Close in wording, no citation either way