mlsanityData to 14 Oct 2026

QuantFactor REINFORCE: Mining Steady Formulaic Alpha Factors with Variance-bounded REINFORCE

arXiv 2409.05144 ↗First posted 8 Sep 2024q-fin.CP cs.AI cs.LGcited by 14

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Alpha factor mining aims to discover investment signals from the historical financial market data, which can be used to predict asset returns and gain excess profits. Powerful deep learning methods for alpha factor mining lack interpretability, making them unacceptable in the risk-sensitive real markets. Formulaic alpha factors are preferred for their interpretability, while the search space is complex and powerful explorative methods are urged. Recently, a promising framework is proposed for generating formulaic alpha factors using deep reinforcement learning, and quickly gained research focuses from both academia and industries. This paper first argues that the originally employed policy training method, i.e., Proximal Policy Optimization (PPO), faces several important issues in the context of alpha factors mining. Herein, a novel reinforcement learning algorithm based on the well-known REINFORCE algorithm is proposed. REINFORCE employs Monte Carlo sampling to estimate the policy gradient-yielding unbiased but high variance estimates. The minimal environmental variability inherent in the underlying state transition function, which adheres to the Dirac distribution, can help alleviate this high variance issue, making REINFORCE algorithm more appropriate than PPO. A new dedicated baseline is designed to theoretically reduce the commonly suffered high variance of REINFORCE. Moreover, the information ratio is introduced as a reward shaping mechanism to encourage the generation of steady alpha factors that can better adapt to changes in market volatility. Evaluations on real assets data indicate the proposed algorithm boosts correlation with returns by 3.83%, and a stronger ability to obtain excess returns compared to the latest alpha factors mining methods, which meets the theoretical results well.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 82 references that are in the map. First posted on arXiv 8 Sep 2024.

Builds on

25 of 82 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2021Monte Carlo Tree Search: A Review of Recent Modifications and Applicationscited by 161
  7. 2020FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Financecited by 87
  8. 2020Sample Efficient Reinforcement Learning with REINFORCEcited by 86
  9. 2013The Optimal Reward Baseline for Gradient-Based Reinforcement Learningcited by 85
  10. 2020Qlib: An AI-oriented Quantitative Investment Platformcited by 81
  11. 2022OpenFE: Automated Feature Generation with Expert-level Performancecited by 52
  12. 2021Leveraging Non-uniformity in First-order Non-convex Optimizationcited by 44
13 more
  1. 2023Generating Synergistic Formulaic Alpha Collections via Reinforcement Learningcited by 39
  2. 2021AlphaEvolve: A Learning Framework to Discover Novel Alphas in Quantitative Investmentcited by 32
  3. 2023The Role of Baselines in Policy Gradient Optimizationcited by 25
  4. 2023TD Convergence: An Optimization Perspectivecited by 16
  5. 2023Reducing Idleness in Financial Cloud Services via Multi-objective Evolutionary Reinforcement Learning based Load Balancercited by 14
  6. 2020Evolutionary Reinforcement Learning via Cooperative Coevolutionary Negatively Correlated Searchcited by 8
  7. 2018Adversarial Deep Reinforcement Learning in Portfolio Managementcited by 6
  8. 2019Parallel Exploration via Negatively Correlated Searchcited by 4
  9. 2024SimLOB: Learning Representations of Limited Order Book for Financial Market Simulationcited by 4
  10. 2022Neural Augmented Kalman Filtering with Bollinger Bands for Pairs Tradingcited by 2
  11. 2022Efficient and Scalable Parametric High-Order Portfolios Design via the Skew-t Distributioncited by 1
  12. 2022A Fast Successive QP Algorithm for General Mean-Variance Portfolio Optimizationcited by 1
  13. 2023Combining Reinforcement Learning and Barrier Functions for Adaptive Risk Management in Portfolio Optimizationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 8.

14 papers.

Sort:

By area:

  1. 2025R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimizationcited by 43
  2. 2025AlphaEval: A Comprehensive and Efficient Evaluation Framework for Formula Alpha Miningcited by 11
  3. 2025Learning from Expert Factors: Trajectory-level Reward Shaping for Formulaic Alpha Miningcited by 5
  4. 2026AlphaPROBE: Alpha Mining via Principled Retrieval and On-graph biased evolutioncited by 3
  5. 2026AlphaMemo: Structured Search-Process Memory for Self-Evolving Alpha Mining Agentscited by 3
  6. 2026AQuA: Recursively Self-Improving Quantitative Trading Research Agentscited by 3
  7. 2025Representation Learning of Limit Order Book: A Comprehensive Study and Benchmarkingcited by 2
  8. 2025Deep Reinforcement Learning for Optimal Asset Allocation Using DDPG with TiDEcited by 1
  9. 2025From Observations to Parameters: Detecting Changepoint in Nonlinear Dynamics with Simulation-based Inferencecited by 0
  10. 2026Online Regime-aware Calibration for Black-box Social Simulators via Posterior-assisted Evolutionary Dynamic Optimizationcited by 0
  11. 2026EVOQUANT: Self-Evolving Verifier-Guided Strategy Optimization for Robust Quantitative Tradingcited by 0
  12. 2026AlphaG-OPD: Reliability-Gated Sibling Counterfactuals for On-Policy Distillation in Symbolic Alpha Factor Discoverycited by 0
2 more
  1. 2026AgonAlpha: Autonomous Alpha Discovery via Prompt Economy and Scalable Agentic Searchcited by 0
  2. 2026AlphaRLM: Aligning Language Models for Literature-Grounded Symbolic Factor Discoverycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AlphaForge: A Framework to Mine and Dynamically Combine Formulaic Alpha Factors
  2. 2026From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discovery
  3. 2024Synergistic Formulaic Alpha Generation for Quantitative Trading based on Reinforcement Learning
  4. 2024Alpha2: Discovering Logical Formulaic Alphas using Deep Reinforcement Learning
  5. 2026AlphaPilot: Multi-Turn Tool-Using LLM Agents for Adaptive Alpha Mining with Reinforcement Learning
  6. 2025Navigating the Alpha Jungle: An LLM-Powered MCTS Framework for Formulaic Factor Mining
  7. 2025Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning
  8. 2026AlphaPareto: Formulaic Alpha Discovery with LLM-Guided Multi-Objective Reinforcement Learning