# Generating Synergistic Formulaic Alpha Collections via Reinforcement Learning

arXiv: https://arxiv.org/abs/2306.12964 · HTML: https://mlsanity.com/paper/2306.12964/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 29 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [A Closer Look at Invalid Action Masking in Policy Gradient Algorithms](https://mlsanity.com/paper/2006.14171/) (2020)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [SymbolicGPT: A Generative Transformer Model for Symbolic Regression](https://mlsanity.com/paper/2106.14131/) (2021)
- [Symbolic Regression via Neural-Guided Genetic Programming Population Seeding](https://mlsanity.com/paper/2111.00053/) (2021)
- [You Impress Me: Dialogue Generation via Mutual Persona Perception](https://mlsanity.com/paper/2004.05388/) (2020)
- [Qlib: An AI-oriented Quantitative Investment Platform](https://mlsanity.com/paper/2009.11189/) (2020)
- [HIST: A Graph-based Framework for Stock Trend Forecasting via Mining Concept-Oriented Shared Information](https://mlsanity.com/paper/2110.13716/) (2021)
- [AlphaEvolve: A Learning Framework to Discover Novel Alphas in Quantitative Investment](https://mlsanity.com/paper/2103.16196/) (2021)
- [REST: Relational Event-driven Stock Trend Forecasting](https://mlsanity.com/paper/2102.07372/) (2021)

## Built on by (most cited first, 30 of 39)

- [Alpha-GPT: Human-AI Interactive Alpha Mining for Quantitative Investment](https://mlsanity.com/paper/2308.00016/) (2023)
- [AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decay](https://mlsanity.com/paper/2502.16789/) (2025)
- [Automate Strategy Finding with LLM in Quant Investment](https://mlsanity.com/paper/2409.06289/) (2024)
- [DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience](https://mlsanity.com/paper/2506.04282/) (2025)
- [Navigating the Alpha Jungle: An LLM-Powered MCTS Framework for Formulaic Factor Mining](https://mlsanity.com/paper/2505.11122/) (2025)
- [Cognitive Alpha Mining via LLM-Driven Code-Based Evolution](https://mlsanity.com/paper/2511.18850/) (2025)
- [QuantFactor REINFORCE: Mining Steady Formulaic Alpha Factors with Variance-bounded REINFORCE](https://mlsanity.com/paper/2409.05144/) (2024)
- [Alpha-GPT 2.0: Human-in-the-Loop AI for Quantitative Investment](https://mlsanity.com/paper/2402.09746/) (2024)
- [AlphaEval: A Comprehensive and Efficient Evaluation Framework for Formula Alpha Mining](https://mlsanity.com/paper/2508.13174/) (2025)
- [MASS: Muli-agent simulation scaling for portfolio construction](https://mlsanity.com/paper/2505.10278/) (2025)
- [Agentic Trading: When LLM Agents Meet Financial Markets](https://mlsanity.com/paper/2605.19337/) (2026)
- [From Feedback Loops to Policy Updates: Reinforcement Fine-Tuning for LLM-Based Alpha Factor Discovery](https://mlsanity.com/paper/2605.15412/) (2026)
- [Learning from Expert Factors: Trajectory-level Reward Shaping for Formulaic Alpha Mining](https://mlsanity.com/paper/2507.20263/) (2025)
- [Learning Universal Multi-level Market Irrationality Factors to Improve Stock Return Forecasting](https://mlsanity.com/paper/2502.04737/) (2025)
- [Alpha Discovery via Grammar-Guided Learning and Search](https://mlsanity.com/paper/2601.22119/) (2026)
- [AlphaPROBE: Alpha Mining via Principled Retrieval and On-graph biased evolution](https://mlsanity.com/paper/2602.11917/) (2026)
- [FactorEngine: A Program-level Knowledge-Infused Factor Mining Framework for Quantitative Investment](https://mlsanity.com/paper/2603.16365/) (2026)
- [AI Trading's Alpha Singularity: Emergent Market Reasoning through Agent-to-Agent Self-Evolution](https://mlsanity.com/paper/2606.29194/) (2026)
- [AQuA: Recursively Self-Improving Quantitative Trading Research Agents](https://mlsanity.com/paper/2608.12841/) (2026)
- [Alpha2: Discovering Logical Formulaic Alphas using Deep Reinforcement Learning](https://mlsanity.com/paper/2406.16505/) (2024)
- [QuantBench: Benchmarking AI Methods for Quantitative Investment](https://mlsanity.com/paper/2504.18600/) (2025)
- [MEME: Modeling the Evolutionary Modes of Financial Markets](https://mlsanity.com/paper/2602.11918/) (2026)
- [AutoScientist-Quant: Self-Evolving Coding Agents for Automatic Research in Quantitative Investment](https://mlsanity.com/paper/2608.28632/) (2026)
- [BacktestBench: Benchmarking Large Language Models for Automated Quantitative Strategy Backtesting](https://mlsanity.com/paper/2605.17937/) (2026)
- [Synergistic Formulaic Alpha Generation for Quantitative Trading based on Reinforcement Learning](https://mlsanity.com/paper/2401.02710/) (2024)
- [Large Language Models and Futures Price Factors in China](https://mlsanity.com/paper/2509.23609/) (2025)
- [The Label Horizon Paradox: Rethinking Supervision Targets in Financial Forecasting](https://mlsanity.com/paper/2602.03395/) (2026)
- [PandaAI: A Practical Agent CQ2 for Neuro-symbolic Data Analysis And Integrated Decision-Making in Quantitative Finance](https://mlsanity.com/paper/2606.06823/) (2026)
- [Agentic Empirical Asset Pricing: Methodological Foundations](https://mlsanity.com/paper/2609.00731/) (2026)
- [VST: Verifiable Structured Transport for Auditable Agent-to-Agent Alpha Discovery](https://mlsanity.com/paper/2609.07065/) (2026)

## Abstract (arXiv)

In the field of quantitative trading, it is common practice to transform raw historical stock data into indicative signals for the market trend. Such signals are called alpha factors. Alphas in formula forms are more interpretable and thus favored by practitioners concerned with risk. In practice, a set of formulaic alphas is often used together for better modeling precision, so we need to find synergistic formulaic alpha sets that work well together. However, most traditional alpha generators mine alphas one by one separately, overlooking the fact that the alphas would be combined later. In this paper, we propose a new alpha-mining framework that prioritizes mining a synergistic set of alphas, i.e., it directly uses the performance of the downstream combination model to optimize the alpha generator. Our framework also leverages the strong exploratory capabilities of reinforcement learning~(RL) to better explore the vast search space of formulaic alphas. The contribution to the combination models' performance is assigned to be the return used in the RL process, driving the alpha generator to find better alphas that improve upon the current set. Experimental evaluations on real-world stock market data demonstrate both the effectiveness and the efficiency of our framework for stock trend forecasting. The investment simulation results show that our framework is able to achieve higher returns compared to previous approaches.