# AlphaPareto: Formulaic Alpha Discovery with LLM-Guided Multi-Objective Reinforcement Learning

arXiv: https://arxiv.org/abs/2609.34188 · HTML: https://mlsanity.com/paper/2609.34188/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 19 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 11 of 11)

- [Qlib: An AI-oriented Quantitative Investment Platform](https://mlsanity.com/paper/2009.11189/) (2020)
- [Alpha-GPT: Human-AI Interactive Alpha Mining for Quantitative Investment](https://mlsanity.com/paper/2308.00016/) (2023)
- [R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimization](https://mlsanity.com/paper/2505.15155/) (2025)
- [AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decay](https://mlsanity.com/paper/2502.16789/) (2025)
- [Generating Synergistic Formulaic Alpha Collections via Reinforcement Learning](https://mlsanity.com/paper/2306.12964/) (2023)
- [Non-Stationary Markov Decision Processes, a Worst-Case Approach using Model-Based Reinforcement Learning, Extended version](https://mlsanity.com/paper/1904.10090/) (2019)
- [AlphaEvolve: A Learning Framework to Discover Novel Alphas in Quantitative Investment](https://mlsanity.com/paper/2103.16196/) (2021)
- [AlphaForge: A Framework to Mine and Dynamically Combine Formulaic Alpha Factors](https://mlsanity.com/paper/2406.18394/) (2024)
- [MAP: Multi-Human-Value Alignment Palette](https://mlsanity.com/paper/2410.19198/) (2024)
- [Alpha-GPT 2.0: Human-in-the-Loop AI for Quantitative Investment](https://mlsanity.com/paper/2402.09746/) (2024)
- [AlphaEval: A Comprehensive and Efficient Evaluation Framework for Formula Alpha Mining](https://mlsanity.com/paper/2508.13174/) (2025)

## Abstract (arXiv)

Formulaic alpha discovery is a core challenge in quantitative trading, as identifying alphas that work well together remains difficult. Recent reinforcement learning (RL) methods formulate this task as a Markov decision process (MDP), but two important issues remain unresolved. First, as the alpha pool evolves, the reward function changes accordingly, making the MDP inherently non-stationary. Second, most existing methods optimize a single objective, typically predictive power, while ignoring other important properties of a high-quality alpha pool. Motivated by these challenges, we propose AlphaPareto, an RL method for formulaic alpha discovery. To address non-stationarity, AlphaPareto augments the state to include both the alpha under construction and the current alpha pool, and applies a large language model (LLM) to encode the pool. This design allows the agent to adapt to the evolving search environment. To overcome the limitation of single-objective reward design, AlphaPareto replaces the scalar reward with a multi-objective vector-valued reward that simultaneously captures predictive power, temporal stability, perturbation robustness, and diversity, and optimizes these objectives through a Pareto-regularized learning procedure. Empirical applications to real-world datasets show that our AlphaPareto method outperforms its competitors.