# AlphaEval: A Comprehensive and Efficient Evaluation Framework for Formula Alpha Mining

arXiv: https://arxiv.org/abs/2508.13174 · HTML: https://mlsanity.com/paper/2508.13174/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 36 references that are in the map.
- First posted on arXiv 10 Aug 2025.

## Builds on (most cited first, 12 of 12)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [A Theoretical Analysis of NDCG Type Ranking Measures](https://mlsanity.com/paper/1304.6480/) (2013)
- [Qlib: An AI-oriented Quantitative Investment Platform](https://mlsanity.com/paper/2009.11189/) (2020)
- [R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimization](https://mlsanity.com/paper/2505.15155/) (2025)
- [AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decay](https://mlsanity.com/paper/2502.16789/) (2025)
- [Generating Synergistic Formulaic Alpha Collections via Reinforcement Learning](https://mlsanity.com/paper/2306.12964/) (2023)
- [AlphaEvolve: A Learning Framework to Discover Novel Alphas in Quantitative Investment](https://mlsanity.com/paper/2103.16196/) (2021)
- [AlphaForge: A Framework to Mine and Dynamically Combine Formulaic Alpha Factors](https://mlsanity.com/paper/2406.18394/) (2024)
- [QuantFactor REINFORCE: Mining Steady Formulaic Alpha Factors with Variance-bounded REINFORCE](https://mlsanity.com/paper/2409.05144/) (2024)
- [Beyond the Black Box: Interpretability of LLMs in Finance](https://mlsanity.com/paper/2505.24650/) (2025)
- [Chain-of-Alpha: Unleashing the Power of Large Language Models for Alpha Mining in Quantitative Trading](https://mlsanity.com/paper/2508.06312/) (2025)
- [Learning from Expert Factors: Trajectory-level Reward Shaping for Formulaic Alpha Mining](https://mlsanity.com/paper/2507.20263/) (2025)

## Built on by (most cited first, 11 of 11)

- [QuantaAlpha: An Evolutionary Framework for LLM-Driven Alpha Mining](https://mlsanity.com/paper/2602.07085/) (2026)
- [Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2512.23515/) (2025)
- [AlphaPROBE: Alpha Mining via Principled Retrieval and On-graph biased evolution](https://mlsanity.com/paper/2602.11917/) (2026)
- [AlphaMemo: Structured Search-Process Memory for Self-Evolving Alpha Mining Agents](https://mlsanity.com/paper/2606.20625/) (2026)
- [EVOQUANT: Self-Evolving Verifier-Guided Strategy Optimization for Robust Quantitative Trading](https://mlsanity.com/paper/2607.12455/) (2026)
- [FaVOR: LLM-Based Agentic Framework for Factor Mining via Empirical Validation](https://mlsanity.com/paper/2608.30192/) (2026)
- [Agentic Empirical Asset Pricing: Methodological Foundations](https://mlsanity.com/paper/2609.00731/) (2026)
- [AlphaPareto: Formulaic Alpha Discovery with LLM-Guided Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2609.34188/) (2026)
- [From Search to Research: Exploring Search Scaling in Autonomous Quantitative Factor Mining](https://mlsanity.com/paper/2609.35559/) (2026)
- [FactorBench: A Portfolio-Aware Benchmark for Automated Factor Mining](https://mlsanity.com/paper/2610.06947/) (2026)
- [AlphaPilot: Multi-Turn Tool-Using LLM Agents for Adaptive Alpha Mining with Reinforcement Learning](https://mlsanity.com/paper/2610.15488/) (2026)

## Abstract (arXiv)

Formula alpha mining, which generates predictive signals from financial data, is critical for quantitative investment. Although various algorithmic approaches-such as genetic programming, reinforcement learning, and large language models-have significantly expanded the capacity for alpha discovery, systematic evaluation remains a key challenge. Existing evaluation metrics predominantly include backtesting and correlation-based measures. Backtesting is computationally intensive, inherently sequential, and sensitive to specific strategy parameters. Correlation-based metrics, though efficient, assess only predictive ability and overlook other crucial properties such as temporal stability, robustness, diversity, and interpretability. Additionally, the closed-source nature of most existing alpha mining models hinders reproducibility and slows progress in this field. To address these issues, we propose AlphaEval, a unified, parallelizable, and backtest-free evaluation framework for automated alpha mining models. AlphaEval assesses the overall quality of generated alphas along five complementary dimensions: predictive power, stability, robustness to market perturbations, financial logic, and diversity. Extensive experiments across representative alpha mining algorithms demonstrate that AlphaEval achieves evaluation consistency comparable to comprehensive backtesting, while providing more comprehensive insights and higher efficiency. Furthermore, AlphaEval effectively identifies superior alphas compared to traditional single-metric screening approaches. All implementations and evaluation tools are open-sourced to promote reproducibility and community engagement.