# Propose, Don't Judge: An Anytime-Valid Referee for LLM Agents That Mine Investment Factors

arXiv: https://arxiv.org/abs/2609.27051 · HTML: https://mlsanity.com/paper/2609.27051/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 22 Sep 2026.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 16 of 16)

- [Estimating means of bounded random variables by betting](https://mlsanity.com/paper/2010.09686/) (2020)
- [R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimization](https://mlsanity.com/paper/2505.15155/) (2025)
- [AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decay](https://mlsanity.com/paper/2502.16789/) (2025)
- [Generating Synergistic Formulaic Alpha Collections via Reinforcement Learning](https://mlsanity.com/paper/2306.12964/) (2023)
- [Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops](https://mlsanity.com/paper/2607.07663/) (2026)
- [PACE: Anytime-Valid Acceptance Tests for Self-Evolving Agents](https://mlsanity.com/paper/2606.08106/) (2026)
- [CLQT: A Closed-Loop, Cost-Aware, Strategy-Consistent Benchmark for Diagnostic Evaluation of LLM Portfolio-Management Agents](https://mlsanity.com/paper/2606.29771/) (2026)
- [Self-Evolving Agents with Anytime-Valid Certificates](https://mlsanity.com/paper/2607.00871/) (2026)
- [An online generalization of the (e-)Benjamini-Hochberg procedure](https://mlsanity.com/paper/2407.20683/) (2024)
- [Anytime-valid FDR control with the stopped e-BH procedure](https://mlsanity.com/paper/2502.08539/) (2025)
- [Multiple testing in multi-stream sequential change detection](https://mlsanity.com/paper/2501.04130/) (2025)
- [Improving online FDR procedures via online analogs of e-closure and compound e-values](https://mlsanity.com/paper/2603.24792/) (2026)
- [Economic Design of Memory-Type Control Charts: The Fallacy of the Formula Proposed by Lorenzen and Vance (1986)](https://mlsanity.com/paper/1708.06160/) (2017)
- [Admissibility and Complete Classes for False Discovery Rate Control with E-values](https://mlsanity.com/paper/2607.14380/) (2026)
- [Dynamic e-closure for online hypotheses with any-time-valid evidence: closure principles and projective mergers](https://mlsanity.com/paper/2608.09927/) (2026)
- [Gaussian-efficient testing by betting on the mean of bounded data](https://mlsanity.com/paper/2608.21694/) (2026)

## Abstract (arXiv)

Language-model agents now run the whole of quantitative factor research: they propose investment factors, backtest them, select the survivors and retire them. We ask which of those jobs an agent should keep. Our answer is governed self-evolution: the agent may propose, and a frozen statistical referee that the agent cannot touch must judge. The referee scores each candidate only on market outcomes revealed after submission, by betting, so its false-discovery guarantee holds at every stopping time for any proposal policy. We cross three proposers (a script, a bandit and a language model) with this referee and with three deliberately leaky ones, in a synthetic world with planted truth, a probe-authoring environment and a ten-year walk-forward on the CSI 500. Who judges sets the number of false admissions: the frozen referee admits 5-11 times fewer sub-threshold factors than the leaky referees under a scripted proposer, and no proposer closes that gap. Who proposes sets the yield: the language model beats the script, matches the bandit, and adds the one capability a bandit lacks, writing its own diagnostic probes. The certificate's price is time: an admitted true factor waits about 500 trading days, and the certified portfolio's Sharpe ratio therefore trails an ungated one. Judging belongs to the procedure; proposing and instrument-making belong to the agent.