The Time Value of Evolution
On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization
Abstract (arXiv)
In evolutionary search, a weak child can be a valuable ancestor that makes high-fitness regions reachable. Immediate-return control is blind to this delayed utility, penalizing mutations through their immediate offspring even when they open productive future lineages. We formalize this hidden dynamic as the time value of evolution within a finite-horizon Markov decision process. To exploit it, we introduce Lineage-Value Policy Gradients (LVPG), a long-horizon actor-critic framework for automated trading policy discovery. Our architecture decouples search control into specialized policy heads over a shared generative backbone: a bootstrapped critic head estimates the value of finite-horizon lineage potential from multi-step mutation trees, while an actor head dynamically modulates mutation intensity over the remaining search budget. We isolate the impact of long-horizon credit assignment against immediate-return optimization across 90 paired runs under matched operators, lineage supervision, folds, seeds, and budgets. Path-based credit assignment substantially accelerates finite-budget search, increasing validation best-so-far AUC by 0.394 Sharpe units. LVPG also produces fewer temporary regressions than immediate-return optimization and recovers from them more often. Finite-horizon lineage value yields more selective non-monotonic search and stronger policies within identical resource constraints.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 7 of its 24 references that are in the map. First posted on arXiv 13 Aug 2026.
Builds on
7 of 24 references.
References · 7
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
- 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
- 2024Direct Preference Optimization with an Offsetcited by 122
- 2024LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristicscited by 97
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026PACEvolve++: Improving Test-time Learning for Evolutionary Search Agents
- 2026DarwinX: Evolving Agent Harnesses Through Natural Selection
- 2024Ancestral Reinforcement Learning: Unifying Zeroth-Order Optimization and Genetic Algorithms for Reinforcement Learning
- 2026Evolution or Illusion? Rethinking Evaluation in LLM Evolutionary Search
- 2026ForkPilot: Self-Evolving Policy for Retrospective Search in Long-Horizon Agents
- 2025Evolution without an Oracle: Driving Effective Evolution with LLM Judges
- 2026Towards Autonomous Formulaic Alpha Discovery: An Evolutionary Computation Perspective
- 2026MILO: Automated Harness Discovery via Orchestrated Multi-Agent Evolution