# ThetaEvolve: Test-time Learning on Open Problems

arXiv: https://arxiv.org/abs/2511.23473 · HTML: https://mlsanity.com/paper/2511.23473/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 47 references that are in the map.
- First posted on arXiv 28 Nov 2025.

## Builds on (most cited first, 27 of 27)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [Reinforcement Learning for Reasoning in Large Language Models with One Training Example](https://mlsanity.com/paper/2504.20571/) (2025)
- [Spurious Rewards: Rethinking Training Signals in RLVR](https://mlsanity.com/paper/2506.10947/) (2025)
- [ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution](https://mlsanity.com/paper/2509.19349/) (2025)
- [On Designing Effective RL Reward at Training Time for LLM Reasoning](https://mlsanity.com/paper/2410.15115/) (2024)
- [Mathematical exploration and discovery at scale](https://mlsanity.com/paper/2511.02864/) (2025)
- [RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments](https://mlsanity.com/paper/2511.07317/) (2025)
- [Constructions in combinatorics via neural networks](https://mlsanity.com/paper/2104.14516/) (2021)
- [PatternBoost: Constructions in Mathematics with a Little Help from AI](https://mlsanity.com/paper/2411.00566/) (2024)

## Built on by (most cited first, 30 of 64)

- [AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization](https://mlsanity.com/paper/2602.20133/) (2026)
- [EvoX: Meta-Evolution for Automated Discovery](https://mlsanity.com/paper/2602.23413/) (2026)
- [CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery](https://mlsanity.com/paper/2604.01658/) (2026)
- [DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution](https://mlsanity.com/paper/2602.02919/) (2026)
- [Structured Scaling of AI Discovery Across Diverse Scientific Domains](https://mlsanity.com/paper/2604.19341/) (2026)
- [Dream-RSI: Recursive Self-Improvement through Evolving Worlds](https://mlsanity.com/paper/2609.14858/) (2026)
- [MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI](https://mlsanity.com/paper/2605.08678/) (2026)
- [Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions](https://mlsanity.com/paper/2607.03935/) (2026)
- [LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling](https://mlsanity.com/paper/2605.08083/) (2026)
- [Escher-Loop: Mutual Evolution by Closed-Loop Self-Referential Optimization](https://mlsanity.com/paper/2604.23472/) (2026)
- [Vector Policy Optimization: Training for Diversity Improves Test-Time Search](https://mlsanity.com/paper/2605.22817/) (2026)
- [Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering](https://mlsanity.com/paper/2607.28568/) (2026)
- [Test-time Recursive Thinking: Self-Improvement without External Feedback](https://mlsanity.com/paper/2602.03094/) (2026)
- [Scaling Discovery through Test-Time Communication](https://mlsanity.com/paper/2609.21032/) (2026)
- [SE-Bench: Benchmarking Self-Evolution with Knowledge Internalization](https://mlsanity.com/paper/2602.04811/) (2026)
- [Effective Harness Engineering for Algorithm Discovery with Coding Agents](https://mlsanity.com/paper/2605.15221/) (2026)
- [EurekAgent: Agent Environment Engineering is All You Need For Autonomous Scientific Discovery](https://mlsanity.com/paper/2606.13662/) (2026)
- [Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks](https://mlsanity.com/paper/2606.29082/) (2026)
- [SwarmResearch: Orchestrating Coding Agents for Open-Ended Discovery](https://mlsanity.com/paper/2607.02807/) (2026)
- [Automated Discovery Has No Universally Superior Harness](https://mlsanity.com/paper/2607.18235/) (2026)
- [Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions](https://mlsanity.com/paper/2606.02859/) (2026)
- [The Agentic Researcher: A Practical Guide to AI-Assisted Research in Mathematics and Machine Learning](https://mlsanity.com/paper/2603.15914/) (2026)
- [PACEvolve++: Improving Test-time Learning for Evolutionary Search Agents](https://mlsanity.com/paper/2605.07039/) (2026)
- [Compute Allocation for Self-Evolving LLMs: From Depth-Breadth to Multi-Armed Bandits](https://mlsanity.com/paper/2605.29268/) (2026)
- [ScienceFlow: A long-horizon agent for ML research, scientific discovery and beyond](https://mlsanity.com/paper/2608.14354/) (2026)
- [CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad](https://mlsanity.com/paper/2603.14575/) (2026)
- [TurboEvolve: Towards Fast and Robust LLM-Driven Program Evolution](https://mlsanity.com/paper/2604.18607/) (2026)
- [FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale](https://mlsanity.com/paper/2605.14445/) (2026)
- [Test-Time Learning with an Evolving Library](https://mlsanity.com/paper/2605.14477/) (2026)
- [AI-Driven Research for Databases](https://mlsanity.com/paper/2604.06566/) (2026)

## Abstract (arXiv)

Recent advances in large language models (LLMs) have enabled breakthroughs in mathematical discovery, exemplified by AlphaEvolve, a closed-source system that evolves programs to improve bounds on open problems. However, it relies on ensembles of frontier LLMs to achieve new bounds and is a pure inference system that models cannot internalize the evolving strategies. We introduce ThetaEvolve, an open-source framework that simplifies and extends AlphaEvolve to efficiently scale both in-context learning and Reinforcement Learning (RL) at test time, allowing models to continually learn from their experiences in improving open optimization problems. ThetaEvolve features a single LLM, a large program database for enhanced exploration, batch sampling for higher throughput, lazy penalties to discourage stagnant outputs, and optional reward shaping for stable training signals, etc. ThetaEvolve is the first evolving framework that enable a small open-source model, like DeepSeek-R1-0528-Qwen3-8B, to achieve new best-known bounds on open problems (circle packing and first auto-correlation inequality) mentioned in AlphaEvolve. Besides, across two models and four open tasks, we find that ThetaEvolve with RL at test-time consistently outperforms inference-only baselines, and the model indeed learns evolving capabilities, as the RL-trained checkpoints demonstrate faster progress and better final performance on both trained target task and other unseen tasks. We release our code publicly: https://github.com/ypwang61/ThetaEvolve