# Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI

arXiv: https://arxiv.org/abs/2507.14172 · HTML: https://mlsanity.com/paper/2507.14172/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Self-evolving LLM agents, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 34 of its 62 references that are in the map.
- First posted on arXiv 10 Jul 2025.

## Placed in

- Synthetic data and self-improvement for LLMs / [Self-evolving LLM agents](https://mlsanity.com/synth/self-evolving-llm-agents/)

## Builds on (most cited first, 30 of 34)

- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025)
- [Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking](https://mlsanity.com/paper/2403.09629/) (2024)
- [Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model](https://mlsanity.com/paper/2401.02051/) (2024)
- [A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA](https://mlsanity.com/paper/2312.03732/) (2023)
- [Is Self-Repair a Silver Bullet for Code Generation?](https://mlsanity.com/paper/2306.09896/) (2023)
- [Large Language Models as General Pattern Machines](https://mlsanity.com/paper/2307.04721/) (2023)
- [RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning](https://mlsanity.com/paper/2410.02089/) (2024)
- [Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs](https://mlsanity.com/paper/2407.01082/) (2024)
- [DeepCoder: Learning to Write Programs](https://mlsanity.com/paper/1611.01989/) (2016)
- [Evolution through Large Models](https://mlsanity.com/paper/2206.08896/) (2022)
- [Language Model Crossover: Variation through Few-Shot Prompting](https://mlsanity.com/paper/2302.12170/) (2023)
- [Hypothesis Search: Inductive Reasoning with Language Models](https://mlsanity.com/paper/2309.05660/) (2023)
- [Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models](https://mlsanity.com/paper/2412.15287/) (2024)
- [Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey](https://mlsanity.com/paper/2012.09830/) (2020)
- [LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations](https://mlsanity.com/paper/2305.18354/) (2023)
- [NoveltyBench: Evaluating Language Models for Humanlike Diversity](https://mlsanity.com/paper/2504.05228/) (2025)
- [Large Language Models Are Not Strong Abstract Reasoners](https://mlsanity.com/paper/2305.19555/) (2023)
- [Optimizing Language Models for Inference Time Objectives using Reinforcement Learning](https://mlsanity.com/paper/2503.19595/) (2025)
- [Code Repair with LLMs gives an Exploration-Exploitation Tradeoff](https://mlsanity.com/paper/2405.17503/) (2024)
- [Quality-Diversity through AI Feedback](https://mlsanity.com/paper/2310.13032/) (2023)
- [H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmark](https://mlsanity.com/paper/2409.01374/) (2024)
- [TF-Coder: Program Synthesis for Tensor Manipulations](https://mlsanity.com/paper/2003.09040/) (2020)

## Built on by (most cited first, 30 of 36)

- [R-Zero: Self-Evolving Reasoning LLM from Zero Data](https://mlsanity.com/paper/2508.05004/) (2025)
- [A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence](https://mlsanity.com/paper/2507.21046/) (2025)
- [AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization](https://mlsanity.com/paper/2602.20133/) (2026)
- [EvoX: Meta-Evolution for Automated Discovery](https://mlsanity.com/paper/2602.23413/) (2026)
- [Guided Self-Evolving LLMs with Minimal Human Supervision](https://mlsanity.com/paper/2512.02472/) (2025)
- [ArcMemo: Abstract Reasoning Composition with Lifelong LLM Memory](https://mlsanity.com/paper/2509.04439/) (2025)
- [PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning](https://mlsanity.com/paper/2607.20064/) (2026)
- [Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering](https://mlsanity.com/paper/2607.28568/) (2026)
- [SEAD: Self-Evolving Agent for Multi-Turn Service Dialogue](https://mlsanity.com/paper/2602.03548/) (2026)
- [Workspace Optimization: How to Train Your Agent](https://mlsanity.com/paper/2605.09650/) (2026)
- [Think Visually, Reason Textually: Vision-Language Synergy in ARC](https://mlsanity.com/paper/2511.15703/) (2025)
- [The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoning](https://mlsanity.com/paper/2603.13372/) (2026)
- [LLM-Based Scientific Equation Discovery via Physics-Informed Token-Regularized Policy Optimization](https://mlsanity.com/paper/2602.10576/) (2026)
- [ARC Prize 2025: Technical Report](https://mlsanity.com/paper/2601.10904/) (2026)
- [Symbol-Equivariant Recurrent Reasoning Models](https://mlsanity.com/paper/2603.02193/) (2026)
- [Self-Improving Language Models with Bidirectional Evolutionary Search](https://mlsanity.com/paper/2605.28814/) (2026)
- [LLM Priors for ERM over Programs](https://mlsanity.com/paper/2510.14331/) (2025)
- [One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models](https://mlsanity.com/paper/2604.18839/) (2026)
- [What Do Evolutionary Coding Agents Evolve?](https://mlsanity.com/paper/2605.20086/) (2026)
- [AI-Driven Research for Databases](https://mlsanity.com/paper/2604.06566/) (2026)
- [Explore Before You Solve: The Speed--Depth Trade-off in Epistemic Agents for ARC-AGI-3](https://mlsanity.com/paper/2605.25931/) (2026)
- [Overcoming the Weakest-Link Effect in LLM-Driven Program Optimization via Heterogeneous Edit Recombination](https://mlsanity.com/paper/2607.28947/) (2026)
- [State2State: Environment-Derived Mid-Training for LLM Agents](https://mlsanity.com/paper/2608.04934/) (2026)
- [TreeGPT: Pure TreeFFN Encoder-Decoder Architecture for Structured Reasoning Without Attention Mechanisms](https://mlsanity.com/paper/2509.05550/) (2025)
- [SignalClaw: LLM-Guided Evolutionary Synthesis of Interpretable Traffic Signal Control Skills](https://mlsanity.com/paper/2604.05535/) (2026)
- [TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning](https://mlsanity.com/paper/2607.29586/) (2026)
- [PotARCin: Multi-Dimensional Evaluation of Skill Acquisition in Abstract Reasoning Tasks](https://mlsanity.com/paper/2609.27288/) (2026)
- [Software Engineering Agents for Embodied Controller Generation : A Study in Minigrid Environments](https://mlsanity.com/paper/2510.21902/) (2025)
- [Dynamic Adversarial Reinforcement Learning for Robust Multimodal Large Language Models](https://mlsanity.com/paper/2602.22227/) (2026)
- [WIST: Web-Grounded Iterative Self-Play Tree for Domain-Targeted Reasoning Improvement](https://mlsanity.com/paper/2603.22352/) (2026)

## Abstract (arXiv)

Many program synthesis tasks prove too challenging for even state-of-the-art language models to solve in single attempts. Search-based evolutionary methods offer a promising alternative by exploring solution spaces iteratively, but their effectiveness remain limited by the fixed capabilities of the underlying generative model. We propose SOAR, a method that learns program synthesis by integrating language models into a self-improving evolutionary loop. SOAR alternates between (1) an evolutionary search that uses an LLM to sample and refine candidate solutions, and (2) a hindsight learning phase that converts search attempts into valid problem-solution pairs used to fine-tune the LLM's sampling and refinement capabilities, -- ,enabling increasingly effective search in subsequent iterations. On the challenging ARC-AGI benchmark, SOAR achieves significant performance gains across model scales and iterations, leveraging positive transfer between the sampling and refinement finetuning tasks. These improvements carry over to test-time adaptation, enabling SOAR to solve 52% of the public test set. Our code is open-sourced at: https://github.com/flowersteam/SOAR