mlsanityData to 14 Oct 2026

Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI

arXiv 2507.14172 ↗First posted 10 Jul 2025cs.LG cs.AI cs.NEcited by 36

On the map: Synthetic data and self-improvement for LLMs → Self-evolving LLM agents

Abstract (arXiv)

Many program synthesis tasks prove too challenging for even state-of-the-art language models to solve in single attempts. Search-based evolutionary methods offer a promising alternative by exploring solution spaces iteratively, but their effectiveness remain limited by the fixed capabilities of the underlying generative model. We propose SOAR, a method that learns program synthesis by integrating language models into a self-improving evolutionary loop. SOAR alternates between (1) an evolutionary search that uses an LLM to sample and refine candidate solutions, and (2) a hindsight learning phase that converts search attempts into valid problem-solution pairs used to fine-tune the LLM's sampling and refinement capabilities, -- ,enabling increasingly effective search in subsequent iterations. On the challenging ARC-AGI benchmark, SOAR achieves significant performance gains across model scales and iterations, leveraging positive transfer between the sampling and refinement finetuning tasks. These improvements carry over to test-time adaptation, enabling SOAR to solve 52% of the public test set. Our code is open-sourced at: https://github.com/flowersteam/SOAR

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Placed in Self-evolving LLM agents, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 34 of its 62 references that are in the map. First posted on arXiv 10 Jul 2025.

Builds on

34 of 62 references.

References · 34

  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  2. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  3. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  4. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  5. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  6. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  7. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  8. 2017Hindsight Experience Replaycited by 1,136
  9. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  10. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
  11. 2024Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Modelcited by 268
  12. 2023A Rank Stabilization Scaling Factor for Fine-Tuning with LoRAcited by 243
22 more
  1. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 197
  2. 2023Large Language Models as General Pattern Machinescited by 197
  3. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 151
  4. 2024Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputscited by 150
  5. 2016DeepCoder: Learning to Write Programscited by 149
  6. 2022Evolution through Large Modelscited by 134
  7. 2023Language Model Crossover: Variation through Few-Shot Promptingcited by 122
  8. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 120
  9. 2024Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Modelscited by 89
  10. 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 85
  11. 2023LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representationscited by 82
  12. 2025NoveltyBench: Evaluating Language Models for Humanlike Diversitycited by 66
  13. 2023Large Language Models Are Not Strong Abstract Reasonerscited by 53
  14. 2025Optimizing Language Models for Inference Time Objectives using Reinforcement Learningcited by 53
  15. 2024Code Repair with LLMs gives an Exploration-Exploitation Tradeoffcited by 51
  16. 2023Quality-Diversity through AI Feedbackcited by 40
  17. 2024H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmarkcited by 33
  18. 2020TF-Coder: Program Synthesis for Tensor Manipulationscited by 27
  19. 2024Is Programming by Example solved by LLMs?cited by 27
  20. 2024CodeIt: Self-Improving Language Models with Prioritized Hindsight Replaycited by 23
  21. 2024Automating the Search for Artificial Life with Foundation Modelscited by 20
  22. 2019Bayesian Synthesis of Probabilistic Programs for Automatic Data Modelingcited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 28.

36 papers.

Sort:

By area:

  1. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 228
  2. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  3. 2026AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimizationcited by 60
  4. 2026EvoX: Meta-Evolution for Automated Discoverycited by 60
  5. 2025Guided Self-Evolving LLMs with Minimal Human Supervisioncited by 35
  6. 2025ArcMemo: Abstract Reasoning Composition with Lifelong LLM Memorycited by 24
  7. 2026PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoningcited by 10
  8. 2026Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineeringcited by 8
  9. 2026SEAD: Self-Evolving Agent for Multi-Turn Service Dialoguecited by 7
  10. 2026Workspace Optimization: How to Train Your Agentcited by 7
  11. 2025Think Visually, Reason Textually: Vision-Language Synergy in ARCcited by 6
  12. 2026The ARC of Progress towards AGI: A Living Survey of Abstraction and Reasoningcited by 6
24 more
  1. 2026LLM-Based Scientific Equation Discovery via Physics-Informed Token-Regularized Policy Optimizationcited by 5
  2. 2026ARC Prize 2025: Technical Reportcited by 4
  3. 2026Symbol-Equivariant Recurrent Reasoning Modelscited by 4
  4. 2026Self-Improving Language Models with Bidirectional Evolutionary Searchcited by 4
  5. 2025LLM Priors for ERM over Programscited by 3
  6. 2026One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Modelscited by 3
  7. 2026What Do Evolutionary Coding Agents Evolve?cited by 3
  8. 2026AI-Driven Research for Databasescited by 2
  9. 2026Explore Before You Solve: The Speed--Depth Trade-off in Epistemic Agents for ARC-AGI-3cited by 2
  10. 2026Overcoming the Weakest-Link Effect in LLM-Driven Program Optimization via Heterogeneous Edit Recombinationcited by 2
  11. 2026State2State: Environment-Derived Mid-Training for LLM Agentscited by 2
  12. 2025TreeGPT: Pure TreeFFN Encoder-Decoder Architecture for Structured Reasoning Without Attention Mechanismscited by 1
  13. 2026SignalClaw: LLM-Guided Evolutionary Synthesis of Interpretable Traffic Signal Control Skillscited by 1
  14. 2026TraceViT: Grounded Trace Supervision for Visual Abstract Reasoningcited by 1
  15. 2026PotARCin: Multi-Dimensional Evaluation of Skill Acquisition in Abstract Reasoning Taskscited by 1
  16. 2025Software Engineering Agents for Embodied Controller Generation : A Study in Minigrid Environmentscited by 0
  17. 2026Dynamic Adversarial Reinforcement Learning for Robust Multimodal Large Language Modelscited by 0
  18. 2026WIST: Web-Grounded Iterative Self-Play Tree for Domain-Targeted Reasoning Improvementcited by 0
  19. 2026From Global to Factor-Wise Expert Composition in Discrete Diffusion Modelscited by 0
  20. 2026DASyR-LLM: Domain-Aware Symbolic Regression with LLMs for Kinetic Model Discoverycited by 0
  21. 2026Position: Reasoning is a Learnable Rule-Based Processcited by 0
  22. 2026Narcissus: Program Synthesis Using Context-Aware LLM Approximationscited by 0
  23. 2026Combining LLMs and Genetic Search for ARC-AGI-2cited by 0
  24. 2026Hypothesis-guided discovery of cognitive algorithms via program refinementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ELMER: Evolutionary Language Model that Explores and Refines
  2. 2025Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering
  3. 2026Simple Baselines are Competitive with Code Evolution
  4. 2026Component-Aware Feedback for Self-Evolving Programs
  5. 2025Evolutionary thoughts: integration of large language models and evolutionary algorithms
  6. 2026AGAR: a reinforcement learning substrate for LLM program evolution
  7. 2026SEMAG: Self-Evolutionary Multi-Agent Code Generation
  8. 2025Program Synthesis via Test-Time Transduction