mlsanityData to 7 Oct 2026

Learning to Self-Evolve

arXiv 2603.18620 ↗First posted 19 Mar 2026cs.CL cs.AIcited by 2

On the map: Synthetic data and self-improvement for LLMs → Self-rewarding alignment and self-training

Abstract (arXiv)

We introduce Learning to Self-Evolve (LSE), a reinforcement learning framework that trains large language models (LLMs) to improve their own contexts at test time. We situate LSE in the setting of test-time self-evolution, where a model iteratively refines its context from feedback on seen problems to perform better on new ones. Existing approaches rely entirely on the inherent reasoning ability of the model and never explicitly train it for this task. LSE reduces the multi-step evolution problem to a single-step RL objective, where each context edit is rewarded by the improvement in downstream performance. We pair this objective with a tree-guided evolution loop. On Text-to-SQL generation (BIRD) and general question answering (MMLU-Redux), a 4B-parameter model trained with LSE outperforms self-evolving policies powered by GPT-5 and Claude Sonnet 4.5, as well as prompt optimization methods including GEPA and TextGrad, and transfers to guide other models without additional training. Our results highlight the effectiveness of treating self-evolution as a learnable skill.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 23 of its 32 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

23 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Self-Rewarding Language Models18 Jan
  2. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution28 Sep

Other references · 21

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  2. 2023Let's Verify Step by Stepcited by 3,598
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  4. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  5. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  6. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  7. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  8. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  9. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
  10. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  11. 2024Automated Design of Agentic Systemscited by 360
  12. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
9 more
  1. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  2. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  3. 2024TextGrad: Automatic "Differentiation" via Textcited by 250
  4. 2025Learning to Reason without External Rewardscited by 199
  5. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 99
  6. 2026Learning to Discover at Test Timecited by 94
  7. 2025End-to-End Test-Time Training for Long Contextcited by 53
  8. 2025Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQLcited by 38
  9. 2024G"odel Agent: A Self-Referential Agent Framework for Recursive Self-Improvementcited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Librariescited by 6
  2. 2026Scaling Self-Evolving Agents via Parametric Memorycited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2026On the Generalization Gap in Self-Evolving Language Model Reasoning
  2. 2024A Survey on Self-Evolution of Large Language Models
  3. 2026When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gain
  4. 2025The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback
  5. 2026TTSR: Test-Time Self-Evolving via Reflection
  6. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
  7. 2026Evolutionary System Prompt Learning for Reinforcement Learning in LLMs
  8. 2026D2Evo: Dual Difficulty-Aware Self-Evolution for Data-Efficient Reinforcement Learning