mlsanityData to 7 Oct 2026

FLEX: Continuous Agent Evolution via Forward Learning from Experience

arXiv 2511.06449 ↗First posted 9 Nov 2025cs.LG cs.AIcited by 51

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Autonomous agents driven by Large Language Models (LLMs) have revolutionized reasoning and problem-solving but remain static after training, unable to grow with experience as intelligent beings do during deployment. We introduce Forward Learning with EXperience (FLEX), a gradient-free learning paradigm that enables LLM agents to continuously evolve through accumulated experience. Specifically, FLEX cultivates scalable and inheritable evolution by constructing a structured experience library through continual reflection on successes and failures during interaction with the environment. FLEX delivers substantial improvements on mathematical reasoning, chemical retrosynthesis, and protein fitness prediction (up to 23% on AIME25, 10% on USPTO50k, and 14% on ProteinGym). We further identify a clear scaling law of experiential growth and the phenomenon of experience inheritance across agents, marking a step toward scalable and inheritable continuous agent evolution. Project Page: https://flex-gensi-thuair.github.io.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 80 of its 115 references that are in the map. First posted on arXiv 9 Nov 2025.

Builds on

80 of 115 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory29 Sep
  2. 2025SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents4 Aug
  3. 2025Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution26 May
  4. 2023CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models23 May

Other references · 76

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  9. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  10. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  11. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  12. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
38 more
  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2017Deep reinforcement learning from human preferencescited by 3,846
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  12. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  13. 2020Learning to summarize from human feedbackcited by 2,562
  14. 2015Trust Region Policy Optimizationcited by 2,503
  15. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  16. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  17. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  18. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  19. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  20. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  21. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  22. 2016An overview of gradient descent optimization algorithmscited by 1,021
  23. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  24. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  25. 2025Group Sequence Policy Optimizationcited by 788
  26. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  27. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  28. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  29. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  30. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  31. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  32. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  33. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  34. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  35. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 463
  36. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  37. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
  38. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 46.

51 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 285
  2. 2026XSkill: Continual Learning from Experience and Skills in Multimodal Agentscited by 71
  3. 2026SkillX: Automatically Constructing Skill Knowledge Bases for Agentscited by 69
  4. 2026AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimizationcited by 46
  5. 2026EvoX: Meta-Evolution for Automated Discoverycited by 45
  6. 2026Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agentscited by 44
  7. 2026Meta Context Engineering via Agentic Skill Evolutioncited by 42
  8. 2026Dynamic Dual-Granularity Skill Bank for Agentic RLcited by 33
  9. 2026ExpSeek: Self-Triggered Experience Seeking for Web Agentscited by 23
  10. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 22
  11. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 19
  12. 2026SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolutioncited by 18
38 more
  1. 2025AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agentscited by 12
  2. 2026Experiential Reflective Learning for Self-Improving LLM Agentscited by 12
  3. 2026GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)cited by 10
  4. 2026M^star: Every Task Deserves Its Own Memory Harnesscited by 9
  5. 2026AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Tracescited by 9
  6. 2026Position: Beyond Model-Centric Prediction -- Agentic Time Series Forecastingcited by 8
  7. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  8. 2026TAME: A Trustworthy Test-Time Evolution of Agent Memory with Systematic Benchmarkingcited by 7
  9. 2026UMEM: Unified Memory Extraction and Management Framework for Generalizable Memorycited by 7
  10. 2026Evolving-RL: End-to-End Optimization of Experience-Driven Self-Evolving Capability within Agentscited by 7
  11. 2026Aligning Agentic World Models via Knowledgeable Experience Learningcited by 5
  12. 2026GSEM: Graph-based Self-Evolving Memory for Experience Augmented Clinical Reasoningcited by 5
  13. 2026Co-Evolving Skill Generation and Policy Optimizationcited by 5
  14. 2025Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agentscited by 3
  15. 2026Memory Transfer Learning: How Memories are Transferred Across Domains in Coding Agentscited by 3
  16. 2026Mem-π: Adaptive Memory through Learning When and What to Generatecited by 3
  17. 2026Closing the Feedback Loop: From Experience Extraction to Insight Governance in Verbal Reinforcement Learningcited by 3
  18. 2026Empirical-MCTS: Continuous Agent Evolution via Dual-Experience Monte Carlo Tree Searchcited by 2
  19. 2026Towards Effective Experiential Learning: Dual Guidance for Utilization and Internalizationcited by 2
  20. 2026Exploring Autonomous Agentic Data Engineering for Model Specializationcited by 2
  21. 2026SwarmResearch: Orchestrating Coding Agents for Open-Ended Discoverycited by 2
  22. 2026Internalizing Agency from Reflective Experiencecited by 1
  23. 2026SLEA-RL: Step-Level Experience Augmented Reinforcement Learning for Multi-Turn Agentic Trainingcited by 1
  24. 2026Context Training with Active Information Seekingcited by 1
  25. 2026EXG: Self-Evolving Agents with Experience Graphscited by 1
  26. 2026Rethinking Memory as Continuously Evolving Connectivitycited by 1
  27. 2026Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learningcited by 1
  28. 2026PATH-Bench: Path-Dependent Evaluation of Lifelong Agentscited by 1
  29. 2026Chain-of-Experience for Continual LLM Improvementcited by 1
  30. 2026EMR: Self-Evolving Medical Multi-Agent System via Experience Mining and Reusecited by 1
  31. 2025NSTR: Neural Spectral Transport Representation for Space-Varying Frequency Fieldscited by 0
  32. 2025Scaling Implicit Fields via Hypernetwork-Driven Multiscale Coordinate Transformationscited by 0
  33. 2026Revolutionizing Long-Term Memory in AI: New Horizons with High-Capacity and High-Speed Storagecited by 0
  34. 2026EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learningcited by 0
  35. 2026MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoningcited by 0
  36. 2026Mechanistic Attention Guidance for Agent Memory Refinementcited by 0
  37. 2026RoMeRL: Balancing Feedback Coverage and the Memory-Reward Trap in Self-Evolving Agent Memory via Reduced-Order Utility Statescited by 0
  38. 2026Skill Issue: Lessons from Optimizing Repository SKILLs for Coding Agentscited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026MetaEvo: A Meta-Optimization Framework for Experience-Driven Agent Evolution
  2. 2026Mem2Evolve: Towards Self-Evolving Agents via Co-Evolutionary Capability Expansion and Experience Distillation
  3. 2026Unified Context Evolution for LLM Agents
  4. 2026FreeEvolve: Learning to Evolve Beyond Fixed Loops
  5. 2026Experience Funnel: A State-Policy Alternating Loop for Self-Evolving Agents
  6. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning
  7. 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environments
  8. 2024A Survey on Self-Evolution of Large Language Models