# Self-Evolving World Models for LLM Agent Planning

arXiv: https://arxiv.org/abs/2606.30639 · HTML: https://mlsanity.com/paper/2606.30639/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in World models for language agents, one of 11 directions in World models.
- Builds on 46 of its 58 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Placed in

- World models / [World models for language agents](https://mlsanity.com/wm/world-models-for-language-agents/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)
- Synthetic data and self-improvement for LLMs / [Self-evolving LLM agents](https://mlsanity.com/synth/self-evolving-llm-agents/)

## Builds on (most cited first, 30 of 46)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [Editing Large Language Models: Problems, Methods, and Opportunities](https://mlsanity.com/paper/2305.13172/) (2023)
- [AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents](https://mlsanity.com/paper/2401.13178/) (2024)
- [ScienceWorld: Is your Agent Smarter than a 5th Grader?](https://mlsanity.com/paper/2203.07540/) (2022)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels](https://mlsanity.com/paper/2603.19312/) (2026)
- [Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors](https://mlsanity.com/paper/2211.11031/) (2022)
- [WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning](https://mlsanity.com/paper/2411.02337/) (2024)
- [R-Zero: Self-Evolving Reasoning LLM from Zero Data](https://mlsanity.com/paper/2508.05004/) (2025)
- [Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control](https://mlsanity.com/paper/2306.07863/) (2023)
- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](https://mlsanity.com/paper/2411.14499/) (2024)
- [Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents](https://mlsanity.com/paper/2411.06559/) (2024)
- [Neural Episodic Control](https://mlsanity.com/paper/1703.01988/) (2017)
- [Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation](https://mlsanity.com/paper/2410.13232/) (2024)
- [Model-Free Episodic Control](https://mlsanity.com/paper/1606.04460/) (2016)
- [Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning](https://mlsanity.com/paper/2511.16043/) (2025)
- [Agent Planning with World Knowledge Model](https://mlsanity.com/paper/2405.14205/) (2024)
- [A Comprehensive Survey on World Models for Embodied AI](https://mlsanity.com/paper/2510.16732/) (2025)
- [AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learning](https://mlsanity.com/paper/2405.16247/) (2024)
- [WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Model](https://mlsanity.com/paper/2504.21024/) (2025)
- [Contextual Experience Replay for Self-Improvement of Language Agents](https://mlsanity.com/paper/2506.06698/) (2025)

## Built on by (most cited first, 13 of 13)

- [Dual-Frontier: When Can an Agent Trust Its World Model?](https://mlsanity.com/paper/2609.26293/) (2026)
- [Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight](https://mlsanity.com/paper/2610.08077/) (2026)
- [Mint-Agent: Introducing Finance-Native Agentic Foundation Models](https://mlsanity.com/paper/2608.16386/) (2026)
- [DreamLedger: Where to Refuse World-Model Imagination Using Execution-Settled Credit](https://mlsanity.com/paper/2608.23863/) (2026)
- [How To Train Your World Model: Fine-tuning vs RAG for LM-based World Modeling](https://mlsanity.com/paper/2610.02542/) (2026)
- [From Traces to Agentic Worlds: Agentic Language World Models for Interactive Environment Simulation](https://mlsanity.com/paper/2610.06100/) (2026)
- [ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?](https://mlsanity.com/paper/2610.07792/) (2026)
- [MemoWM: How World Models Change What Agents Need to Remember](https://mlsanity.com/paper/2610.10778/) (2026)
- [Safe, Persistent, and Evolving Agent Harness for Understanding Partially Observable Worlds](https://mlsanity.com/paper/2610.11552/) (2026)
- [EvoBelief: Falsifiable World Models for Self-Improving LLM Agents](https://mlsanity.com/paper/2610.12789/) (2026)
- [Predict Before You Modify: Learning Improvement Dynamics for Self-Evolving Agents](https://mlsanity.com/paper/2610.13991/) (2026)
- [Trace-to-Architecture: Capability Bottleneck Diagnosis for Harness Revision](https://mlsanity.com/paper/2610.16725/) (2026)
- [D-World: An Explicit World-Language-Action Model with Recursive Self-Improvement under Dual Control](https://mlsanity.com/paper/2610.17114/) (2026)

## Abstract (arXiv)

World models offer a principled way to equip long-horizon LLM agents with foresight: predictions of action consequences before execution. However, unreliable foresight can be ignored, misused, or even degrade downstream decision-making. In this paper, we introduce WorldEvolver, a self-evolving world model framework that revises its deployment-time context while keeping the downstream agent and all model parameters frozen. WorldEvolver integrates three modules: (i) Episodic Memory, which exploits real action transitions through retrieval-based simulation; (ii) Semantic Memory, which extracts persistent heuristic rules from prediction-observation mismatches; and (iii) Selective Foresight, which filters low-confidence predictions before integrating them into agent reasoning context. We evaluate WorldEvolver on ALFWorld and ScienceWorld, measuring world model prediction accuracy on Word2World and downstream agent success rate on AgentBoard. Extensive experiments show that WorldEvolver achieves the highest prediction accuracy across three backbones and leads other world model baselines on downstream agent success rate, demonstrating that test-time memory revision enhances both predictive fidelity and planning performance.