# Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks

arXiv: https://arxiv.org/abs/2610.11794 · HTML: https://mlsanity.com/paper/2610.11794/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in World models for language agents, one of 11 directions in World models.
- Builds on 21 of its 50 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Placed in

- World models / [World models for language agents](https://mlsanity.com/wm/world-models-for-language-agents/)
- Synthetic data and self-improvement for LLMs / [Self-evolving LLM agents](https://mlsanity.com/synth/self-evolving-llm-agents/)

## Builds on (most cited first, 21 of 21)

- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Mastering Atari Games with Limited Data](https://mlsanity.com/paper/2111.00210/) (2021)
- [Memento: Fine-tuning LLM Agents without Fine-tuning LLMs](https://mlsanity.com/paper/2508.16153/) (2025)
- [Memento-Skills: Let Agents Design Agents](https://mlsanity.com/paper/2603.18743/) (2026)
- [Continual Harness: Online Adaptation for Self-Improving Foundation Agents](https://mlsanity.com/paper/2605.09998/) (2026)
- [ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence](https://mlsanity.com/paper/2603.24621/) (2026)
- [WALL-E: World Alignment by Rule Learning Improves World Model-based LLM Agents](https://mlsanity.com/paper/2410.07484/) (2024)
- [Code World Models for General Game Playing](https://mlsanity.com/paper/2510.04542/) (2025)
- [Executable World Models for ARC-AGI-3 in the Era of Coding Agents](https://mlsanity.com/paper/2605.05138/) (2026)
- [Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning](https://mlsanity.com/paper/2608.27549/) (2026)
- [A Review for Deep Reinforcement Learning in Atari:Benchmarks, Challenges, and Solutions](https://mlsanity.com/paper/2112.04145/) (2021)
- [OPINE-World: Programmatic World Modeling with Ontology-error-Prioritized Interactive Exploration for ARC-AGI-3](https://mlsanity.com/paper/2607.01531/) (2026)
- [GDI: Rethinking What Makes Reinforcement Learning Different From Supervised Learning](https://mlsanity.com/paper/2106.06232/) (2021)
- [Workspace Optimization: How to Train Your Agent](https://mlsanity.com/paper/2605.09650/) (2026)
- [Tycho: Active Abstraction with Programmatic World Models for ARC-AGI-3](https://mlsanity.com/paper/2607.28287/) (2026)
- [Memento 2: Learning by Stateful Reflective Memory](https://mlsanity.com/paper/2512.22716/) (2025)
- [Graph-Based Exploration for ARC-AGI-3 Interactive Reasoning Tasks](https://mlsanity.com/paper/2512.24156/) (2025)
- [Twin: Playing an Unknown Game with a Test-Time Digital Twin](https://mlsanity.com/paper/2608.14490/) (2026)
- [Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve ARC-AGI-3?](https://mlsanity.com/paper/2607.15439/) (2026)
- [NVIDIA-labs OO Agents: Native Python Object-Oriented Agents](https://mlsanity.com/paper/2607.20709/) (2026)
- [Belief-Calibrated Optimization: An Explicit World Model for Agentic Optimization](https://mlsanity.com/paper/2609.01861/) (2026)

## Abstract (arXiv)

Learning to act in unfamiliar environments requires agents to infer how the world works and revise that understanding as new evidence arrives. Yet limited observations can support multiple world models that explain past interactions but predict different outcomes in unseen states. We introduce Memento 3, building on the Memento series to enable frozen LLM agents to continually learn explicit world models through external memory. The agent maintains a natural-language rulebook as persistent semantic memory, recording revisable hypotheses about environment dynamics while leaving unknown aspects underspecified. It compiles this rulebook into executable code for prediction and planning. Through a continual loop of observation, reflection, rule revision, compilation, and verification, the agent uses prediction errors to refine both the rulebook and its code. Updated code is accepted only when the LLM judges it faithful to the rulebook and cell-exact replay reproduces the observed transitions. We investigate this process as a model-based route to recursive self-improvement (RSI): the agent autonomously explores the environment, revises its world model, and uses verified updates to guide subsequent interaction and learning, while the underlying LLM remains fixed. A population extension maintains multiple world models in parallel, sharing interaction evidence and using their predictions to guide exploration. On ARC-AGI-3, the single-model agent clears every level of all 25 public games, achieves a mean Relative Human Action Efficiency (RHAE) of 100.0, and uses 44% of the human action count. In an Atari Pong case study, a learned feedback controller wins 21:0 in each of three evaluated episodes with different openings, without further LLM calls.