# One Life to Learn: Inferring Symbolic World Models for Stochastic Environments from Unguided Exploration

arXiv: https://arxiv.org/abs/2510.12088 · HTML: https://mlsanity.com/paper/2510.12088/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in World models for language agents, one of 11 directions in World models.
- Builds on 30 of its 68 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Placed in

- World models / [World models for language agents](https://mlsanity.com/wm/world-models-for-language-agents/)

## Builds on (most cited first, 30 of 30)

- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Eureka: Human-Level Reward Design via Coding Large Language Models](https://mlsanity.com/paper/2310.12931/) (2023)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [Planning to Explore via Self-Supervised World Models](https://mlsanity.com/paper/2005.05960/) (2020)
- [Transformers are Sample-Efficient World Models](https://mlsanity.com/paper/2209.00588/) (2022)
- [Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks](https://mlsanity.com/paper/2306.13831/) (2023)
- [Language to Rewards for Robotic Skill Synthesis](https://mlsanity.com/paper/2306.08647/) (2023)
- [BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning](https://mlsanity.com/paper/1810.08272/) (2018)
- [Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning](https://mlsanity.com/paper/2305.14909/) (2023)
- [Benchmarking the Spectrum of Agent Capabilities](https://mlsanity.com/paper/2109.06780/) (2021)
- [Evaluating the World Model Implicit in a Generative Model](https://mlsanity.com/paper/2406.03689/) (2024)
- [WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment](https://mlsanity.com/paper/2402.12275/) (2024)
- [Pretraining Representations for Data-Efficient Reinforcement Learning](https://mlsanity.com/paper/2106.04799/) (2021)
- [DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents](https://mlsanity.com/paper/2406.06769/) (2024)
- [Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning](https://mlsanity.com/paper/2402.16801/) (2024)
- [An Introduction to Probabilistic Programming](https://mlsanity.com/paper/1809.10756/) (2018)
- [Learning to Synthesize Programs as Interpretable and Generalizable Policies](https://mlsanity.com/paper/2108.13643/) (2021)
- [Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search](https://mlsanity.com/paper/2405.15383/) (2024)
- [ReGAL: Refactoring Programs to Discover Generalizable Abstractions](https://mlsanity.com/paper/2401.16467/) (2024)
- [OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environments](https://mlsanity.com/paper/2306.08649/) (2023)
- [PoE-World: Compositional World Modeling with Products of Programmatic Experts](https://mlsanity.com/paper/2505.10819/) (2025)
- [MaestroMotif: Skill Design from Artificial Intelligence Feedback](https://mlsanity.com/paper/2412.08542/) (2024)
- [Assessing Adaptive World Models in Machines with Novel Games](https://mlsanity.com/paper/2507.12821/) (2025)
- [PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors](https://mlsanity.com/paper/2507.15550/) (2025)
- [PDDLEGO: Iterative Planning in Textual Environments](https://mlsanity.com/paper/2405.19793/) (2024)
- [Synthesizing world models for bilevel planning](https://mlsanity.com/paper/2503.20124/) (2025)
- [Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systems](https://mlsanity.com/paper/2505.17968/) (2025)
- [Improving Transformer World Models for Data-Efficient RL](https://mlsanity.com/paper/2502.01591/) (2025)

## Built on by (most cited first, 10 of 10)

- [Calibrate-Then-Act: Cost-Aware Exploration in LLM Agents](https://mlsanity.com/paper/2602.16699/) (2026)
- [Evolving Programmatic Skill Networks](https://mlsanity.com/paper/2601.03509/) (2026)
- [PatchWorld: Gradient-Free Optimization of Executable World Models for Agent Environments](https://mlsanity.com/paper/2605.30880/) (2026)
- [Mini Amusement Parks (MAPs): A Testbed for Modelling Business Decisions](https://mlsanity.com/paper/2511.15830/) (2025)
- [Joint Learning of Hierarchical Neural Options and Abstract World Model](https://mlsanity.com/paper/2602.02799/) (2026)
- [Autonomous Assessment of Generalizability of AI Agent Capabilities](https://mlsanity.com/paper/2512.16733/) (2025)
- [GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization](https://mlsanity.com/paper/2605.31464/) (2026)
- [Object-Centric Environment Modeling for Agentic Tasks](https://mlsanity.com/paper/2607.02846/) (2026)
- [Schema: Discovering Unknown Environments via Agentic Program Induction](https://mlsanity.com/paper/2609.39140/) (2026)
- [RoleWorld: Knowledge Transfer in World Modeling with Functional Role Abstractions](https://mlsanity.com/paper/2610.13172/) (2026)

## Abstract (arXiv)

Symbolic world modeling requires inferring and representing an environment's transitional dynamics as an executable program. Prior work has focused on largely deterministic environments with abundant interaction data, simple mechanics, and human guidance. We address a more realistic and challenging setting, learning in a complex, stochastic environment where the agent has only "one life" to explore a hostile environment without human guidance. We introduce OneLife, a framework that models world dynamics through conditionally-activated programmatic laws within a probabilistic programming framework. Each law operates through a precondition-effect structure, activating in relevant world states. This creates a dynamic computation graph that routes inference and optimization only through relevant laws, avoiding scaling challenges when all laws contribute to predictions about a complex, hierarchical state, and enabling the learning of stochastic dynamics even with sparse rule activation. To evaluate our approach under these demanding constraints, we introduce a new evaluation protocol that measures (a) state ranking, the ability to distinguish plausible future states from implausible ones, and (b) state fidelity, the ability to generate future states that closely resemble reality. We develop and evaluate our framework on Crafter-OO, our reimplementation of the Crafter environment that exposes a structured, object-oriented symbolic state and a pure transition function that operates on that state alone. OneLife can successfully learn key environment dynamics from minimal, unguided interaction, outperforming a strong baseline on 16 out of 23 scenarios tested. We also test OneLife's planning ability, with simulated rollouts successfully identifying superior strategies. Our work establishes a foundation for autonomously constructing programmatic world models of unknown, complex environments.