mlsanityData to 7 Oct 2026

Imagined Autocurricula

arXiv 2509.13341 ↗First posted 11 Sep 2025cs.AI cs.LGcited by 4

On the map: World models → World models for language agents

Abstract (arXiv)

Training agents to act in embodied environments typically requires vast training data or access to accurate simulation, neither of which exists for many cases in the real world. Instead, world models are emerging as an alternative leveraging offline, passively collected data, they make it possible to generate diverse worlds for training agents in simulation. In this work, we harness world models to generate imagined environments to train robust agents capable of generalizing to novel task variations. One of the challenges in doing this is ensuring the agent trains on useful generated data. We thus propose a novel approach, IMAC (Imagined Autocurricula), leveraging Unsupervised Environment Design (UED), which induces an automatic curriculum over generated worlds. In a series of challenging, procedurally generated environments, we show it is possible to achieve strong transfer performance on held-out environments, having trained only inside a world model learned from a narrower dataset. We believe this opens the path to utilizing larger-scale, foundation world models for generally capable agents.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 48 of its 59 references that are in the map. First posted on arXiv 11 Sep 2025.

Builds on

48 of 59 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  6. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  7. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  8. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  9. 2018Group Normalizationcited by 1,514
  10. 2018World Modelscited by 1,444
  11. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  12. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
36 more
  1. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  3. 2017Hindsight Experience Replaycited by 1,096
  4. 2015Prioritized Experience Replaycited by 1,070
  5. 2020Mastering Atari with Discrete World Modelscited by 990
  6. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  7. 2024Genie: Generative Interactive Environmentscited by 706
  8. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  9. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  10. 2023Learning Interactive Real-World Simulatorscited by 440
  11. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  12. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  13. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  14. 2017Automated Curriculum Learning for Neural Networkscited by 251
  15. 2024Reinforcement learningcited by 236
  16. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  17. 2017Teacher-Student Curriculum Learningcited by 197
  18. 2020The NetHack Learning Environmentcited by 196
  19. 2020Prioritized Level Replaycited by 190
  20. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  21. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  22. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  23. 2018Assessing Generalization in Deep Reinforcement Learningcited by 132
  24. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 113
  25. 2018Episodic Curiosity through Reachabilitycited by 109
  26. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 102
  27. 2020Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learningcited by 95
  28. 2018Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generationcited by 80
  29. 2018Gotta Learn Fast: A New Benchmark for Generalization in RLcited by 59
  30. 2020Interference and Generalization in Temporal Difference Learningcited by 41
  31. 2021Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environmentcited by 41
  32. 2023Conditional Mutual Information for Disentangled Representations in Reinforcement Learningcited by 27
  33. 2020Instance based Generalization in Reinforcement Learningcited by 17
  34. 2024Adversarial Environment Design via Regret-Guided Diffusion Modelscited by 9
  35. 2023A Survey on Offline Model-Based Reinforcement Learningcited by 7
  36. 2020Exemplar Normalization for Learning Deep Representationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 6
  2. 2025Beyond Fixed Tasks: Unsupervised Environment Design for Task-Level Pairscited by 2
  3. 2026Improving Regret Approximation for Unsupervised Dynamic Environment Generationcited by 1
  4. 2026PROWL: Prioritized Regret-Driven Optimization for World Model Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning Task-Sufficient World Models by Synergizing Agentic Exploration and Structured Modeling
  2. 2025Exploration-Driven Generative Interactive Environments
  3. 2026SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning
  4. 2025AutoEnv: Automated Environments for Measuring Cross-Environment Agent Learning
  5. 2025World-in-World: World Models in a Closed-Loop World
  6. 2026AutoWorldModel-Bench: A State-Centric Benchmark for Automated World-Model Research
  7. 2023Reward-Free Curricula for Training Robust World Models
  8. 2021Dropout's Dream Land: Generalization from Learned Simulators to Reality