mlsanityData to 7 Oct 2026

Discovering and Achieving Goals via World Models

arXiv 2110.09514 ↗First posted 18 Oct 2021cs.LG cs.AI cs.CVcited by 131

On the map: World models → Embodied and robot world models

Abstract (arXiv)

How can artificial agents learn to solve many diverse tasks in complex visual environments in the absence of any supervision? We decompose this question into two problems: discovering new goals and learning to reliably achieve them. We introduce Latent Explorer Achiever (LEXA), a unified solution to these that learns a world model from image inputs and uses it to train an explorer and an achiever policy from imagined rollouts. Unlike prior methods that explore by reaching previously visited states, the explorer plans to discover unseen surprising states through foresight, which are then used as diverse targets for the achiever to practice. After the unsupervised phase, LEXA solves tasks specified as goal images zero-shot without any additional learning. LEXA substantially outperforms previous approaches to unsupervised goal-reaching, both on prior benchmarks and on a new challenging benchmark with a total of 40 test tasks spanning across four standard robotic manipulation and locomotion domains. LEXA further achieves goals that require interacting with multiple objects in sequence. Finally, to demonstrate the scalability and generality of LEXA, we train a single general agent across four distinct environments. Code and videos at https://orybkin.github.io/lexa/

Key facts. Cited by 131 papers in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 46 of its 58 references that are in the map. First posted on arXiv 18 Oct 2021.

Builds on

46 of 58 references.

References · 46

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  8. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  9. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  10. 2017Hindsight Experience Replaycited by 1,096
  11. 2020Mastering Atari with Discrete World Modelscited by 990
  12. 2018Exploration by Random Network Distillationcited by 966
34 more
  1. 2018DeepMind Control Suitecited by 935
  2. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  3. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  4. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  5. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  6. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  7. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  8. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
  9. 2019Self-Supervised Exploration via Disagreementcited by 263
  10. 2016Variational Intrinsic Controlcited by 262
  11. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  12. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  13. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  14. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
  15. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  16. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  17. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 139
  18. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  19. 2018Variational Option Discovery Algorithmscited by 121
  20. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
  21. 2018Zero-Shot Visual Imitationcited by 114
  22. 2018Model-Based Active Explorationcited by 105
  23. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  24. 2020Automatic Curriculum Learning through Value Disagreementcited by 86
  25. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  26. 2019Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discoverycited by 65
  27. 2018Learning and Querying Fast Generative Models for Reinforcement Learningcited by 48
  28. 2020Model-Based Visual Planning with Self-Supervised Functional Distancescited by 41
  29. 2021Clockwork Variational Autoencoderscited by 40
  30. 2020Ready Policy One: World Building Through Active Learningcited by 33
  31. 2019Learning to Reach Goals via Iterated Supervised Learningcited by 26
  32. 2019Self-supervised Learning of Image Embedding for Continuous Controlcited by 16
  33. 2021Variational Empowerment as Representation Learning for Goal-Based Reinforcement Learningcited by 15
  34. 2019MULEX: Disentangling Exploitation from Exploration in Deep RLcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 23; 2023: 29; 2024: 34; 2025: 32; 2026: 13.

131 papers.

Sort:

By area:

  1. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
  2. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  3. 2022Multi-Game Decision Transformerscited by 212
  4. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  5. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  6. 2023Structured World Models from Human Videoscited by 144
  7. 2022Deep Hierarchical Planning from Pixelscited by 96
  8. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learningcited by 93
  9. 2024Foundation Policies with Hilbert Representationscited by 89
  10. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
  11. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  12. 2024World Models for Autonomous Driving: An Initial Surveycited by 79
38 more
  1. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Makingcited by 53
  2. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 51
  3. 2022Skill-based Model-based Reinforcement Learningcited by 50
  4. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 50
  5. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  6. 2024Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradientcited by 49
  7. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  8. 2023SMART: Self-supervised Multi-task pretrAining with contRol Transformerscited by 44
  9. 2025What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?cited by 37
  10. 2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31
  11. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximizationcited by 30
  12. 2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failurescited by 30
  13. 2022Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulationcited by 29
  14. 2024Entity-Centric Reinforcement Learning for Object Manipulation from Pixelscited by 27
  15. 2025SIMA 2: A Generalist Embodied Agent for Virtual Worldscited by 26
  16. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
  17. 2023Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shapingcited by 25
  18. 2022The Free Energy Principle for Perception and Action: A Deep Learning Perspectivecited by 24
  19. 2023SPRINT: Scalable Policy Pre-Training via Language Instruction Relabelingcited by 24
  20. 2025Reward Models in Deep Reinforcement Learning: A Surveycited by 24
  21. 2023ALAN: Autonomously Exploring Robotic Agents in the Real Worldcited by 23
  22. 2023Accelerating Exploration with Unlabeled Prior Datacited by 21
  23. 2025Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flowcited by 21
  24. 2023Reparameterized Policy Learning for Multimodal Trajectory Optimizationcited by 20
  25. 2023Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Datacited by 19
  26. 2024Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learningcited by 18
  27. 2024TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representationscited by 17
  28. 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 16
  29. 2023Curious Replay for Model-based Adaptationcited by 16
  30. 2025A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Searchcited by 16
  31. 2023COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RLcited by 14
  32. 2024Solving Minimum-Cost Reach Avoid using Reinforcement Learningcited by 14
  33. 2022Latent State Marginalization as a Low-cost Approach for Improving Explorationcited by 13
  34. 2023Imitating Graph-Based Planning with Goal-Conditioned Policiescited by 13
  35. 2023Reward-Free Curricula for Training Robust World Modelscited by 13
  36. 2023Breadcrumbs to the Goal: Goal-Conditioned Exploration from Human-in-the-Loop Feedbackcited by 13
  37. 2024MAMBA: an Effective World Model Approach for Meta-Reinforcement Learningcited by 13
  38. 2024CarDreamer: Open-Source Learning Platform for World Model based Autonomous Drivingcited by 13

50 most cited of 131 All 131 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026The Planning Limits of Latent World Models
  2. 2025GoalLadder: Incremental Goal Discovery with Vision-Language Models
  3. 2026One from Infinity: Actualizing Futures from Pretrained World Models into Robot Actions
  4. 2026Latent Action as Intention Enables Efficient Future Imagination for World Action Models
  5. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration
  6. 2025Act2Goal: From World Model To General Goal-conditioned Policy
  7. 2026OpenFrontier: General Navigation with Visual-Language Grounded Frontiers
  8. 2024RoboDreamer: Learning Compositional World Models for Robot Imagination