# Latent World Models For Intrinsically Motivated Exploration

arXiv: https://arxiv.org/abs/2010.02302 · HTML: https://mlsanity.com/paper/2010.02302/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Latent predictive world models, one of 11 directions in World models.
- Builds on 2 of its 36 references that are in the map.
- First posted on arXiv 5 Oct 2020.

## Placed in

- World models / [Latent predictive world models](https://mlsanity.com/wm/latent-predictive-world-models/)

## Builds on (most cited first, 2 of 2)

- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Planning to Explore via Self-Supervised World Models](https://mlsanity.com/paper/2005.05960/) (2020)

## Built on by (most cited first, 15 of 15)

- [Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning](https://mlsanity.com/paper/2312.05230/) (2023)
- [An information-theoretic perspective on intrinsic motivation in reinforcement learning: a survey](https://mlsanity.com/paper/2209.08890/) (2022)
- [Flipping Coins to Estimate Pseudocounts for Exploration in Reinforcement Learning](https://mlsanity.com/paper/2306.03186/) (2023)
- [Time to augment self-supervised visual representation learning](https://mlsanity.com/paper/2207.13492/) (2022)
- [Concept Graph Neural Networks for Surgical Video Understanding](https://mlsanity.com/paper/2202.13402/) (2022)
- [Leveraging Skills from Unlabeled Prior Data for Efficient Online Exploration](https://mlsanity.com/paper/2410.18076/) (2024)
- [DisTop: Discovering a Topological representation to learn diverse and rewarding skills](https://mlsanity.com/paper/2106.03853/) (2021)
- [Maximum Entropy Model-based Reinforcement Learning](https://mlsanity.com/paper/2112.01195/) (2021)
- [Beyond Surprise: Improving Exploration Through Surprise Novelty](https://mlsanity.com/paper/2308.04836/) (2023)
- [The Exploration-Exploitation Dilemma Revisited: An Entropy Perspective](https://mlsanity.com/paper/2408.09974/) (2024)
- [Sample-efficient Real-time Planning with Curiosity Cross-Entropy Method and Contrastive Learning](https://mlsanity.com/paper/2303.03787/) (2023)
- [Curiosity-Driven Multi-Agent Exploration with Mixed Objectives](https://mlsanity.com/paper/2210.16468/) (2022)
- [Time-Myopic Go-Explore: Learning A State Representation for the Go-Explore Paradigm](https://mlsanity.com/paper/2301.05635/) (2023)
- [Exploration by Random Distribution Distillation](https://mlsanity.com/paper/2505.11044/) (2025)
- [Temporal Alignment for History Representation in Reinforcement Learning](https://mlsanity.com/paper/2204.03525/) (2022)

## Abstract (arXiv)

In this work we consider partially observable environments with sparse rewards. We present a self-supervised representation learning method for image-based observations, which arranges embeddings respecting temporal distance of observations. This representation is empirically robust to stochasticity and suitable for novelty detection from the error of a predictive forward model. We consider episodic and life-long uncertainties to guide the exploration. We propose to estimate the missing information about the environment with the world model, which operates in the learned latent space. As a motivation of the method, we analyse the exploration problem in a tabular Partially Observable Labyrinth. We demonstrate the method on image-based hard exploration environments from the Atari benchmark and report significant improvement with respect to prior work. The source code of the method and all the experiments is available at https://github.com/htdt/lwm.