# Learning to Predict Without Looking Ahead: World Models Without Forward Prediction

arXiv: https://arxiv.org/abs/1910.13038 · HTML: https://mlsanity.com/paper/1910.13038/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Latent dynamics models and model-based RL, one of 11 directions in World models.
- Builds on 6 of its 68 references that are in the map.
- First posted on arXiv 29 Oct 2019.

## Placed in

- World models / [Latent dynamics models and model-based RL](https://mlsanity.com/wm/latent-dynamics-models-and-model-based-rl/)

## Builds on (most cited first, 6 of 6)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning](https://mlsanity.com/paper/1708.02596/) (2017)
- [On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models](https://mlsanity.com/paper/1511.09249/) (2015)
- [Deep Neuroevolution of Recurrent and Discrete World Models](https://mlsanity.com/paper/1906.08857/) (2019)

## Built on by (most cited first, 15 of 15)

- [Diffusion Autoencoders: Toward a Meaningful and Decodable Representation](https://mlsanity.com/paper/2111.15640/) (2021)
- [The Alignment Problem from a Deep Learning Perspective](https://mlsanity.com/paper/2209.00626/) (2022)
- [Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs](https://mlsanity.com/paper/2110.05038/) (2021)
- [Generative Planning for Temporally Coordinated Exploration in Reinforcement Learning](https://mlsanity.com/paper/2201.09765/) (2022)
- [Visualizing MuZero Models](https://mlsanity.com/paper/2102.12924/) (2021)
- [Distillation of RL Policies with Formal Guarantees via Variational Abstraction of Markov Decision Processes (Technical Report)](https://mlsanity.com/paper/2112.09655/) (2021)
- [EUCLID: Towards Efficient Unsupervised Reinforcement Learning with Multi-choice Dynamics Model](https://mlsanity.com/paper/2210.00498/) (2022)
- [Detecting and Adapting to Novelty in Games](https://mlsanity.com/paper/2106.02204/) (2021)
- [Towards Unraveling and Improving Generalization in World Models](https://mlsanity.com/paper/2501.00195/) (2024)
- [Go Beyond Imagination: Maximizing Episodic Reachability with World Models](https://mlsanity.com/paper/2308.13661/) (2023)
- [Hard Attention Control By Mutual Information Maximization](https://mlsanity.com/paper/2103.06371/) (2021)
- [Intuitive Physics Guided Exploration for Sample Efficient Sim2real Transfer](https://mlsanity.com/paper/2104.08795/) (2021)
- [Predictive Control Using Learned State Space Models via Rolling Horizon Evolution](https://mlsanity.com/paper/2106.13911/) (2021)
- [Performance-Driven Controller Tuning via Derivative-Free Reinforcement Learning](https://mlsanity.com/paper/2209.04854/) (2022)
- [Optimizing Sensory Neurons: Nonlinear Attention Mechanisms for Accelerated Convergence in Permutation-Invariant Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/2506.00691/) (2025)

## Abstract (arXiv)

Much of model-based reinforcement learning involves learning a model of an agent's world, and training an agent to leverage this model to perform a task more efficiently. While these models are demonstrably useful for agents, every naturally occurring model of the world of which we are aware---e.g., a brain---arose as the byproduct of competing evolutionary pressures for survival, not minimization of a supervised forward-predictive loss via gradient descent. That useful models can arise out of the messy and slow optimization process of evolution suggests that forward-predictive modeling can arise as a side-effect of optimization under the right circumstances. Crucially, this optimization process need not explicitly be a forward-predictive loss. In this work, we introduce a modification to traditional reinforcement learning which we call observational dropout, whereby we limit the agents ability to observe the real environment at each timestep. In doing so, we can coerce an agent into learning a world model to fill in the observation gaps during reinforcement learning. We show that the emerged world model, while not explicitly trained to predict the future, can help the agent learn key skills required to perform well in its environment. Videos of our results available at https://learningtopredict.github.io/