# Learning to Act by Predicting the Future

arXiv: https://arxiv.org/abs/1611.01779 · HTML: https://mlsanity.com/paper/1611.01779/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2016.

## Built on by (most cited first, 30 of 36)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [A Survey of Embodied AI: From Simulators to Research Tasks](https://mlsanity.com/paper/2103.04918/) (2021)
- [Contrastive Learning as Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2206.07568/) (2022)
- [Pathdreamer: A World Model for Indoor Navigation](https://mlsanity.com/paper/2105.08756/) (2021)
- [Navigating to Objects in the Real World](https://mlsanity.com/paper/2212.00922/) (2022)
- [Robust Autonomy Emerges from Self-Play](https://mlsanity.com/paper/2502.03349/) (2025)
- [DINO-Foresight: Looking into the Future with DINO](https://mlsanity.com/paper/2412.11673/) (2024)
- [Which Mutual-Information Representation Learning Objectives are Sufficient for Control?](https://mlsanity.com/paper/2106.07278/) (2021)
- [Learning Belief Representations for Imitation Learning in POMDPs](https://mlsanity.com/paper/1906.09510/) (2019)
- [Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations](https://mlsanity.com/paper/2509.20478/) (2025)
- [A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals](https://mlsanity.com/paper/2408.05804/) (2024)
- [EgoEnv: Human-centric environment representations from egocentric video](https://mlsanity.com/paper/2207.11365/) (2022)
- [Teaching Agents how to Map: Spatial Reasoning for Multi-Object Navigation](https://mlsanity.com/paper/2107.06011/) (2021)
- [Navigating to Objects in Unseen Environments by Distance Prediction](https://mlsanity.com/paper/2202.03735/) (2022)
- [Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following](https://mlsanity.com/paper/2502.05454/) (2025)
- [Imitation by Predicting Observations](https://mlsanity.com/paper/2107.03851/) (2021)
- [OIL: Observational Imitation Learning](https://mlsanity.com/paper/1803.01129/) (2018)
- [MaAST: Map Attention with Semantic Transformersfor Efficient Visual Navigation](https://mlsanity.com/paper/2103.11374/) (2021)
- [Task-Induced Representation Learning](https://mlsanity.com/paper/2204.11827/) (2022)
- [Structure-Encoding Auxiliary Tasks for Improved Visual Representation in Vision-and-Language Navigation](https://mlsanity.com/paper/2211.11116/) (2022)
- [Advancing DRL Agents in Commercial Fighting Games: Training, Integration, and Agent-Human Alignment](https://mlsanity.com/paper/2406.01103/) (2024)
- [Advancing Semantic Future Prediction through Multimodal Visual Sequence Transformers](https://mlsanity.com/paper/2501.08303/) (2025)
- [Social Processes: Self-Supervised Meta-Learning over Conversational Groups for Forecasting Nonverbal Social Cues](https://mlsanity.com/paper/2107.13576/) (2021)
- [Embodied AI-Driven Operation of Smart Cities: A Concise Review](https://mlsanity.com/paper/2108.09823/) (2021)
- [GraphMapper: Efficient Visual Navigation by Scene Graph Generation](https://mlsanity.com/paper/2205.08325/) (2022)
- [Foresee then Evaluate: Decomposing Value Estimation with Latent Future Prediction](https://mlsanity.com/paper/2103.02225/) (2021)
- [Distilling Reinforcement Learning Tricks for Video Games](https://mlsanity.com/paper/2107.00703/) (2021)
- [SVL: Goal-Conditioned Reinforcement Learning as Survival Learning](https://mlsanity.com/paper/2604.17551/) (2026)
- [Deep Learning for Embodied Vision Navigation: A Survey](https://mlsanity.com/paper/2108.04097/) (2021)
- [Learning Goal-Conditioned Representations for Language Reward Models](https://mlsanity.com/paper/2407.13887/) (2024)

## Abstract (arXiv)

We present an approach to sensorimotor control in immersive environments. Our approach utilizes a high-dimensional sensory stream and a lower-dimensional measurement stream. The cotemporal structure of these streams provides a rich supervisory signal, which enables training a sensorimotor control model by interacting with the environment. The model is trained using supervised learning techniques, but without extraneous supervision. It learns to act based on raw sensory input from a complex three-dimensional environment. The presented formulation enables learning without a fixed goal at training time, and pursuing dynamically changing goals at test time. We conduct extensive experiments in three-dimensional simulations based on the classical first-person game Doom. The results demonstrate that the presented approach outperforms sophisticated prior formulations, particularly on challenging tasks. The results also show that trained models successfully generalize across environments and goals. A model trained using the presented approach won the Full Deathmatch track of the Visual Doom AI Competition, which was held in previously unseen environments.