# Multi-Step Prediction in Linearized Latent State Spaces for Representation Learning

arXiv: https://arxiv.org/abs/2209.01127 · HTML: https://mlsanity.com/paper/2209.01127/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 17 references that are in the map.
- First posted on arXiv 2 Sep 2022.

## Builds on (most cited first, 10 of 10)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [A Generalist Agent](https://mlsanity.com/paper/2205.06175/) (2022)
- [Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images](https://mlsanity.com/paper/1506.07365/) (2015)
- [A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems](https://mlsanity.com/paper/2203.01387/) (2022)
- [Deep Reinforcement Learning in Large Discrete Action Spaces](https://mlsanity.com/paper/1512.07679/) (2015)
- [Prediction, Consistency, Curvature: Representation Learning for Locally-Linear Control](https://mlsanity.com/paper/1909.01506/) (2019)
- [Predictive Coding for Locally-Linear Control](https://mlsanity.com/paper/2003.01086/) (2020)
- [Partially Observable Total-Cost Markov Decision Processes with Weakly Continuous Transition Probabilities](https://mlsanity.com/paper/1401.2168/) (2014)

## Built on by (most cited first, 1 of 1)

- [Multi-Step Embed to Control: A Novel Deep Learning-based Approach for Surrogate Modelling in Reservoir Simulation](https://mlsanity.com/paper/2409.09920/) (2024)

## Abstract (arXiv)

In this paper, we derive a novel method as a generalization over LCEs such as E2C. The method develops the idea of learning a locally linear state space, by adding a multi-step prediction, thus allowing for more explicit control over the curvature. We show, that the method outperforms E2C without drastic model changes which come with other works, such as PCC and P3C. We discuss the relation between E2C and the presented method and derived update equations. We provide empirical evidence, which suggests that by considering the multi-step prediction our method - ms-E2C - allows to learn much better latent state spaces in terms of curvature and next state predictability. Finally, we also discuss certain stability challenges we encounter with multi-step predictions and the ways to mitigate them.