# Prediction and Control with Temporal Segment Models

arXiv: https://arxiv.org/abs/1703.04070 · HTML: https://mlsanity.com/paper/1703.04070/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2017.

## Built on by (most cited first, 14 of 14)

- [PLAS: Latent Action Space for Offline Reinforcement Learning](https://mlsanity.com/paper/2011.07213/) (2020)
- [Plan To Predict: Learning an Uncertainty-Foreseeing Model for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2301.08502/) (2023)
- [On Rollouts in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2501.16918/) (2025)
- [State Sequences Prediction via Fourier Transform for Representation Learning](https://mlsanity.com/paper/2310.15888/) (2023)
- [MUSBO: Model-based Uncertainty Regularized and Sample Efficient Batch Optimization for Deployment Constrained Reinforcement Learning](https://mlsanity.com/paper/2102.11448/) (2021)
- [Latent Policies for Adversarial Imitation Learning](https://mlsanity.com/paper/2206.11299/) (2022)
- [Backward Imitation and Forward Reinforcement Learning via Bi-directional Model Rollouts](https://mlsanity.com/paper/2208.02434/) (2022)
- [RL, but don't do anything I wouldn't do](https://mlsanity.com/paper/2410.06213/) (2024)
- [Learning Reactive and Predictive Differentiable Controllers for Switching Linear Dynamical Models](https://mlsanity.com/paper/2103.14256/) (2021)
- [Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models](https://mlsanity.com/paper/2204.08573/) (2022)
- [Hierarchical Strategies for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2212.07397/) (2022)
- [Systematic Generalization in Neural Networks-based Multivariate Time Series Forecasting Models](https://mlsanity.com/paper/2102.05602/) (2021)
- [ED2: Environment Dynamics Decomposition World Models for Continuous Control](https://mlsanity.com/paper/2112.02817/) (2021)
- [Reaching Consensus in Cooperative Multi-Agent Reinforcement Learning with Goal Imagination](https://mlsanity.com/paper/2403.03172/) (2024)

## Abstract (arXiv)

We introduce a method for learning the dynamics of complex nonlinear systems based on deep generative models over temporal segments of states and actions. Unlike dynamics models that operate over individual discrete timesteps, we learn the distribution over future state trajectories conditioned on past state, past action, and planned future action trajectories, as well as a latent prior over action trajectories. Our approach is based on convolutional autoregressive models and variational autoencoders. It makes stable and accurate predictions over long horizons for complex, stochastic systems, effectively expressing uncertainty and modeling the effects of collisions, sensory noise, and action delays. The learned dynamics model and action prior can be used for end-to-end, fully differentiable trajectory optimization and model-based policy optimization, which we use to evaluate the performance and sample-efficiency of our method.