# On the Benefits of Instance Decomposition in Video Prediction Models

arXiv: https://arxiv.org/abs/2501.10562 · HTML: https://mlsanity.com/paper/2501.10562/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jan 2025.

## Built on by (most cited first, 1 of 1)

- [Flow and Depth Assisted Video Prediction with Latent Transformer](https://mlsanity.com/paper/2511.16484/) (2025)

## Abstract (arXiv)

Video prediction is a crucial task for intelligent agents such as robots and autonomous vehicles, since it enables them to anticipate and act early on time-critical incidents. State-of-the-art video prediction methods typically model the dynamics of a scene jointly and implicitly, without any explicit decomposition into separate objects. This is challenging and potentially sub-optimal, as every object in a dynamic scene has their own pattern of movement, typically somewhat independent of others. In this paper, we investigate the benefit of explicitly modeling the objects in a dynamic scene separately within the context of latent-transformer video prediction models. We conduct detailed and carefully-controlled experiments on both synthetic and real-world datasets; our results show that decomposing a dynamic scene leads to higher quality predictions compared with models of a similar capacity that lack such decomposition.