mlsanityData to 7 Oct 2026

Video Occupancy Models

arXiv 2407.09533 ↗First posted 25 Jun 2024cs.CV cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

We introduce a new family of video prediction models designed to support downstream control tasks. We call these models Video Occupancy models (VOCs). VOCs operate in a compact latent space, thus avoiding the need to make predictions about individual pixels. Unlike prior latent-space world models, VOCs directly predict the discounted distribution of future states in a single step, thus avoiding the need for multistep roll-outs. We show that both properties are beneficial when building predictive models of video for use in downstream control. Code is available at hrefhttps://github.com/manantomar/video-occupancy-modelstextttgithub.com/manantomar/video-occupancy-models.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

21 of 32 references.

References · 21

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2017Neural Discrete Representation Learningcited by 4,650
  7. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  8. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  9. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  10. 2018DeepMind Control Suitecited by 935
  11. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  12. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
9 more
  1. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  2. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  3. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
  4. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 222
  5. 2021Learning One Representation to Optimize All Rewardscited by 110
  6. 2016Deep Successor Reinforcement Learningcited by 72
  7. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 64
  8. 2020Generative Temporal Difference Learning for Infinite-Horizon Predictioncited by 43
  9. 2024A Distributional Analogue to the Successor Representationcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  2. 2025Temporal Difference Flowscited by 20
  3. 2025Intention-Conditioned Flow Occupancy Modelscited by 7
  4. 2026Compositional Planning with Jumpy World Modelscited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2025OccTENS: 3D Occupancy World Model via Temporal Next-Scale Prediction
  2. 2025SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Model
  3. 2026CascadeOcc: Rethinking 3D Occupancy World Models with Cascaded VQ Representations
  4. 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model
  5. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction
  6. 2025COME: Adding Scene-Centric Forecasting Control to Occupancy World Model
  7. 2025Back to the Features: DINO as a Foundation for Video World Models
  8. 2024Predicting Long-horizon Futures by Conditioning on Geometry and Time