mlsanityData to 14 Oct 2026

Back to the Features: DINO as a Foundation for Video World Models

arXiv 2507.19468 ↗First posted 25 Jul 2025cs.CVcited by 58

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

We present DINO-world, a powerful generalist video world model trained to predict future frames in the latent space of DINOv2. By leveraging a pre-trained image encoder and training a future predictor on a large-scale uncurated video dataset, DINO-world learns the temporal dynamics of diverse scenes, from driving and indoor scenes to simulated environments. We show that DINO-world outperforms previous models on a variety of video prediction benchmarks, e.g. segmentation and depth forecasting, and demonstrates strong understanding of intuitive physics. Furthermore, we show that it is possible to fine-tune the predictor on observation-action trajectories. The resulting action-conditioned world model can be used for planning by simulating candidate trajectories in latent space.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 54 of its 70 references that are in the map. First posted on arXiv 25 Jul 2025.

Builds on

54 of 70 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  2. 2024WorldSimBench: Towards Video Generation Models as World Simulators23 Oct
  3. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond6 May
  4. 2024Genie: Generative Interactive Environments23 Feb
  5. 2023Learning Interactive Real-World Simulators9 Oct
  6. 2022MAGVIT: Masked Generative Video Transformer10 Dec
  7. 2021VideoGPT: Video Generation using VQ-VAE and Transformers20 Apr

Other references · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2017Neural Discrete Representation Learningcited by 4,712
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  8. 2017The Kinetics Human Action Video Datasetcited by 1,909
  9. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  10. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  11. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  12. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
35 more
  1. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  2. 2023Vision Transformers Need Registerscited by 871
  3. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  4. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  5. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  6. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  7. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  8. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
  9. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  10. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  11. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  12. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
  13. 2022Transformers are Sample-Efficient World Modelscited by 328
  14. 2023Photorealistic Video Generation with Diffusion Modelscited by 306
  15. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
  16. 2018Stochastic Video Generation with a Learned Priorcited by 245
  17. 2017Stochastic Variational Video Predictioncited by 234
  18. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 212
  19. 2021Anticipative Video Transformercited by 200
  20. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 185
  21. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 172
  22. 2025Do generative video models understand physical principles?cited by 161
  23. 2016Video Pixel Networkscited by 108
  24. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 89
  25. 2018IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoningcited by 73
  26. 2025Intuitive physics understanding emerges from self-supervised pretraining on natural videoscited by 70
  27. 2017Recurrent Environment Simulatorscited by 68
  28. 2017Predicting Deeper into the Future of Semantic Segmentationcited by 51
  29. 2024DINO-Foresight: Looking into the Future with DINOcited by 45
  30. 2022Anticipative Feature Fusion Transformer for Multi-Modal Action Anticipationcited by 41
  31. 2018Predicting Future Instance Segmentation by Forecasting Convolutional Featurescited by 37
  32. 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modelingcited by 35
  33. 2024Rethinking Patch Dependence for Masked Autoencoderscited by 29
  34. 2023GRASP: A novel benchmark for evaluating language GRounding And Situated Physics understanding in multimodal language modelscited by 28
  35. 2019Segmenting the Futurecited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 22; 2026: 36.

58 papers.

Sort:

By area:

  1. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 117
  2. 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
  3. 2025VL-JEPA: Joint Embedding Predictive Architecture for Vision-languagecited by 68
  4. 2026Learning Latent Action World Models In The Wildcited by 47
  5. 2025What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?cited by 43
  6. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 33
  7. 2026Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Modelscited by 22
  8. 2025Block-Recurrent Dynamics in Vision Transformerscited by 20
  9. 2026BridgeV2W: Bridging Video Generation Models to Embodied World Models via Embodiment Maskscited by 20
  10. 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Predictioncited by 16
  11. 2025Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresightcited by 14
  12. 2026Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directionscited by 13
38 more
  1. 2026VGGT-World: Transforming VGGT into an Autoregressive Geometry World Modelcited by 13
  2. 2025Rethinking JEPA: Compute-Efficient Video SSL with Frozen Teacherscited by 12
  3. 2026VideoWorld 2: Learning Transferable Knowledge from Real-world Videoscited by 12
  4. 2025Learning Interactive World Model for Object-Centric Reinforcement Learningcited by 11
  5. 2026Language-Conditioned World Modeling for Visual Navigationcited by 10
  6. 2026A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokenscited by 10
  7. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  8. 2025LongScape: Advancing Long-Horizon Embodied World Models with Context-Aware MoEcited by 7
  9. 2025GoViG: Goal-Conditioned Visual Navigation Instruction Generation via Multimodal Reasoningcited by 6
  10. 2025MoWM: Mixture-of-World-Models for Embodied Planning via Latent-to-Pixel Feature Modulationcited by 5
  11. 2025Foundation Visual Encoders Are Secretly Few-Shot Anomaly Detectorscited by 5
  12. 2025SoundReactor: Frame-level Online Video-to-Audio Generationcited by 5
  13. 2025SeeU: Seeing the Unseen World via 4D Dynamics-aware Generationcited by 5
  14. 2025VFMF: World Modeling by Forecasting Vision Foundation Model Featurescited by 5
  15. 2026Building Social World Models with Large Language Modelscited by 5
  16. 2026Latent World Models with Monotone Planning Costs for Image-Goal Navigationcited by 5
  17. 2026MaDiS: Taming Masked Diffusion Language Models for Sign Language Generationcited by 4
  18. 2026Learning Visual Feature-Based World Models via Residual Latent Actioncited by 4
  19. 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AIcited by 3
  20. 2025A Geometric Unification of Concept Learning with Concept Conescited by 3
  21. 2025Spectral Representation-based Reinforcement Learningcited by 3
  22. 2026Failure Detection for Surgical Robot Imitation Policies via Flow-Matching World Modelingcited by 3
  23. 2025X-WIN: Building Chest Radiograph World Model via Predictive Sensingcited by 2
  24. 2026Envisioning the Future, One Step at a Timecited by 2
  25. 2026Text-Conditional JEPA for Learning Semantically Rich Visual Representationscited by 2
  26. 2026Learning Task-Sufficient World Models by Synergizing Agentic Exploration and Structured Modelingcited by 2
  27. 2026Uncertainty-Aware World Model for Aerial Image-Goal Navigationcited by 2
  28. 2025PhysFire-WM: A Physics-Informed World Model for Emulating Fire Spread Dynamicscited by 1
  29. 2026Control-DINO: Feature Space Conditioning for Controllable Image-to-Video Diffusioncited by 1
  30. 2026Representations Before Pixels: Semantics-Guided Hierarchical Video Predictioncited by 1
  31. 2026Learning Long-term Motion Embeddings for Efficient Kinematics Generationcited by 1
  32. 2026Envision4D: Envisioning Visual Futures via Feed-forward 4D Gaussian Splatting for Autonomous Drivingcited by 1
  33. 2026Rethinking Representations for World-Action Modelingcited by 1
  34. 2025SRNN: Spatiotemporal Relational Neural Network for Intuitive Physics Understandingcited by 0
  35. 2026Pixel-level Scene Understanding in One Token: Visual States Need What-is-Where Compositioncited by 0
  36. 2026Video Generation with Predictive Latentscited by 0
  37. 2026Back to Parsimonious Latents: Learning Task-Centric World Models from Visual Foundationscited by 0
  38. 2026Self-Supervised Learning of Structured Dynamics from Videoscited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026MiniWorld: Democratizing the Training of Video World Models from Scratch
  2. 2026DriveVA: Video Action Models are Zero-Shot Drivers
  3. 2026World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models
  4. 2026Diffusion Transformer World-Action Model for AV Scene Prediction
  5. 2026ReWorld: Representation Learning for World Action Models
  6. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens
  7. 2026LIVE: Long-horizon Interactive Video World Modeling
  8. 2026DreamWAM: Beyond RGB Future Prediction for World Action Models