# Back to the Features: DINO as a Foundation for Video World Models

arXiv: https://arxiv.org/abs/2507.19468 · HTML: https://mlsanity.com/paper/2507.19468/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 54 of its 70 references that are in the map.
- First posted on arXiv 25 Jul 2025.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)

## Builds on (most cited first, 30 of 54)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024)
- [Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs](https://mlsanity.com/paper/2401.06209/) (2024)
- [VideoGPT: Video Generation using VQ-VAE and Transformers](https://mlsanity.com/paper/2104.10157/) (2021)
- [Movie Gen: A Cast of Media Foundation Models](https://mlsanity.com/paper/2410.13720/) (2024)
- [GAIA-1: A Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2309.17080/) (2023)
- [Revisiting Feature Prediction for Learning Visual Representations from Video](https://mlsanity.com/paper/2404.08471/) (2024)
- [TD-MPC2: Scalable, Robust World Models for Continuous Control](https://mlsanity.com/paper/2310.16828/) (2023)
- [VideoPoet: A Large Language Model for Zero-Shot Video Generation](https://mlsanity.com/paper/2312.14125/) (2023)
- [Phenaki: Variable Length Video Generation From Open Domain Textual Description](https://mlsanity.com/paper/2210.02399/) (2022)
- [Learning Interactive Real-World Simulators](https://mlsanity.com/paper/2310.06114/) (2023)
- [DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning](https://mlsanity.com/paper/2411.04983/) (2024)
- [MAGVIT: Masked Generative Video Transformer](https://mlsanity.com/paper/2212.05199/) (2022)

## Built on by (most cited first, 30 of 58)

- [V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning](https://mlsanity.com/paper/2603.14482/) (2026)
- [A Comprehensive Survey on World Models for Embodied AI](https://mlsanity.com/paper/2510.16732/) (2025)
- [VL-JEPA: Joint Embedding Predictive Architecture for Vision-language](https://mlsanity.com/paper/2512.10942/) (2025)
- [Learning Latent Action World Models In The Wild](https://mlsanity.com/paper/2601.05230/) (2026)
- [What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?](https://mlsanity.com/paper/2512.24497/) (2025)
- [Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future](https://mlsanity.com/paper/2512.16760/) (2025)
- [Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models](https://mlsanity.com/paper/2605.06388/) (2026)
- [Block-Recurrent Dynamics in Vision Transformers](https://mlsanity.com/paper/2512.19941/) (2025)
- [BridgeV2W: Bridging Video Generation Models to Embodied World Models via Embodiment Masks](https://mlsanity.com/paper/2602.03793/) (2026)
- [NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Prediction](https://mlsanity.com/paper/2512.01550/) (2025)
- [Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresight](https://mlsanity.com/paper/2510.08713/) (2025)
- [Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directions](https://mlsanity.com/paper/2601.07823/) (2026)
- [VGGT-World: Transforming VGGT into an Autoregressive Geometry World Model](https://mlsanity.com/paper/2603.12655/) (2026)
- [Rethinking JEPA: Compute-Efficient Video SSL with Frozen Teachers](https://mlsanity.com/paper/2509.24317/) (2025)
- [VideoWorld 2: Learning Transferable Knowledge from Real-world Videos](https://mlsanity.com/paper/2602.10102/) (2026)
- [Learning Interactive World Model for Object-Centric Reinforcement Learning](https://mlsanity.com/paper/2511.02225/) (2025)
- [Language-Conditioned World Modeling for Visual Navigation](https://mlsanity.com/paper/2603.26741/) (2026)
- [A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokens](https://mlsanity.com/paper/2604.04913/) (2026)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [LongScape: Advancing Long-Horizon Embodied World Models with Context-Aware MoE](https://mlsanity.com/paper/2509.21790/) (2025)
- [GoViG: Goal-Conditioned Visual Navigation Instruction Generation via Multimodal Reasoning](https://mlsanity.com/paper/2508.09547/) (2025)
- [MoWM: Mixture-of-World-Models for Embodied Planning via Latent-to-Pixel Feature Modulation](https://mlsanity.com/paper/2509.21797/) (2025)
- [Foundation Visual Encoders Are Secretly Few-Shot Anomaly Detectors](https://mlsanity.com/paper/2510.01934/) (2025)
- [SoundReactor: Frame-level Online Video-to-Audio Generation](https://mlsanity.com/paper/2510.02110/) (2025)
- [SeeU: Seeing the Unseen World via 4D Dynamics-aware Generation](https://mlsanity.com/paper/2512.03350/) (2025)
- [VFMF: World Modeling by Forecasting Vision Foundation Model Features](https://mlsanity.com/paper/2512.11225/) (2025)
- [Building Social World Models with Large Language Models](https://mlsanity.com/paper/2606.11482/) (2026)
- [Latent World Models with Monotone Planning Costs for Image-Goal Navigation](https://mlsanity.com/paper/2608.09073/) (2026)
- [MaDiS: Taming Masked Diffusion Language Models for Sign Language Generation](https://mlsanity.com/paper/2601.19577/) (2026)
- [Learning Visual Feature-Based World Models via Residual Latent Action](https://mlsanity.com/paper/2605.07079/) (2026)

## Abstract (arXiv)

We present DINO-world, a powerful generalist video world model trained to predict future frames in the latent space of DINOv2. By leveraging a pre-trained image encoder and training a future predictor on a large-scale uncurated video dataset, DINO-world learns the temporal dynamics of diverse scenes, from driving and indoor scenes to simulated environments. We show that DINO-world outperforms previous models on a variety of video prediction benchmarks, e.g. segmentation and depth forecasting, and demonstrates strong understanding of intuitive physics. Furthermore, we show that it is possible to fine-tune the predictor on observation-action trajectories. The resulting action-conditioned world model can be used for planning by simulating candidate trajectories in latent space.