mlsanityData to 7 Oct 2026

DF3: World Modeling via Decoder-Free Feature Forecasting in Autonomous Navigation

arXiv 2608.02428 ↗First posted 3 Aug 2026cs.CVcited by 0

On the map: World models → World models for navigation

Abstract (arXiv)

Forecasting future states from video sequences is a critical challenge for autonomous robotic systems and a fundamental objective of world modeling. Prior generative methods operating at the pixel level inevitably overemphasize task-irrelevant details, leading to prohibitive computational overhead. While latent-based approaches attempt to mitigate this by predicting features directly, the persistent reliance on heavy decoders for state-to-task mapping remains a computational bottleneck. In this work, we propose Decoder-Free Feature Forecasting (DF3), a novel framework that models world evolution entirely within the latent space and directly derives task outputs, completely eliminating the need for a decoder. Specifically, DF3 injects learnable spatial queries into the terminal blocks of a frozen vision foundation model to extract future state representations directly. By employing a lightweight, unified Motion-Aware Context Fusion (MACF) mechanism that seamlessly integrates coarse flow warping with fine-grained latent cross-correlation, these queries interact with historical token representations to explicitly align and forecast the feature of the next frame. Subsequently, a specialized set of task queries probes these forecasted features for the downstream task. Extensive experiments on public benchmarks and zero-shot deployment in a robotic simulator demonstrate that DF3 achieves performance comparable to state-of-the-art methods while offering superior efficiency and flexibility for integrated perception and control.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 36 of its 41 references that are in the map. First posted on arXiv 3 Aug 2026.

Builds on

36 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Navigation World Models4 Dec

Other references · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  6. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  7. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  8. 2025DINOv3cited by 1,297
  9. 2023Mastering Diverse Domains through World Modelscited by 1,193
  10. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  11. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  12. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
23 more
  1. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  2. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
  3. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  4. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  5. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  6. 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 212
  7. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 178
  8. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 161
  9. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  10. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Modelcited by 91
  11. 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Modelscited by 80
  12. 2023RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Drivingcited by 76
  13. 2025Your ViT is Secretly an Image Segmentation Modelcited by 61
  14. 2025Back to the Features: DINO as a Foundation for Video World Modelscited by 56
  15. 2024DINO-Foresight: Looking into the Future with DINOcited by 44
  16. 2023ViPlanner: Visual Semantic Imperative Learning for Local Navigationcited by 40
  17. 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 29
  18. 2026Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Drivingcited by 26
  19. 2026UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Drivingcited by 20
  20. 2024Exploring the Interplay Between Video Generation and World Models in Autonomous Driving: A Surveycited by 14
  21. 2025SparseWorld: A Flexible, Adaptive, and Efficient 4D Occupancy World Model Powered by Sparse and Dynamic Queriescited by 10
  22. 2024Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Drivingcited by 8
  23. 2026VidEoMT: Your ViT is Secretly Also a Video Segmentation Modelcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Efficient Image-Goal Navigation with Representative Latent World Model
  2. 2026NavWM: A Unified Navigation World Model for Foresight-Driven Planning
  3. 2026FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation
  4. 2026EditWM: Event-Decomposed World Modeling with Incremental Correction for End-to-End Autonomous Driving
  5. 2026Hydra: A Navigation World Action Model with Discrete Latent Planning and Continuous Flow-Matching Execution
  6. 2026IDOL: Inverse-Dynamics-Guided Future Prediction for End-to-End Autonomous Driving
  7. 2026PACT-WAM: Predicting Actions and Visual Foresight with Compact Temporal Encoding for Robot Manipulation
  8. 2026WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving