DF3: World Modeling via Decoder-Free Feature Forecasting in Autonomous Navigation
On the map: World models → World models for navigation
Abstract (arXiv)
Forecasting future states from video sequences is a critical challenge for autonomous robotic systems and a fundamental objective of world modeling. Prior generative methods operating at the pixel level inevitably overemphasize task-irrelevant details, leading to prohibitive computational overhead. While latent-based approaches attempt to mitigate this by predicting features directly, the persistent reliance on heavy decoders for state-to-task mapping remains a computational bottleneck. In this work, we propose Decoder-Free Feature Forecasting (DF3), a novel framework that models world evolution entirely within the latent space and directly derives task outputs, completely eliminating the need for a decoder. Specifically, DF3 injects learnable spatial queries into the terminal blocks of a frozen vision foundation model to extract future state representations directly. By employing a lightweight, unified Motion-Aware Context Fusion (MACF) mechanism that seamlessly integrates coarse flow warping with fine-grained latent cross-correlation, these queries interact with historical token representations to explicitly align and forecast the feature of the next frame. Subsequently, a specialized set of task queries probes these forecasted features for the downstream task. Extensive experiments on public benchmarks and zero-shot deployment in a robotic simulator demonstrate that DF3 achieves performance comparable to state-of-the-art methods while offering superior efficiency and flexibility for integrated perception and control.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 36 of its 41 references that are in the map. First posted on arXiv 3 Aug 2026.
Builds on
36 of 41 references.
Closest ancestors · 1
Same direction, newest first
- 2024Navigation World Models4 Dec
Other references · 35
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2020End-to-End Object Detection with Transformerscited by 7,384
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2025DINOv3cited by 1,297
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
23 more
- 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
- 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
- 2022Flexible Diffusion Modeling of Long Videoscited by 294
- 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
- 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 212
- 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Drivingcited by 178
- 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 161
- 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
- 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Modelcited by 91
- 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Modelscited by 80
- 2023RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Drivingcited by 76
- 2025Your ViT is Secretly an Image Segmentation Modelcited by 61
- 2025Back to the Features: DINO as a Foundation for Video World Modelscited by 56
- 2024DINO-Foresight: Looking into the Future with DINOcited by 44
- 2023ViPlanner: Visual Semantic Imperative Learning for Local Navigationcited by 40
- 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 29
- 2026Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Drivingcited by 26
- 2026UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Drivingcited by 20
- 2024Exploring the Interplay Between Video Generation and World Models in Autonomous Driving: A Surveycited by 14
- 2025SparseWorld: A Flexible, Adaptive, and Efficient 4D Occupancy World Model Powered by Sparse and Dynamic Queriescited by 10
- 2024Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Drivingcited by 8
- 2026VidEoMT: Your ViT is Secretly Also a Video Segmentation Modelcited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Efficient Image-Goal Navigation with Representative Latent World Model
- 2026NavWM: A Unified Navigation World Model for Foresight-Driven Planning
- 2026FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation
- 2026EditWM: Event-Decomposed World Modeling with Incremental Correction for End-to-End Autonomous Driving
- 2026Hydra: A Navigation World Action Model with Discrete Latent Planning and Continuous Flow-Matching Execution
- 2026IDOL: Inverse-Dynamics-Guided Future Prediction for End-to-End Autonomous Driving
- 2026PACT-WAM: Predicting Actions and Visual Foresight with Compact Temporal Encoding for Robot Manipulation
- 2026WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving