# DF3: World Modeling via Decoder-Free Feature Forecasting in Autonomous Navigation

arXiv: https://arxiv.org/abs/2608.02428 · HTML: https://mlsanity.com/paper/2608.02428/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in World models for navigation, one of 11 directions in World models.
- Builds on 36 of its 41 references that are in the map.
- First posted on arXiv 3 Aug 2026.

## Placed in

- World models / [World models for navigation](https://mlsanity.com/wm/world-models-for-navigation/)

## Builds on (most cited first, 30 of 36)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning](https://mlsanity.com/paper/2411.04983/) (2024)
- [Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability](https://mlsanity.com/paper/2405.17398/) (2024)
- [Flexible Diffusion Modeling of Long Videos](https://mlsanity.com/paper/2205.11495/) (2022)
- [Navigation World Models](https://mlsanity.com/paper/2412.03572/) (2024)
- [GenAD: Generative End-to-End Autonomous Driving](https://mlsanity.com/paper/2402.11502/) (2024)
- [Foundation Models in Robotics: Applications, Challenges, and the Future](https://mlsanity.com/paper/2312.07843/) (2023)
- [GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2503.20523/) (2025)
- [Enhancing End-to-End Autonomous Driving with Latent World Model](https://mlsanity.com/paper/2406.08481/) (2024)
- [Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond](https://mlsanity.com/paper/2405.03520/) (2024)
- [World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model](https://mlsanity.com/paper/2507.00603/) (2025)
- [Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Models](https://mlsanity.com/paper/2506.09042/) (2025)
- [RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Driving](https://mlsanity.com/paper/2301.10222/) (2023)
- [Your ViT is Secretly an Image Segmentation Model](https://mlsanity.com/paper/2503.19108/) (2025)
- [Back to the Features: DINO as a Foundation for Video World Models](https://mlsanity.com/paper/2507.19468/) (2025)
- [ViPlanner: Visual Semantic Imperative Learning for Local Navigation](https://mlsanity.com/paper/2310.00982/) (2023)
- [DINO-Foresight: Looking into the Future with DINO](https://mlsanity.com/paper/2412.11673/) (2024)
- [From Forecasting to Planning: Policy World Model for Collaborative State-Action Prediction](https://mlsanity.com/paper/2510.19654/) (2025)

## Abstract (arXiv)

Forecasting future states from video sequences is a critical challenge for autonomous robotic systems and a fundamental objective of world modeling. Prior generative methods operating at the pixel level inevitably overemphasize task-irrelevant details, leading to prohibitive computational overhead. While latent-based approaches attempt to mitigate this by predicting features directly, the persistent reliance on heavy decoders for state-to-task mapping remains a computational bottleneck. In this work, we propose Decoder-Free Feature Forecasting (DF3), a novel framework that models world evolution entirely within the latent space and directly derives task outputs, completely eliminating the need for a decoder. Specifically, DF3 injects learnable spatial queries into the terminal blocks of a frozen vision foundation model to extract future state representations directly. By employing a lightweight, unified Motion-Aware Context Fusion (MACF) mechanism that seamlessly integrates coarse flow warping with fine-grained latent cross-correlation, these queries interact with historical token representations to explicitly align and forecast the feature of the next frame. Subsequently, a specialized set of task queries probes these forecasted features for the downstream task. Extensive experiments on public benchmarks and zero-shot deployment in a robotic simulator demonstrate that DF3 achieves performance comparable to state-of-the-art methods while offering superior efficiency and flexibility for integrated perception and control.