ForgeDrive: Bidirectional Cross-Conditioning for Unified Visual-Action Generation in Autonomous Driving
On the map: World models → World models for autonomous driving
also in 1 more
Abstract (arXiv)
World-model-based autonomous driving endows the model with the ability to understand scene evolution. Yet this promise is undermined by the prevailing imagine-then-act paradigm, which allows errors from the more challenging visual generation stage to cascade into action planning. We introduce ForgeDrive, a unified autoregressive diffusion framework with visual-action cross-conditioning that closes this gap through act-then-imagine paradigm. ForgeDrive factorizes the future as a sequence of per-timestep frame-action pairs, intertwining each action with its corresponding visual observation. During training, we decouple the diffusion timesteps of the two modalities and introduce a UniDiffuser-style noise scheduler to get the ability to infer either modality from its counterpart and deepen understanding of relationships between images and actions. At inference, we propose a novel act-then-imagine inference paradigm, and find that at each step, action generation is a capability internalized during training, requiring no clean future frame as a prerequisite at inference time; instead, the generated action can improve the accuracy of future frame generation, which in turn enhances the quality of the next action. Additionally, we augment each step with future ego-status prediction, further sharpening planning ability. Extensive experiments on NAVSIM demonstrate that ForgeDrive not only unifies driving simulation, planning, and visual odometry into a single model, but also outperforms existing strong planners without any post-training strategy.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 21 of its 52 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
21 of 52 references.
Closest ancestors · 8
Same direction, newest first
- 2026DriveFuture: Future-Aware Latent World Models for Autonomous Driving10 May
- 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving28 Mar
- 2026DreamerAD: Efficient Reinforcement Learning via Latent World Model for Autonomous Driving25 Mar
- 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving25 Mar
- 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
- 2023ADriver-I: A General World Model for Autonomous Driving22 Nov
- 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun
Other references · 13
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2017Decoupled Weight Decay Regularizationcited by 1,722
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2026World Action Models are Zero-shot Policiescited by 411
- 2026Causal World Modeling for Robot Controlcited by 289
- 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 244
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
- 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 151
- 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 125
- 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 93
- 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Drivingcited by 52
1 more
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving
- 2026DriveVA: Video Action Models are Zero-Shot Drivers
- 2026UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation
- 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers
- 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving World
- 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
- 2026FAR-Drive: Frame-AutoRegressive Video Generation in Closed-Loop Autonomous Driving
- 2026Learning Vision-Language-Action World Models for Autonomous Driving