mlsanityData to 7 Oct 2026

UniDriveDreamer: A Single-Stage Multimodal World Model for Autonomous Driving

arXiv 2602.02002 ↗First posted 2 Feb 2026cs.CVcited by 9

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

World models have demonstrated significant promise for data synthesis in autonomous driving. However, existing methods predominantly concentrate on single-modality generation, typically focusing on either multi-camera video or LiDAR sequence synthesis. In this paper, we propose UniDriveDreamer, a single-stage unified multimodal world model for autonomous driving, which directly generates multimodal future observations without relying on intermediate representations or cascaded modules. Our framework introduces a LiDAR-specific variational autoencoder (VAE) designed to encode input LiDAR sequences, alongside a video VAE for multi-camera images. To ensure cross-modal compatibility and training stability, we propose Unified Latent Anchoring (ULA), which explicitly aligns the latent distributions of the two modalities. The aligned features are fused and processed by a diffusion transformer that jointly models their geometric correspondence and temporal evolution. Additionally, structured scene layout information is projected per modality as a conditioning signal to guide the synthesis. Extensive experiments demonstrate that UniDriveDreamer outperforms previous state-of-the-art methods in both video and LiDAR generation, while also yielding measurable improvements in downstream

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 45 of its 53 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

45 of 53 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving26 Mar
  2. 2025ReconDreamer++: Harmonizing Generative and Reconstructive Models for Driving Scene Representation24 Mar
  3. 2024ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restoration29 Nov
  4. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation17 Oct
  5. 2024Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generation3 Jun
  6. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
  7. 2024GenAD: Generalized Predictive Model for Autonomous Driving14 Mar
  8. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
  9. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
  10. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep

Other references · 35

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  4. 2022Flow Matching for Generative Modelingcited by 5,426
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  7. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  9. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  10. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  11. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  12. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
23 more
  1. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 250
  2. 2025World Simulation with Video Foundation Models for Physical AIcited by 183
  3. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 165
  4. 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 147
  5. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  6. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  7. 2023BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layoutcited by 102
  8. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 102
  9. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 95
  10. 2022Learning to Generate Realistic LiDAR Point Cloudscited by 91
  11. 2024MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Controlcited by 90
  12. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokenscited by 87
  13. 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Modelscited by 80
  14. 2018Deep Generative Modeling of LiDAR Datacited by 70
  15. 2025GigaWorld-0: World Models as Data Engine to Empower Embodied AIcited by 60
  16. 2024LidarDM: Generative LiDAR Simulation in a Generated Worldcited by 53
  17. 2024RangeLDM: Fast Realistic LiDAR Point Cloud Generationcited by 45
  18. 2025ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstructioncited by 27
  19. 2022Are We Ready for Vision-Centric Driving Streaming Perception? The ASAP Benchmarkcited by 24
  20. 2025UniFuture: A 4D Driving World Model for Future Generation and Perceptioncited by 21
  21. 2025Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistencycited by 21
  22. 2025DriveGen3D: Boosting Feed-Forward Driving Scene Generation with Efficient Video Diffusioncited by 14
  23. 2025OmniGen: Unified Multimodal Sensor Generation for Autonomous Drivingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latents
  2. 2024X-Drive: Cross-modality consistent multi-sensor data synthesis for driving scenarios
  3. 2026UniDWM: Towards a Unified Driving World Model via Multifaceted Representation Learning
  4. 2026LMGenDrive: Bridging Multimodal Understanding and Generative World Modeling for End-to-End Driving
  5. 2025MoVieDrive: Urban Scene Synthesis with Multi-Modal Multi-View Video Diffusion Transformer
  6. 2026UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving
  7. 2025MultiEditor: Controllable Multimodal Object Editing for Driving Scenarios Using 3D Gaussian Splatting Priors
  8. 2024HoloDrive: Holistic 2D-3D Multi-Modal Street Scene Generation for Autonomous Driving