mlsanityData to 7 Oct 2026

InstaDrive: Instance-Aware Driving World Models for Realistic and Consistent Video Generation

arXiv 2602.03242 ↗First posted 3 Feb 2026cs.CVcited by 8

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

Autonomous driving relies on robust models trained on high-quality, large-scale multi-view driving videos. While world models offer a cost-effective solution for generating realistic driving videos, they struggle to maintain instance-level temporal consistency and spatial geometric fidelity. To address these challenges, we propose InstaDrive, a novel framework that enhances driving video realism through two key advancements: (1) Instance Flow Guider, which extracts and propagates instance features across frames to enforce temporal consistency, preserving instance identity over time. (2) Spatial Geometric Aligner, which improves spatial reasoning, ensures precise instance positioning, and explicitly models occlusion hierarchies. By incorporating these instance-aware mechanisms, InstaDrive achieves state-of-the-art video generation quality and enhances downstream autonomous driving tasks on the nuScenes dataset. Additionally, we utilize CARLA's autopilot to procedurally and stochastically simulate rare but safety-critical driving scenarios across diverse maps and regions, enabling rigorous safety evaluation for autonomous systems. Our project page is https://shanpoyang654.github.io/InstaDrive/page.html.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 47 of its 59 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

47 of 59 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024InfinityDrive: Breaking Time Limits in Driving World Models2 Dec
  2. 2024DriveScape: Towards High-Resolution Controllable Multi-View Driving Video Generation9 Sep
  3. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
  4. 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
  5. 2023ADriver-I: A General World Model for Autonomous Driving22 Nov
  6. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving18 Sep
  7. 2017CARLA: An Open Urban Driving Simulator10 Nov

Other references · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2022Scalable Diffusion Models with Transformerscited by 5,931
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  7. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  8. 2022Video Diffusion Modelscited by 2,280
  9. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  10. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  11. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  12. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
28 more
  1. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  3. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  4. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  5. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  6. 2023ModelScope Text-to-Video Technical Reportcited by 651
  7. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  8. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  9. 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
  10. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  11. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  12. 2022MAGVIT: Masked Generative Video Transformercited by 341
  13. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  14. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  15. 2016Learning from Simulated and Unsupervised Images through Adversarial Trainingcited by 287
  16. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 250
  17. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  18. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 204
  19. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 165
  20. 2023Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generationcited by 137
  21. 2023Consistent View Synthesis with Pose-Guided Diffusion Modelscited by 117
  22. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  23. 2023BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layoutcited by 102
  24. 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
  25. 2023PyramidFlow: High-Resolution Defect Contrastive Localization using Pyramid Normalizing Flowcited by 74
  26. 2024Ca2-VDM: Efficient Autoregressive Video Diffusion Model with Causal Generation and Cache Sharingcited by 70
  27. 2024Allegro: Open the Black Box of Commercial-Level Video Generation Modelcited by 68
  28. 2024Qihoo-T2X: An Efficient Proxy-Tokenized Diffusion Transformer for Text-to-Any-Taskcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026ConsisDrive: Identity-Preserving Driving World Models for Video Generation by Instance Mask
  2. 2024Physical Informed Driving World Model
  3. 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving
  4. 2024Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generation
  5. 2026VehDyn: A Driving World Model Benchmark for Vehicle Dynamics
  6. 2026FAR-Drive: Frame-AutoRegressive Video Generation in Closed-Loop Autonomous Driving
  7. 2026CoDrive: Cross-Vehicle World-Consistent Video Generation with Precise Trajectory Control for Cooperative Driving
  8. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation