mlsanityData to 14 Oct 2026

MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Control

arXiv 2411.13807 ↗First posted 21 Nov 2024cs.CVcited by 92

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

The rapid advancement of diffusion models has greatly improved video synthesis, especially in controllable video generation, which is vital for applications like autonomous driving. Although DiT with 3D VAE has become a standard framework for video generation, it introduces challenges in controllable driving video generation, especially for geometry control, rendering existing control methods ineffective. To address these issues, we propose MagicDrive-V2, a novel approach that integrates the MVDiT block and spatial-temporal conditional encoding to enable multi-view video generation and precise geometric control. Additionally, we introduce an efficient method for obtaining contextual descriptions for videos to support diverse textual control, along with a progressive training strategy using mixed video data to enhance training efficiency and generalizability. Consequently, MagicDrive-V2 enables multi-view driving video synthesis with 3.3× resolution and 4× frame count (compared to current SOTA), rich contextual control, and geometric controls. Extensive experiments demonstrate MagicDrive-V2's ability, unlocking broader applications in autonomous driving.

Key facts. Cited by 92 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 40 of its 50 references that are in the map. First posted on arXiv 21 Nov 2024.

Builds on

40 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Control4 Oct

Other references · 39

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2022Flow Matching for Generative Modelingcited by 5,702
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  9. 2024GPT-4o System Cardcited by 4,588
  10. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  12. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
27 more
  1. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  4. 2022Video Diffusion Modelscited by 2,305
  5. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  6. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  7. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  8. 2022Planning-oriented Autonomous Drivingcited by 1,087
  9. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  10. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  11. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  12. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 347
  13. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 336
  14. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 319
  15. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  16. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 261
  17. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generationcited by 191
  18. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 168
  19. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  20. 2023BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layoutcited by 102
  21. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  22. 2024MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenescited by 71
  23. 2024Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generationcited by 55
  24. 2024DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perceptioncited by 32
  25. 2024DiVE: DiT-based Video Generation with Enhanced Controlcited by 31
  26. 2022Are We Ready for Vision-Centric Driving Streaming Perception? The ASAP Benchmarkcited by 26
  27. 2023Non-Cross Diffusion for Semantic Consistencycited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 42; 2026: 50.

92 papers.

Sort:

By area:

  1. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  2. 2025Controllable Video Generation: A Surveycited by 98
  3. 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Modelscited by 84
  4. 20253D and 4D World Modeling: A Surveycited by 67
  5. 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving Worldcited by 47
  6. 2026DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generationcited by 45
  7. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 38
  8. 2025OmniNWM: Omniscient Driving Navigation World Modelscited by 33
  9. 2026Rethinking Video Generation Model for the Embodied Worldcited by 33
  10. 2025The Role of World Models in Shaping Autonomous Driving: A Comprehensive Surveycited by 29
  11. 20253D Scene Generation: A Surveycited by 24
  12. 2025Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistencycited by 22
38 more
  1. 2025X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllabilitycited by 20
  2. 2025ORV: 4D Occupancy-centric Robot Video Generationcited by 17
  3. 2025Challenger: Affordable Adversarial Driving Video Generationcited by 16
  4. 2025WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perceptioncited by 16
  5. 2025DriveGen3D: Boosting Feed-Forward Driving Scene Generation with Efficient Video Diffusioncited by 15
  6. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  7. 2025Rethinking Driving World Model as Synthetic Data Generator for Perception Taskscited by 14
  8. 2025WorldSplat: Gaussian-Centric Feed-Forward 4D Scene Generation for Autonomous Drivingcited by 13
  9. 2026When World Models Dream Wrong: Physical-Conditioned Adversarial Attacks against World Modelscited by 13
  10. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representationcited by 13
  11. 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Drivingcited by 12
  12. 2025GenieDrive: Towards Physics-Aware Driving World Model with 4D Occupancy Guided Video Generationcited by 11
  13. 2026Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigationcited by 11
  14. 2025DiVE: Efficient Multi-View Driving Scenes Generation Based on Video Diffusion Transformercited by 10
  15. 2025LongDWM: Cross-Granularity Distillation for Building a Long-Term Driving World Modelcited by 9
  16. 2026UniDriveDreamer: A Single-Stage Multimodal World Model for Autonomous Drivingcited by 9
  17. 2026Attend Before Attention: Efficient and Scalable Video Understanding via Autoregressive Gazingcited by 9
  18. 2025MuDG: Taming Multi-modal Diffusion with Gaussian Splatting for Urban Scene Reconstructioncited by 8
  19. 20254D Driving Scene Generation With Stereo Forcingcited by 8
  20. 2025Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Methodcited by 8
  21. 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 8
  22. 2026World Engine: Towards the Era of Post-Training for Autonomous Drivingcited by 8
  23. 2025The Safety Challenge of World Models for Embodied AI Agents: A Reviewcited by 6
  24. 2026Motion-Aware Caching for Efficient Autoregressive Video Generationcited by 6
  25. 2025Dreamland: Controllable World Creation with Simulator and Generative Modelscited by 5
  26. 2025STAGE: A Stream-Centric Generative World Model for Long-Horizon Driving-Scene Simulationcited by 5
  27. 2025ECCV 2024 W-CODA: 1st Workshop on Multimodal Perception and Comprehension of Corner Cases in Autonomous Drivingcited by 5
  28. 2025LSD-3D: Large-Scale 3D Driving Scene Generation with Geometry Groundingcited by 5
  29. 2025RLGF: Reinforcement Learning with Geometric Feedback for Autonomous Driving Video Generationcited by 5
  30. 2025Spatial Retrieval Augmented Autonomous Drivingcited by 5
  31. 2025OmniGen: Unified Multimodal Sensor Generation for Autonomous Drivingcited by 5
  32. 2026Driving with DINO: Vision Foundation Features as a Unified Bridge for Sim-to-Real Generation in Autonomous Drivingcited by 5
  33. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
  34. 2026Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Drivingcited by 5
  35. 2026CausalDrive: Real-time Causal World Models for Autonomous Drivingcited by 5
  36. 2025ExtraGS: Geometric-Aware Trajectory Extrapolation with Uncertainty-Guided Generative Priorscited by 4
  37. 2026HorizonDrive: Self-Corrective Autoregressive World Model for Long-horizon Driving Simulationcited by 4
  38. 2026DriftWorld: Fast World Modeling through Driftingcited by 4

50 most cited of 92 All 92 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024DriveScape: Towards High-Resolution Controllable Multi-View Driving Video Generation
  2. 2024UniMLVG: Unified Framework for Multi-view Long Video Generation with Comprehensive Control Capabilities for Autonomous Driving
  3. 2024MyGo: Consistent and Controllable Multi-View Driving Video Generation with Camera Control
  4. 2024DreamDrive: Generative 4D Scene Modeling from Street View Images
  5. 2026M4World: A Multi-view Multimodal Driving World Model for Interactive Object Manipulation and Minute-long Streaming
  6. 2024GenDDS: Generating Diverse Driving Video Scenarios with Prompt-to-Video Generative Model
  7. 2023DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion model
  8. 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving