mlsanityData to 7 Oct 2026

DiVE: DiT-based Video Generation with Enhanced Control

arXiv 2409.01595 ↗First posted 3 Sep 2024cs.CVcited by 31

Not on the map. Why?

Abstract (arXiv)

Generating high-fidelity, temporally consistent videos in autonomous driving scenarios faces a significant challenge, e.g. problematic maneuvers in corner cases. Despite recent video generation works are proposed to tackcle the mentioned problem, i.e. models built on top of Diffusion Transformers (DiT), works are still missing which are targeted on exploring the potential for multi-view videos generation scenarios. Noticeably, we propose the first DiT-based framework specifically designed for generating temporally and multi-view consistent videos which precisely match the given bird's-eye view layouts control. Specifically, the proposed framework leverages a parameter-free spatial view-inflated attention mechanism to guarantee the cross-view consistency, where joint cross-attention modules and ControlNet-Transformer are integrated to further improve the precision of control. To demonstrate our advantages, we extensively investigate the qualitative comparisons on nuScenes dataset, particularly in some most challenging corner cases. In summary, the effectiveness of our proposed method in producing long, controllable, and highly consistent videos under difficult conditions is proven to be effective.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 15 of its 15 references that are in the map. First posted on arXiv 3 Sep 2024.

Builds on

15 of 15 references.

References · 15

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  7. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  8. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  10. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  11. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  12. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
3 more
  1. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 250
  2. 2024PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Modelscited by 139
  3. 2024Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generationcited by 54

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 20; 2026: 8.

31 papers.

Sort:

By area:

  1. 2025Controllable Video Generation: A Surveycited by 93
  2. 2024MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Controlcited by 90
  3. 20253D and 4D World Modeling: A Surveycited by 64
  4. 2024DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenescited by 34
  5. 2024UniMLVG: Unified Framework for Multi-view Long Video Generation with Comprehensive Control Capabilities for Autonomous Drivingcited by 27
  6. 2025MaskGWM: A Generalizable Driving World Model with Video Mask Reconstructioncited by 27
  7. 2025WorldLens: Full-Spectrum Evaluations of Driving World Models in Real Worldcited by 22
  8. 2025UniFuture: A 4D Driving World Model for Future Generation and Perceptioncited by 21
  9. 2025Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistencycited by 21
  10. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  11. 2025Rethinking Driving World Model as Synthetic Data Generator for Perception Taskscited by 14
  12. 2025CoGen: 3D Consistent Video Generation via Adaptive Conditioning for Autonomous Drivingcited by 12
19 more
  1. 2025WorldSplat: Gaussian-Centric Feed-Forward 4D Scene Generation for Autonomous Drivingcited by 11
  2. 2025DiVE: Efficient Multi-View Driving Scenes Generation Based on Video Diffusion Transformercited by 10
  3. 2025LongDWM: Cross-Granularity Distillation for Building a Long-Term Driving World Modelcited by 9
  4. 2025OccTENS: 3D Occupancy World Model via Temporal Next-Scale Predictioncited by 9
  5. 2025Any2Caption:Interpreting Any Condition to Caption for Controllable Video Generationcited by 8
  6. 2025PosePilot: Steering Camera Pose for Generative World Models with Self-supervised Depthcited by 4
  7. 2025OneVAE: Joint Discrete and Continuous Optimization Helps Discrete Video VAE Train Bettercited by 3
  8. 2026FAR-Drive: Frame-AutoRegressive Video Generation in Closed-Loop Autonomous Drivingcited by 3
  9. 2025CorrectAD: A Self-Correcting Agentic System to Improve End-to-end Planning in Autonomous Drivingcited by 2
  10. 2026OmniDrive: An LLM-Choreographed Multi-Agent World Model with Unified Latent Co-Compression for Multi-View Driving Video Generationcited by 2
  11. 2026ParaScale: Scale-Calibrated Camera-Motion Transfer via a Gauge-Invariant Parallax Numbercited by 2
  12. 2025CVD-STORM: Cross-View Video Diffusion with Spatial-Temporal Reconstruction Model for Autonomous Drivingcited by 1
  13. 2025Autoregressive Video Autoencoder with Decoupled Temporal and Spatial Contextcited by 1
  14. 2026OrthoMotion:Disentangling Camera and Subject Motion via Geometry Semantics Orthogonal Attentioncited by 1
  15. 2026MSEditor: Toward Consistent Multi-Shot Video Editingcited by 1
  16. 2025MM-MovieDubber: Towards Multi-Modal Learning for Multi-Modal Movie Dubbingcited by 0
  17. 2026LSA: Localized Semantic Alignment for Enhancing Temporal Consistency in Traffic Video Generationcited by 0
  18. 2026HelloWorld: Towards Practical Applications of Generative Driving World Modelscited by 0
  19. 2026CoDrive: Cross-Vehicle World-Consistent Video Generation with Precise Trajectory Control for Cooperative Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion model
  2. 2024DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation
  3. 2025DiTraj: training-free trajectory control for video diffusion transformer
  4. 2026ReaDiT Guidance: Control for Image and Video Generation using Diffusion Transformer Features
  5. 2024DriveScape: Towards High-Resolution Controllable Multi-View Driving Video Generation
  6. 2024Seeing Beyond Views: Multi-View Driving Scene Video Generation with Holistic Attention
  7. 2025MoVieDrive: Urban Scene Synthesis with Multi-Modal Multi-View Video Diffusion Transformer
  8. 2025DriveGen3D: Boosting Feed-Forward Driving Scene Generation with Efficient Video Diffusion