mlsanityData to 14 Oct 2026

Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Control

arXiv 2405.17414 ↗First posted 27 May 2024cs.CV cs.GRcited by 117

On the map: Diffusion models → Video generation

Abstract (arXiv)

Research on video generation has recently made tremendous progress, enabling high-quality videos to be generated from text prompts or images. Adding control to the video generation process is an important goal moving forward and recent approaches that condition video generation models on camera trajectories make strides towards it. Yet, it remains challenging to generate a video of the same scene from multiple different camera trajectories. Solutions to this multi-video generation problem could enable large-scale 3D scene generation with editable camera trajectories, among other applications. We introduce collaborative video diffusion (CVD) as an important step towards this vision. The CVD framework includes a novel cross-video synchronization module that promotes consistency between corresponding frames of the same video rendered from different camera poses using an epipolar attention mechanism. Trained on top of a state-of-the-art camera-control module for video generation, CVD generates multiple videos rendered from different camera trajectories with significantly better consistency than baselines, as shown in extensive experiments. Project page: https://collaborativevideodiffusion.github.io/.

Key facts. Cited by 117 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 57 of its 69 references that are in the map. First posted on arXiv 27 May 2024.

Builds on

57 of 69 references.

Closest ancestors · 18

Same direction, newest first

  1. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generation2 Apr
  2. 2023Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models3 Dec
  3. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation28 Nov
  4. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models28 Nov
  5. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  6. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Models12 Oct
  7. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation1 Sep
  8. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  9. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models17 May
  10. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  11. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators23 Mar
  12. 2023Pix2Video: Video Editing using Image Diffusion22 Mar
6 more
  1. 2023Video Probabilistic Diffusion Models in Projected Latent Space15 Feb
  2. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
  3. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec
  4. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation23 Nov
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
  6. 2022Video Diffusion Models7 Apr

Other references · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  12. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
27 more
  1. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  2. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  3. 2018Demystifying MMD GANscited by 1,293
  4. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  5. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
  6. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
  7. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  8. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  9. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 521
  10. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  11. 2023Universal Guidance for Diffusion Modelscited by 437
  12. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 378
  13. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
  14. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 274
  15. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 264
  16. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  17. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 243
  18. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 214
  19. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  20. 2023State of the Art on Diffusion Models for Visual Computingcited by 127
  21. 2023Consistent View Synthesis with Pose-Guided Diffusion Modelscited by 117
  22. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
  23. 2023DiffCollage: Parallel Generation of Large Content with Diffusion Modelscited by 103
  24. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
  25. 2022DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Modelscited by 51
  26. 2023MAS: Multi-view Ancestral Sampling for 3D motion generation using 2D diffusioncited by 37
  27. 2023MVHuman: Tailoring 2D Diffusion with Multi-view Sampling For Realistic 3D Human Generationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 23; 2025: 57; 2026: 37.

117 papers.

Sort:

By area:

  1. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  2. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  3. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  4. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  5. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 154
  6. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 140
  7. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 117
  8. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 99
  9. 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 98
  10. 2025Controllable Video Generation: A Surveycited by 98
  11. 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 93
  12. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 77
38 more
  1. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 70
  2. 2024SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generationcited by 68
  3. 2025Follow-Your-Creation: Empowering 4D Creation through Video Inpaintingcited by 62
  4. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 57
  5. 2024InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Modelscited by 56
  6. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  7. 2024Diffusion Self-Distillation for Zero-Shot Customized Image Generationcited by 47
  8. 2025CamCloneMaster: Enabling Reference-based Camera Control for Video Generationcited by 46
  9. 2025ATI: Any Trajectory Instruction for Controllable Video Generationcited by 43
  10. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  11. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  12. 2025FloVD: Optical Flow Meets Video Diffusion Model for Enhanced Camera-Controlled Video Synthesiscited by 37
  13. 2025Reangle-A-Video: 4D Video Generation as Video-to-Video Translationcited by 35
  14. 2025Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistencycited by 35
  15. 2025Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Controlcited by 34
  16. 20254DNeX: Feed-Forward 4D Generative Modeling Made Easycited by 34
  17. 2025OmniNWM: Omniscient Driving Navigation World Modelscited by 33
  18. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Controlcited by 28
  19. 20244Real-Video: Learning Generalizable Photo-Realistic 4D Video Diffusioncited by 27
  20. 2025Dynamic Camera Poses and Where to Find Themcited by 27
  21. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 27
  22. 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Modelscited by 27
  23. 2024Generative Photography: Scene-Consistent Camera Control for Realistic Text-to-Image Synthesiscited by 26
  24. 2025VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generationcited by 25
  25. 2024Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attentioncited by 24
  26. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 24
  27. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modelingcited by 21
  28. 2025X-Dyna: Expressive Dynamic Human Image Animationcited by 18
  29. 2026Vista4D: Video Reshooting with 4D Point Cloudscited by 18
  30. 2025MVP4D: Multi-View Portrait Video Diffusion for Animatable 4D Avatarscited by 17
  31. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  32. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applicationscited by 14
  33. 20254Real-Video-V2: Fused View-Time Attention and Feedforward Reconstruction for 4D Scene Generationcited by 14
  34. 2025BulletTime: Decoupled Control of Time and Camera Pose for Video Generationcited by 14
  35. 2024NVComposer: Boosting Generative Novel View Synthesis with Multiple Sparse and Unposed Imagescited by 13
  36. 2024Latent-Reframe: Enabling Camera Control for Video Diffusion Model without Trainingcited by 13
  37. 2025DanceTogether! Identity-Preserving Multi-Person Interactive Video Generationcited by 13
  38. 2025Dynamic View Synthesis as an Inverse Problemcited by 12

50 most cited of 117 All 117 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Vid3D: Synthesis of Dynamic 3D Scenes using 2D Video Diffusion
  2. 2025SV4D 2.0: Enhancing Spatio-Temporal Consistency in Multi-View Video Diffusion for High-Quality 4D Generation
  3. 2026DCDM: Divide-and-Conquer Diffusion Models for Consistency-Preserving Video Generation
  4. 2023GD-VDM: Generated Depth for better Diffusion-based Video Generation
  5. 2026Image-to-Video Diffusion: From Foundations to Open Frontiers
  6. 2024Ctrl-V: Higher Fidelity Video Generation with Bounding-Box Controlled Object Motion
  7. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Control
  8. 2025Zero4D: Training-Free 4D Video Generation From Single Video Using Off-the-Shelf Video Diffusion