mlsanityData to 7 Oct 2026

Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesis

arXiv 2405.14868 ↗First posted 23 May 2024cs.CV cs.AI cs.LGcited by 116

Not on the map. Why?

Cited by 35 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Accurate reconstruction of complex dynamic scenes from just a single viewpoint continues to be a challenging task in computer vision. Current dynamic novel view synthesis methods typically require videos from many different camera viewpoints, necessitating careful recording setups, and significantly restricting their utility in the wild as well as in terms of embodied AI applications. In this paper, we propose GCD, a controllable monocular dynamic view synthesis pipeline that leverages large-scale diffusion priors to, given a video of any scene, generate a synchronous video from any other chosen perspective, conditioned on a set of relative camera pose parameters. Our model does not require depth as input, and does not explicitly model 3D scene geometry, instead performing end-to-end video-to-video translation in order to achieve its goal efficiently. Despite being trained on synthetic multi-view video data only, zero-shot real-world generalization experiments show promising results in multiple domains, including robotics, object permanence, and driving environments. We believe our framework can potentially unlock powerful applications in rich dynamic scene understanding, perception for robotics, and interactive 3D video viewing experiences for virtual reality.

Key facts. Cited by 116 papers in the map (data through 7 Oct 2026). Builds on 79 of its 90 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

79 of 90 references.

References · 79

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  6. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  8. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2022Video Diffusion Modelscited by 2,280
  11. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  12. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
38 more
  1. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  3. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  4. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  5. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  7. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  8. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  9. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  10. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  11. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  12. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  13. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  14. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  15. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  16. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  17. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  18. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  19. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  20. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  21. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  22. 2021Neural Fields in Visual Computing and Beyondcited by 539
  23. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  24. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  25. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 470
  26. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  27. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  28. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  29. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  30. 2022Kubric: A scalable dataset generatorcited by 373
  31. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  32. 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 317
  33. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  34. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  35. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  36. 2020Neural Radiance Flow for 4D View Synthesis and Video Processingcited by 270
  37. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  38. 2022DynIBaR: Neural Dynamic Image-Based Renderingcited by 224

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 43; 2026: 59.

116 papers.

Sort:

By area:

  1. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  2. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
  3. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  4. 2024Navigation World Modelscited by 262
  5. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  6. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  7. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  8. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
  9. 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 92
  10. 2024Dreamitate: Real-World Visuomotor Policy Learning via Video Generationcited by 84
  11. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
  12. 2025Follow-Your-Creation: Empowering 4D Creation through Video Inpaintingcited by 61
38 more
  1. 2024StreetCrafter: Street View Synthesis with Controllable Video Diffusion Modelscited by 49
  2. 2024Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modelingcited by 42
  3. 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videoscited by 39
  4. 2025GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Trackingcited by 37
  5. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 37
  6. 2025Reangle-A-Video: 4D Video Generation as Video-to-Video Translationcited by 35
  7. 2025UFM: A Simple Path towards Unified Dense Correspondence with Flowcited by 35
  8. 2025EX-4D: EXtreme Viewpoint 4D Video Synthesis via Depth Watertight Meshcited by 32
  9. 2025Any4D: Unified Feed-Forward Metric 4D Reconstructioncited by 32
  10. 2025POMATO: Marrying Pointmap Matching with Temporal Motion for Dynamic 3D Reconstructioncited by 24
  11. 2025Geometry-aware 4D Video Generation for Robot Manipulationcited by 23
  12. 2025Generative Video Motion Editing with 3D Point Trackscited by 21
  13. 2024Differentiable Robot Renderingcited by 20
  14. 2025EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidancecited by 20
  15. 2026VGGT-Omegacited by 20
  16. 2025AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletonscited by 17
  17. 2025Zero4D: Training-Free 4D Video Generation From Single Video Using Off-the-Shelf Video Diffusioncited by 15
  18. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  19. 2026Vista4D: Video Reshooting with 4D Point Cloudscited by 15
  20. 2025Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Controlcited by 14
  21. 2025EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memorycited by 14
  22. 2026VGG-T3: Offline Feed-Forward 3D Reconstruction at Scalecited by 14
  23. 2025VideoPanda: Video Panoramic Diffusion with Multi-view Attentioncited by 13
  24. 2025Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videoscited by 13
  25. 2026Walk through Paintings: Egocentric World Models from Internet Priorscited by 13
  26. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 13
  27. 2025Dynamic View Synthesis as an Inverse Problemcited by 12
  28. 20254Real-Video-V2: Fused View-Time Attention and Feedforward Reconstruction for 4D Scene Generationcited by 12
  29. 2025BulletTime: Decoupled Control of Time and Camera Pose for Video Generationcited by 12
  30. 2026AnyView: Synthesizing Any Novel View in Dynamic Scenescited by 12
  31. 2024SpatialDreamer: Self-supervised Stereo Video Synthesis from Monocular Inputcited by 11
  32. 2024Lifting Motion to the 3D World via 2D Diffusioncited by 11
  33. 2025Vivid4D: Improving 4D Reconstruction from Monocular Video by Video Inpaintingcited by 11
  34. 2025See4D: Pose-Free 4D Generation via Auto-Regressive Video Inpaintingcited by 11
  35. 2025TACO: Taming Diffusion for in-the-wild Video Amodal Completioncited by 10
  36. 2025ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encodingcited by 10
  37. 2026LaVR: Scene Latent Conditioned Generative Video Trajectory Re-Rendering using Large 4D Reconstruction Modelscited by 10
  38. 2024MOVIS: Enhancing Multi-Object Novel View Synthesis for Indoor Scenescited by 9

50 most cited of 116 All 116 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Models
  2. 2026Stabilizing Camera-Controlled Novel View Synthesis at Inference Time
  3. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis
  4. 2023ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models
  5. 2026OrbitNVS: Harnessing Video Diffusion Priors for Novel View Synthesis
  6. 2023Consistent View Synthesis with Pose-Guided Diffusion Models
  7. 2025DriveGen3D: Boosting Feed-Forward Driving Scene Generation with Efficient Video Diffusion
  8. 2025Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusion