mlsanityData to 14 Oct 2026

ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuning

arXiv 2411.05003 ↗First posted 7 Nov 2024cs.CV cs.AI cs.GRcited by 70

Not on the map. Why?

Cited by 28 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recently, breakthroughs in video modeling have allowed for controllable camera trajectories in generated videos. However, these methods cannot be directly applied to user-provided videos that are not generated by a video model. In this paper, we present ReCapture, a method for generating new videos with novel camera trajectories from a single user-provided video. Our method allows us to re-generate the reference video, with all its existing scene motion, from vastly different angles and with cinematic camera motion. Notably, using our method we can also plausibly hallucinate parts of the scene that were not observable in the reference video. Our method works by (1) generating a noisy anchor video with a new camera trajectory using multiview diffusion models or depth-based point cloud rendering and then (2) regenerating the anchor video into a clean and temporally consistent reangled video using our proposed masked video fine-tuning technique.

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). Builds on 93 of its 105 references that are in the map. First posted on arXiv 7 Nov 2024.

Builds on

93 of 105 references.

References · 93

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  9. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  10. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  11. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  12. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
38 more
  1. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  2. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  3. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  5. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  6. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  7. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  8. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  9. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  10. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  11. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
  12. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,081
  13. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  14. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  15. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  16. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 751
  17. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  18. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  19. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  20. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  21. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
  22. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  23. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 579
  24. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 546
  25. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  26. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
  27. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
  28. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
  29. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 465
  30. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  31. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  32. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  33. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  34. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  35. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
  36. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 388
  37. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 350
  38. 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 318

50 most cited of 93 All 93 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 31; 2026: 33.

70 papers.

Sort:

By area:

  1. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  2. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  3. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 161
  4. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 154
  5. 2025Video World Models with Long-term Spatial Memorycited by 136
  6. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 125
  7. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 117
  8. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65
  9. 2025Follow-Your-Creation: Empowering 4D Creation through Video Inpaintingcited by 62
  10. 2025CamCloneMaster: Enabling Reference-based Camera Control for Video Generationcited by 46
  11. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  12. 2025GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priorscited by 38
38 more
  1. 2025FloVD: Optical Flow Meets Video Diffusion Model for Enhanced Camera-Controlled Video Synthesiscited by 37
  2. 2025Reangle-A-Video: 4D Video Generation as Video-to-Video Translationcited by 35
  3. 2025EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidancecited by 21
  4. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modelingcited by 21
  5. 2025Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Controlcited by 18
  6. 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memoriescited by 18
  7. 2026MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulationcited by 17
  8. 2025Zero4D: Training-Free 4D Video Generation From Single Video Using Off-the-Shelf Video Diffusioncited by 16
  9. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 15
  10. 2025MotionV2V: Editing Motion in a Videocited by 14
  11. 2024NVComposer: Boosting Generative Novel View Synthesis with Multiple Sparse and Unposed Imagescited by 13
  12. 2024Latent-Reframe: Enabling Camera Control for Video Diffusion Model without Trainingcited by 13
  13. 2026ReRoPE: Repurposing RoPE for Relative Camera Controlcited by 13
  14. 2025Dynamic View Synthesis as an Inverse Problemcited by 12
  15. 2025Light-X: Generative 4D Video Rendering with Camera and Illumination Controlcited by 12
  16. 2026AnyView: Synthesizing Any Novel View in Dynamic Scenescited by 12
  17. 2025ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encodingcited by 11
  18. 2025SpaceTimePilot: Generative Rendering of Dynamic Scenes Across Space and Timecited by 11
  19. 2026SpatialEdit: Benchmarking Fine-Grained Image Spatial Editingcited by 10
  20. 2024Learning Camera Movement Control from Real-World Drone Videoscited by 9
  21. 2025WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generationcited by 9
  22. 2025Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometrycited by 7
  23. 2025Eye2Eye: A Simple Approach for Monocular-to-Stereo Video Synthesiscited by 6
  24. 2025PostCam: Camera-Controllable Novel-View Video Generation with Query-Shared Cross-Attentioncited by 6
  25. 2026AlayaWorld: Long-Horizon and Playable Video World Generationcited by 6
  26. 2025OmniView: An All-Seeing Diffusion Model for 3D and 4D View Synthesiscited by 4
  27. 20254D Gaussian Splatting as a Learned Dynamical Systemcited by 4
  28. 2026Versatile Editing of Video Content, Actions, and Dynamics without Trainingcited by 4
  29. 2026Pantheon360: Taming Digital Twin Generation via 3D-Aware 360deg Video Diffusioncited by 4
  30. 2026CameraNoise: Enabling Faithful Camera Control in Video Diffusion through Geometry-Flow-Guided Noise Warpingcited by 4
  31. 2026Memory-V2V: Memory-Augmented Video-to-Video Diffusion for Consistent Multi-Turn Editingcited by 3
  32. 2026Geo-Align: Video Generation Alignment via Metric Geometry Rewardcited by 3
  33. 2026UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Modelscited by 3
  34. 2025BulletGen: Improving 4D Reconstruction with Bullet-Time Generationcited by 2
  35. 2026Reshoot-Anything: A Self-Supervised Model for In-the-Wild Video Reshootingcited by 2
  36. 2026MoCam: Unified Novel View Synthesis via Structured Denoising Dynamicscited by 2
  37. 2026DisCo: World Models with Discrete Camera Motion Controlcited by 2
  38. 2026Track2View: 4D-Consistent Camera-Controlled Video Generation via Paired 3D Point Trackscited by 2

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising
  2. 2026PartiCam: Camera Controlled Video Generation with Reward Guidance
  3. 2026Efficient Camera-Controlled Video Generation of Static Scenes via Sparse Diffusion and 3D Rendering
  4. 2022Video Diffusion Models
  5. 2024Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolation
  6. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applications
  7. 2024Controllable Longer Image Animation with Diffusion Models
  8. 2025Frame In-N-Out: Unbounded Controllable Image-to-Video Generation