mlsanityData to 14 Oct 2026

FloVD: Optical Flow Meets Video Diffusion Model for Enhanced Camera-Controlled Video Synthesis

arXiv 2502.08244 ↗First posted 12 Feb 2025cs.CVcited by 37

On the map: Diffusion models → Video generation

Abstract (arXiv)

We present FloVD, a novel video diffusion model for camera-controllable video generation. FloVD leverages optical flow to represent the motions of the camera and moving objects. This approach offers two key benefits. Since optical flow can be directly estimated from videos, our approach allows for the use of arbitrary training videos without ground-truth camera parameters. Moreover, as background optical flow encodes 3D correlation across different viewpoints, our method enables detailed camera control by leveraging the background motion. To synthesize natural object motion while supporting detailed camera control, our framework adopts a two-stage video synthesis pipeline consisting of optical flow generation and flow-conditioned video synthesis. Extensive experiments demonstrate the superiority of our method over previous approaches in terms of accurate camera control and natural object motion synthesis.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 41 of its 50 references that are in the map. First posted on arXiv 12 Feb 2025.

Builds on

41 of 50 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Model21 Oct
  2. 2024Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attention14 Oct
  3. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  4. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control17 Jul
  5. 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation4 Jun
  6. 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Control27 May
  7. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generation2 Apr
  8. 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motion5 Feb
  9. 2023MicroCinema: A Divide-and-Conquer Approach for Text-to-Video Generation30 Nov
  10. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  11. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  12. 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesis27 Apr
4 more
  1. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  2. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Models24 Mar
  3. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generation23 Nov
  4. 2022Video Diffusion Models7 Apr

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  3. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  4. 2016Improved Techniques for Training GANscited by 2,970
  5. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  6. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  7. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  8. 2024Depth Anything V2cited by 1,422
  9. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  10. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  11. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  12. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
13 more
  1. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 332
  2. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 264
  3. 2024Global Structure-from-Motion Revisitedcited by 218
  4. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 211
  5. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 211
  6. 2022Thin-Plate Spline Motion Model for Image Animationcited by 210
  7. 2023Inversion-Free Image Editing with Natural Languagecited by 106
  8. 2023Generative Image Dynamicscited by 94
  9. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 70
  10. 2024HumanVid: Demystifying Training Data for Camera-controllable Human Image Animationcited by 64
  11. 2020Animating Pictures with Eulerian Motion Fieldscited by 55
  12. 2021Controllable Animation of Fluid Elements in Still Imagescited by 45
  13. 2019Animating Landscape: Self-Supervised Learning of Decoupled Motion and Appearance for Single-Image Video Synthesiscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 17; 2026: 20.

37 papers.

Sort:

By area:

  1. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  2. 2025EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidancecited by 21
  3. 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memoriescited by 18
  4. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 15
  5. 2025Video-As-Prompt: Unified Semantic Control for Video Generationcited by 14
  6. 20253D Scene Prompting for Scene-Consistent Camera-Controllable Video Generationcited by 13
  7. 2025DualCamCtrl: Dual-Branch Diffusion Model for Geometry-Aware Camera-Controlled Video Generationcited by 11
  8. 2025Lynx: Towards High-Fidelity Personalized Video Generationcited by 9
  9. 2025VideoFrom3D: 3D Scene Video Generation via Complementary Image and Video Diffusion Modelscited by 8
  10. 20253D Scene Understanding Through Local Random Access Sequence Modelingcited by 5
  11. 2025TeRA: Rethinking Text-guided Realistic 3D Avatar Generationcited by 5
  12. 2025Open-world Hand-Object Interaction Video Generation Based on Structure and Contact-aware Representationcited by 5
25 more
  1. 2025WorldReel: 4D Video Generation with Consistent Geometry and Motion Modelingcited by 5
  2. 2025LAMP: Language-Assisted Motion Planning for Controllable Video Generationcited by 4
  3. 2025VHOI: Controllable Video Generation of Human-Object Interactions from Sparse Trajectories via Motion Densificationcited by 4
  4. 2025Learning Zero-Shot Subject-Driven Video Generation Using 1% Computecited by 2
  5. 2025CloseUpShot: Close-up Novel View Synthesis from Sparse-views via Point-conditioned Diffusion Modelcited by 2
  6. 2026MoRight: Motion Control Done Rightcited by 2
  7. 2026World-Ego Modeling for Long-Horizon Evolution in Hybrid Embodied Taskscited by 2
  8. 2026Unified 3D Scene Understanding Through Physical World Modelingcited by 2
  9. 2025FlowLoss: Dynamic Flow-Conditioned Loss Strategy for Video Diffusion Modelscited by 1
  10. 2026OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidancecited by 1
  11. 2026Auteur: Language-Driven Cinematographic Framing for Human-Centric Video Generationcited by 1
  12. 2026Go-with-the-Track: Video Compositing and Motion Control with Point Trackingcited by 1
  13. 2026From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Plantingcited by 1
  14. 2025GeoDiffMM: Geometry-Guided Conditional Diffusion for Motion Magnificationcited by 0
  15. 2026Beyond Inpainting: Unleash 3D Understanding for Precise Camera-Controlled Video Generationcited by 0
  16. 2026Video2LoRA: Unified Semantic-Controlled Video Generation via Per-Reference-Video LoRAcited by 0
  17. 2026DynaVid: Learning to Generate Highly Dynamic Videos using Synthetic Motion Datacited by 0
  18. 2026OmniCamera: A Unified Framework for Multi-task Video Generation with Arbitrary Camera Controlcited by 0
  19. 2026Perceptual 3D Simulation With Physical World Modelingcited by 0
  20. 2026LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation with Video Diffusion Modelscited by 0
  21. 2026ContactFlow: A video action conditioning that transfers across embodimentscited by 0
  22. 2026PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generationcited by 0
  23. 2026Bi-FlowGS: Bridging Generative View Completion and Gaussian Geometry through Bidirectional Flow Co-Refinementcited by 0
  24. 2026Action Forcing: Training World Models on Unsupervised Video by Recovering Underlying Egomotion Basescited by 0
  25. 2026Unifying Video Tasks via Spatiotemporal Analogycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis
  2. 2025Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising
  3. 2024Controllable Longer Image Animation with Diffusion Models
  4. 2026DreamLoop: Controllable Cinemagraph Generation from a Single Photograph
  5. 2025MotionFlow:Learning Implicit Motion Flow for Complex Camera Trajectory Control in Video Generation
  6. 2025LiON-LoRA: Rethinking LoRA Fusion to Unify Controllable Spatial and Temporal Generation for Video Diffusion
  7. 2025MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generation
  8. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers