mlsanityData to 7 Oct 2026

FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control

arXiv 2607.29627 ↗First posted 31 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Generative video compositing, which involves inserting external assets seamlessly into existing video sequences, is essential for content creation and visual effects. However, existing approaches suffer from a control-fidelity trade-off: they either hallucinate motion from static images, failing to preserve the dynamics of pre-animated assets, or lack fine-grained spatial control for precise asset placement along user-defined trajectories. We propose FlexComposer, a unified framework that standardizes video compositing as a trajectory-guided conditional generation task, enabling the seamless integration of both static images and dynamic footage. Our approach introduces three key designs: (1) a Unified Canonical Foreground Representation that decouples an object's intrinsic motion from its global displacement, standardizing heterogeneous inputs into a stabilized, centered latent space; (2) a Spatial-Aware Latent Injection strategy that exploits the translation equivariance of VAE latent spaces to transport canonical features onto target trajectories via a parameter-free mechanism; and (3) a Hybrid Dataset and Synthetic-to-Real Curriculum that synergizes procedural simulation, real-world cinematic footage, and generative data to implicitly learn physically plausible illumination and shadow harmonization. This unified design handles diverse inputs from product photos to dynamic subjects achieving high-fidelity motion control and environmental integration without the need for explicit 3D reconstruction or auxiliary learnable adapters. Extensive experiments demonstrate that FlexComposer outperforms state-of-the-art methods in visual quality, temporal consistency, and trajectory adherence.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 85 of its 97 references that are in the map. First posted on arXiv 31 Jul 2026.

Builds on

85 of 97 references.

References · 85

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Segment Anythingcited by 8,158
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2022Flow Matching for Generative Modelingcited by 5,426
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  9. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  10. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  11. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  12. 2022Video Diffusion Modelscited by 2,280
38 more
  1. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  2. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  3. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  4. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  5. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  6. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  7. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  8. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  9. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  10. 2025VACE: All-in-One Video Creation and Editingcited by 467
  11. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  12. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  13. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  14. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  15. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  16. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  17. 2022Kubric: A scalable dataset generatorcited by 373
  18. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  19. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 312
  20. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  21. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
  22. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
  23. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
  24. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
  25. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
  26. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
  27. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 172
  28. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 161
  29. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  30. 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 151
  31. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
  32. 2024Boximator: Generating Rich and Controllable Motions for Video Synthesiscited by 107
  33. 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 92
  34. 2023TrailBlazer: Trajectory Control for Diffusion-Based Video Generationcited by 90
  35. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
  36. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 84
  37. 2023Consistent Video-to-Video Transfer Using Synthetic Datasetcited by 79
  38. 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizercited by 79

50 most cited of 85 All 85 (JSON)

Built on by

Not cited yet.