FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control
Not on the map. Why?
Abstract (arXiv)
Generative video compositing, which involves inserting external assets seamlessly into existing video sequences, is essential for content creation and visual effects. However, existing approaches suffer from a control-fidelity trade-off: they either hallucinate motion from static images, failing to preserve the dynamics of pre-animated assets, or lack fine-grained spatial control for precise asset placement along user-defined trajectories. We propose FlexComposer, a unified framework that standardizes video compositing as a trajectory-guided conditional generation task, enabling the seamless integration of both static images and dynamic footage. Our approach introduces three key designs: (1) a Unified Canonical Foreground Representation that decouples an object's intrinsic motion from its global displacement, standardizing heterogeneous inputs into a stabilized, centered latent space; (2) a Spatial-Aware Latent Injection strategy that exploits the translation equivariance of VAE latent spaces to transport canonical features onto target trajectories via a parameter-free mechanism; and (3) a Hybrid Dataset and Synthetic-to-Real Curriculum that synergizes procedural simulation, real-world cinematic footage, and generative data to implicitly learn physically plausible illumination and shadow harmonization. This unified design handles diverse inputs from product photos to dynamic subjects achieving high-fidelity motion control and environmental integration without the need for explicit 3D reconstruction or auxiliary learnable adapters. Extensive experiments demonstrate that FlexComposer outperforms state-of-the-art methods in visual quality, temporal consistency, and trajectory adherence.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 85 of its 97 references that are in the map. First posted on arXiv 31 Jul 2026.
Builds on
85 of 97 references.
References · 85
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023Segment Anythingcited by 8,158
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
- 2022Video Diffusion Modelscited by 2,280
38 more
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
- 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
- 2025VACE: All-in-One Video Creation and Editingcited by 467
- 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
- 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
- 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
- 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
- 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
- 2022Kubric: A scalable dataset generatorcited by 373
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
- 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 312
- 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
- 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
- 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
- 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
- 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
- 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
- 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 172
- 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 161
- 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
- 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 151
- 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
- 2024Boximator: Generating Rich and Controllable Motions for Video Synthesiscited by 107
- 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 92
- 2023TrailBlazer: Trajectory Control for Diffusion-Based Video Generationcited by 90
- 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
- 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 84
- 2023Consistent Video-to-Video Transfer Using Synthetic Datasetcited by 79
- 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizercited by 79
50 most cited of 85 All 85 (JSON)
Built on by
Not cited yet.