mlsanityData to 7 Oct 2026

OmniCamera: A Unified Framework for Multi-task Video Generation with Arbitrary Camera Control

arXiv 2604.06010 ↗First posted 7 Apr 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Video fundamentally intertwines two crucial axes: the dynamic content of a scene and the camera motion through which it is observed. However, existing generation models often entangle these factors, limiting independent control. In this work, we introduce OmniCamera, a unified framework designed to explicitly disentangle and command these two dimensions. This compositional approach enables flexible video generation by allowing arbitrary pairings of camera and content conditions, unlocking unprecedented creative control. To overcome the fundamental challenges of modality conflict and data scarcity inherent in such a system, we present two key innovations. First, we construct OmniCAM, a novel hybrid dataset combining curated real-world videos with synthetic data that provides diverse paired examples for robust multi-task learning. Second, we propose a Dual-level Curriculum Co-Training strategy that mitigates modality interference and synergistically learns from diverse data sources. This strategy operates on two levels: first, it progressively introduces control modalities by difficulties (condition-level), and second, trains for precise control on synthetic data before adapting to real data for photorealism (data-level). As a result, OmniCamera achieves state-of-the-art performance, enabling flexible control for complex camera movements while maintaining superior visual quality.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 45 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

41 of 45 references.

References · 41

  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2022Flow Matching for Generative Modelingcited by 5,426
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  7. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  8. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  9. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  10. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  11. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  12. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
29 more
  1. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  2. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
  3. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  4. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  5. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  6. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  7. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  8. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  9. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  10. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
  11. 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 247
  12. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 224
  13. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
  14. 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generationcited by 163
  15. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  16. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 143
  17. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
  18. 2024Training-free Camera Control for Video Generationcited by 119
  19. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 116
  20. 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 97
  21. 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 92
  22. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
  23. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 54
  24. 2025UNIC: Unified In-Context Video Editingcited by 53
  25. 2025CamCloneMaster: Enabling Reference-based Camera Control for Video Generationcited by 44
  26. 2024I2VControl-Camera: Precise Video Camera Control with Adjustable Motion Strengthcited by 38
  27. 2025GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Trackingcited by 37
  28. 2025FloVD: Optical Flow Meets Video Diffusion Model for Enhanced Camera-Controlled Video Synthesiscited by 37
  29. 2025OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditionscited by 17

Built on by

Not cited yet.