mlsanityData to 14 Oct 2026

TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Models

arXiv 2503.05638 ↗First posted 7 Mar 2025cs.CV cs.AI cs.GRcited by 125

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present TrajectoryCrafter, a novel approach to redirect camera trajectories for monocular videos. By disentangling deterministic view transformations from stochastic content generation, our method achieves precise control over user-specified camera trajectories. We propose a novel dual-stream conditional video diffusion model that concurrently integrates point cloud renders and source videos as conditions, ensuring accurate view transformations and coherent 4D content generation. Instead of leveraging scarce multi-view videos, we curate a hybrid training dataset combining web-scale monocular videos with static multi-view datasets, by our innovative double-reprojection strategy, significantly fostering robust generalization across diverse scenes. Extensive evaluations on multi-view and large-scale monocular videos demonstrate the superior performance of our method.

Key facts. Cited by 125 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 97 of its 103 references that are in the map. First posted on arXiv 7 Mar 2025.

Builds on

97 of 103 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024RoomPainter: View-Integrated Diffusion for Consistent Indoor Scene Texturing21 Dec
  2. 2024Wonderland: Navigating 3D Scenes from a Single Image16 Dec
  3. 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpoints10 Dec
  4. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scale9 Dec
  5. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models27 Nov
  6. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion7 Nov
  7. 2024StereoCrafter: Diffusion-based Generation of Long and High-fidelity Stereoscopic 3D from Monocular Videos11 Sep
  8. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis3 Sep
  9. 2024HoloDreamer: Holistic 3D Panoramic World Generation from Text Descriptions21 Jul
  10. 2024MultiDiff: Consistent Novel View Synthesis from a Single Image26 Jun
  11. 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer24 May
  12. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Models16 May
3 more
  1. 2023ReconFusion: 3D Reconstruction with Diffusion Priors5 Dec
  2. 2023HiFi-123: Towards High-fidelity One Image to 3D Content Generation10 Oct
  3. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar

Other references · 82

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  9. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  10. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  11. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  12. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
38 more
  1. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  2. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  3. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  4. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  5. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  7. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  8. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  9. 2024Depth Anything V2cited by 1,422
  10. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  11. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
  12. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  13. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 751
  14. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 746
  15. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  16. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 671
  17. 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 667
  18. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
  19. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
  20. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  21. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 575
  22. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
  23. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
  24. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 535
  25. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 511
  26. 2023Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splattingcited by 476
  27. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 471
  28. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 471
  29. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  30. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  31. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  32. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
  33. 2022Kubric: A scalable dataset generatorcited by 381
  34. 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 344
  35. 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
  36. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 296
  37. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 287
  38. 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 279

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 38; 2026: 87.

125 papers.

Sort:

By area:

  1. 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 287
  2. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 170
  3. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 170
  4. 2025Video World Models with Long-term Spatial Memorycited by 136
  5. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 76
  6. 20253D and 4D World Modeling: A Surveycited by 67
  7. 2025Follow-Your-Creation: Empowering 4D Creation through Video Inpaintingcited by 62
  8. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 62
  9. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
  10. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  11. 2025GSFixer: Improving 3D Gaussian Splatting with Reference-Guided Video Diffusion Priorscited by 38
  12. 2025Matrix-3D: Omnidirectional Explorable 3D World Generationcited by 35
38 more
  1. 2025Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Controlcited by 34
  2. 20254DNeX: Feed-Forward 4D Generative Modeling Made Easycited by 34
  3. 2025EX-4D: EXtreme Viewpoint 4D Video Synthesis via Depth Watertight Meshcited by 32
  4. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 27
  5. 2025EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidancecited by 21
  6. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modelingcited by 21
  7. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 15
  8. 20254Real-Video-V2: Fused View-Time Attention and Feedforward Reconstruction for 4D Scene Generationcited by 14
  9. 2025Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videoscited by 14
  10. 20253D Scene Prompting for Scene-Consistent Camera-Controllable Video Generationcited by 13
  11. 2025Dynamic View Synthesis as an Inverse Problemcited by 12
  12. 2025See4D: Pose-Free 4D Generation via Auto-Regressive Video Inpaintingcited by 12
  13. 2025Light-X: Generative 4D Video Rendering with Camera and Illumination Controlcited by 12
  14. 2026AnyView: Synthesizing Any Novel View in Dynamic Scenescited by 12
  15. 2026Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulationcited by 12
  16. 2026Bernini: Latent Semantic Planning for Video Diffusioncited by 12
  17. 2025Vivid4D: Improving 4D Reconstruction from Monocular Video by Video Inpaintingcited by 11
  18. 2026World-R1: Reinforcing 3D Constraints for Text-to-Video Generationcited by 11
  19. 2026Prisma-World: Camera-Controllable Multi-Agent Video World Modelcited by 11
  20. 2025Follow-Your-Instruction: A Comprehensive MLLM Agent for World Data Synthesiscited by 10
  21. 2026ShareVerse: Multi-Agent Consistent Video Generation for Shared World Modelingcited by 10
  22. 2025WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generationcited by 9
  23. 2025SeqTex: Generate Mesh Textures in Video Sequencecited by 7
  24. 2026Memorize When Needed: Decoupled Memory Control for Spatially Consistent Long-Horizon Video Generationcited by 7
  25. 2026VideoWeave: Unlocking Geometric Consistency in Video Generation via Joint Geometry-Video Modelingcited by 7
  26. 2025GS2E: Gaussian Splatting is an Effective Data Generator for Event Stream Generationcited by 6
  27. 2025InverseCrafter: Efficient Video ReCapture as a Latent Domain Inverse Problemcited by 6
  28. 2026AlayaWorld: Long-Horizon and Playable Video World Generationcited by 6
  29. 2026Rays as Pixels: Learning A Joint Distribution of Videos and Camera Trajectoriescited by 5
  30. 2026World Tracing: Generative Pixel-Aligned Geometry Beyond the Visiblecited by 5
  31. 2025Virtually Being: Customizing Camera-Controllable Video Diffusion Models with Multi-View Performance Capturescited by 4
  32. 2026Syn4D: A Multiview Synthetic 4D Datasetcited by 4
  33. 2026MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffoldcited by 4
  34. 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivitycited by 4
  35. 2025Generative Photographic Control for Scene-Consistent Video Cinematic Editingcited by 3
  36. 2025Breaking the Vicious Cycle: Coherent 3D Gaussian Splatting from Sparse and Motion-Blurred Viewscited by 3
  37. 2025LangDriveCTRL: Natural Language Controllable Driving Scene Editing with Multi-modal Agentscited by 3
  38. 2026Memory-V2V: Memory-Augmented Video-to-Video Diffusion for Consistent Multi-Turn Editingcited by 3

50 most cited of 125 All 125 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026FreeOrbit4D: Training-Free Arbitrary Camera Redirection for Monocular Videos via Foreground-Complete 4D Reconstruction
  2. 2025MV-Performer: Taming Video Diffusion Model for Faithful and Synchronized Multi-view Performer Synthesis
  3. 2026Demiurge: Forging Navigable 4D Worlds via Autoregressive Video Rerendering
  4. 2025BulletTime: Decoupled Control of Time and Camera Pose for Video Generation
  5. 20264DStreamCtrl: Interactive Video Generation with Online 4D Control
  6. 2026PartiCam: Camera Controlled Video Generation with Reward Guidance
  7. 2025DreamJourney: Perpetual View Generation with Video Diffusion Models
  8. 2025OmniView: An All-Seeing Diffusion Model for 3D and 4D View Synthesis