mlsanityData to 14 Oct 2026

Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videos

arXiv 2507.12646 ↗First posted 16 Jul 2025cs.CVcited by 14

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We explore novel-view synthesis for dynamic scenes from monocular videos. Prior approaches rely on costly test-time optimization of 4D representations or do not preserve scene geometry when trained in a feed-forward manner. Our approach is based on three key insights: (1) covisible pixels (that are visible in both the input and target views) can be rendered by first reconstructing the dynamic 3D scene and rendering the reconstruction from the novel-views and (2) hidden pixels in novel views can be "inpainted" with feed-forward 2D video diffusion models. Notably, our video inpainting diffusion model (CogNVS) can be self-supervised from 2D videos, allowing us to train it on a large corpus of in-the-wild videos. This in turn allows for (3) CogNVS to be applied zero-shot to novel test videos via test-time finetuning. We empirically verify that CogNVS outperforms almost all prior art for novel-view synthesis of dynamic scenes from monocular videos.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 88 of its 98 references that are in the map. First posted on arXiv 16 Jul 2025.

Builds on

88 of 98 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Models7 Mar
  2. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models27 Nov
  3. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis3 Sep
  4. 2024Controlling Space and Time with Diffusion Models10 Jul
  5. 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer24 May
  6. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion14 Nov
  7. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior25 Oct
  8. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  9. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
  10. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  11. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  12. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
3 more
  1. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  2. 2022Novel View Synthesis with Diffusion Models6 Oct
  3. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 73

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  5. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  6. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  7. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  8. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  9. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  10. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  11. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  12. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
38 more
  1. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
  2. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  3. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  4. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 751
  5. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  6. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 737
  7. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 735
  8. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 724
  9. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 671
  10. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
  11. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
  12. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 511
  13. 2023Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splattingcited by 476
  14. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
  15. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  16. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  17. 2021Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughscited by 398
  18. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 349
  19. 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 344
  20. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  21. 2017"Zero-Shot" Super-Resolution using Deep Internal Learningcited by 286
  22. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 250
  23. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  24. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  25. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 234
  26. 2018YouTube-VOS: Sequence-to-Sequence Video Object Segmentationcited by 232
  27. 2021BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Renderingcited by 229
  28. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 228
  29. 2022DynIBaR: Neural Dynamic Image-Based Renderingcited by 224
  30. 2022Test-Time Training with Masked Autoencoderscited by 212
  31. 2022Monocular Dynamic View Synthesis: A Reality Checkcited by 200
  32. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 179
  33. 2023WonderJourney: Going from Anywhere to Everywherecited by 158
  34. 2020TAO: A Large-Scale Benchmark for Tracking Any Objectcited by 147
  35. 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 144
  36. 2023Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fieldscited by 143
  37. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 139
  38. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 117

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 2025Any4D: Unified Feed-Forward Metric 4D Reconstructioncited by 35
  2. 2025ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encodingcited by 11
  3. 2025InverseCrafter: Efficient Video ReCapture as a Latent Domain Inverse Problemcited by 6
  4. 2026Feed-Forward 3D Scene Modeling: A Problem-Driven Perspectivecited by 6
  5. 2026FrameCrafter: Novel View Synthesis as Video Completioncited by 4
  6. 2026Geo-Align: Video Generation Alignment via Metric Geometry Rewardcited by 3
  7. 2026Lift4D: Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wildcited by 2
  8. 2026MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generationcited by 2
  9. 2026DRoPS: Dynamic 3D Reconstruction of Pre-Scanned Objectscited by 1
  10. 2026VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matchingcited by 0
  11. 2026WarpHammer: Densifying Scene Warps with 3D Object Priors for Extreme View Synthesiscited by 0
  12. 2026FlashRender: Few-Step Generative Rendering via Camera-Controlled Video MeanFlowcited by 0
2 more
  1. 2026Can 4D Foundation Models Remember?cited by 0
  2. 2026Eulerian Motion Reconstruction for Water Scenerycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Diffusion Priors for Dynamic View Synthesis from Monocular Videos
  2. 2025Dynamic View Synthesis as an Inverse Problem
  3. 2023ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models
  4. 2023iNVS: Repurposing Diffusion Inpainters for Novel View Synthesis
  5. 2026AnyView: Synthesizing Any Novel View in Dynamic Scenes
  6. 2025ViDAR: Video Diffusion-Aware 4D Reconstruction From Monocular Inputs
  7. 2025Novel View Synthesis from A Few Glimpses via Test-Time Natural Video Completion
  8. 2024Novel View Synthesis with Pixel-Space Diffusion Models