mlsanityData to 7 Oct 2026

Monocular Dynamic View Synthesis: A Reality Check

arXiv 2210.13445 ↗First posted 24 Oct 2022cs.CVcited by 199

Not on the map. Why?

Abstract (arXiv)

We study the recent progress on dynamic view synthesis (DVS) from monocular video. Though existing approaches have demonstrated impressive results, we show a discrepancy between the practical capture process and the existing experimental protocols, which effectively leaks in multi-view signals during training. We define effective multi-view factors (EMFs) to quantify the amount of multi-view signal present in the input capture sequence based on the relative camera-scene motion. We introduce two new metrics: co-visibility masked image metrics and correspondence accuracy, which overcome the issue in existing protocols. We also propose a new iPhone dataset that includes more diverse real-life deformation sequences. Using our proposed experimental protocol, we show that the state-of-the-art approaches observe a 1-2 dB drop in masked PSNR in the absence of multi-view cues and 4-5 dB drop when modeling complex motion. Code and data can be found at https://hangg7.com/dycheck.

Key facts. Cited by 199 papers in the map (data through 7 Oct 2026). Builds on 41 of its 66 references that are in the map. First posted on arXiv 24 Oct 2022.

Builds on

41 of 66 references.

References · 41

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  4. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  5. 2021Vision Transformers for Dense Predictioncited by 1,789
  6. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  7. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  8. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  9. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  10. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  11. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  12. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
29 more
  1. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  2. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  3. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 470
  4. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  5. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  6. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  7. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  8. 2020A Metric Learning Reality Checkcited by 317
  9. 2020Neural Radiance Flow for 4D View Synthesis and Video Processingcited by 270
  10. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
  11. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  12. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  13. 2021Neural Articulated Radiance Fieldcited by 196
  14. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 195
  15. 2022TAVA: Template-free Animatable Volumetric Actorscited by 172
  16. 2020Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Cameracited by 166
  17. 2021Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Renderingcited by 158
  18. 2022D2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Videocited by 136
  19. 2021End-to-End Referring Video Object Segmentation with Multimodal Transformerscited by 135
  20. 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 115
  21. 2016Controlling Perceptual Factors in Neural Style Transfercited by 99
  22. 2020Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loopcited by 81
  23. 2020Articulation-aware Canonical Surface Mappingcited by 80
  24. 2020Transforming and Projecting Images into Class-conditional Generative Networkscited by 76
  25. 2019Canonical Surface Mapping via Geometric Cycle Consistencycited by 62
  26. 2014Category-Specific Object Reconstruction from a Single Imagecited by 56
  27. 2020Neural Non-Rigid Trackingcited by 49
  28. 2019SurfelWarp: Efficient Non-Volumetric Single View Dynamic Reconstructioncited by 31
  29. 2022Real-Time Neural Character Rendering with Pose-Guided Multiplane Imagescited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 33; 2024: 41; 2025: 66; 2026: 53.

199 papers.

Sort:

By area:

  1. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  2. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
  3. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  4. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  5. 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 341
  6. 2022DynIBaR: Neural Dynamic Image-Based Renderingcited by 224
  7. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 224
  8. 2022Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Renderingcited by 217
  9. 2024VastGaussian: Vast 3D Gaussians for Large Scene Reconstructioncited by 209
  10. 2023Robust Dynamic Radiance Fieldscited by 193
  11. 2023Tracking Everything Everywhere All at Oncecited by 190
  12. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 185
38 more
  1. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 177
  2. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  3. 2022NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fieldscited by 151
  4. 2023SUDS: Scalable Urban Dynamic Scenescited by 151
  5. 2023Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Videocited by 124
  6. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 120
  7. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 116
  8. 2023NeRF-DS: Neural Radiance Fields for Dynamic Specular Objectscited by 110
  9. 2024Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splattingcited by 104
  10. 2024Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videoscited by 96
  11. 2023NerfAcc: Efficient Sampling Accelerates NeRFscited by 90
  12. 2023Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenescited by 89
  13. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  14. 2025Easi3R: Estimating Disentangled Motion from DUSt3R Without Trainingcited by 88
  15. 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 88
  16. 2024AbsGS: Recovering Fine Details for 3D Gaussian Splattingcited by 80
  17. 2025PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videoscited by 78
  18. 2023Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFscited by 76
  19. 2023Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clipscited by 73
  20. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 72
  21. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
  22. 2024MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splattingcited by 62
  23. 2023Neural Parametric Gaussians for Monocular Non-Rigid Object Reconstructioncited by 60
  24. 2024MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priorscited by 51
  25. 2023EPIC Fields: Marrying 3D Geometry and Video Understandingcited by 49
  26. 2024Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videoscited by 48
  27. 2024EgoLifter: Open-world 3D Segmentation for Egocentric Perceptioncited by 46
  28. 20254DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videoscited by 46
  29. 2023Flow supervision for Deformable NeRFcited by 42
  30. 2026NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videoscited by 39
  31. 2023DynPoint: Dynamic Neural Point For View Synthesiscited by 37
  32. 2025GS-DiT: Advancing Video Generation with Pseudo 4D Gaussian Fields through Efficient Dense 3D Point Trackingcited by 37
  33. 2023Deformable Neural Radiance Fields using RGB and Event Camerascited by 31
  34. 2025MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Secondcited by 31
  35. 2022State of the Art in Dense Monocular Non-Rigid 3D Reconstructioncited by 30
  36. 2025MoDec-GS: Global-to-Local Motion Decomposition and Temporal Interval Adjustment for Compact Dynamic 3D Gaussian Splattingcited by 30
  37. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  38. 20244D Gaussian Splatting in the Wild with Uncertainty-Aware Regularizationcited by 28

50 most cited of 199 All 199 (JSON)