# Pseudo-Generalized Dynamic View Synthesis from a Video

arXiv: https://arxiv.org/abs/2310.08587 · HTML: https://mlsanity.com/paper/2310.08587/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 74 of its 92 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Builds on (most cited first, 30 of 74)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields](https://mlsanity.com/paper/2304.06706/) (2023)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video](https://mlsanity.com/paper/2012.12247/) (2020)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [Fast Dynamic Radiance Fields with Time-Aware Neural Voxels](https://mlsanity.com/paper/2205.15285/) (2022)
- [OneFormer: One Transformer to Rule Universal Image Segmentation](https://mlsanity.com/paper/2211.06220/) (2022)
- [Novel View Synthesis with Diffusion Models](https://mlsanity.com/paper/2210.04628/) (2022)
- [TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement](https://mlsanity.com/paper/2306.08637/) (2023)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)

## Built on by (most cited first, 27 of 27)

- [MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffolds](https://mlsanity.com/paper/2405.17421/) (2024)
- [CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models](https://mlsanity.com/paper/2411.18613/) (2024)
- [Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videos](https://mlsanity.com/paper/2406.18717/) (2024)
- [GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesis](https://mlsanity.com/paper/2312.11458/) (2023)
- [Easi3R: Estimating Disentangled Motion from DUSt3R Without Training](https://mlsanity.com/paper/2503.24391/) (2025)
- [MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenes](https://mlsanity.com/paper/2405.14475/) (2024)
- [GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh](https://mlsanity.com/paper/2404.07991/) (2024)
- [Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videos](https://mlsanity.com/paper/2412.03526/) (2024)
- [MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Second](https://mlsanity.com/paper/2507.10065/) (2025)
- [Fast View Synthesis of Casual Videos with Soup-of-Planes](https://mlsanity.com/paper/2312.02135/) (2023)
- [Novel View Synthesis with Pixel-Space Diffusion Models](https://mlsanity.com/paper/2411.07765/) (2024)
- [DGS-LRM: Real-Time Deformable 3D Gaussian Reconstruction From Monocular Videos](https://mlsanity.com/paper/2506.09997/) (2025)
- [4Real-Video-V2: Fused View-Time Attention and Feedforward Reconstruction for 4D Scene Generation](https://mlsanity.com/paper/2506.18839/) (2025)
- [4D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Time](https://mlsanity.com/paper/2506.18890/) (2025)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)
- [D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Video](https://mlsanity.com/paper/2406.10078/) (2024)
- [Orientation-anchored Hyper-Gaussian for 4D Reconstruction from Casual Videos](https://mlsanity.com/paper/2509.23492/) (2025)
- [PAGE-4D: Disentangled pose and geometry estimation for vggt-4d perception](https://mlsanity.com/paper/2510.17568/) (2025)
- [Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometry](https://mlsanity.com/paper/2506.13697/) (2025)
- [Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models](https://mlsanity.com/paper/2511.00503/) (2025)
- [Uncertainty Matters in Dynamic Gaussian Splatting for Monocular 4D Reconstruction](https://mlsanity.com/paper/2510.12768/) (2025)
- [Prior-Enhanced Gaussian Splatting for Dynamic Scene Reconstruction from Casual Video](https://mlsanity.com/paper/2512.11356/) (2025)
- [ViDAR: Video Diffusion-Aware 4D Reconstruction From Monocular Inputs](https://mlsanity.com/paper/2506.18792/) (2025)
- [NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows](https://mlsanity.com/paper/2406.10543/) (2024)
- [Inferring Compositional 4D Scenes without Ever Seeing One](https://mlsanity.com/paper/2512.05272/) (2025)
- [OSN: Infinite Representations of Dynamic 3D Scenes from Monocular Videos](https://mlsanity.com/paper/2407.05615/) (2024)
- [SMG: Semantic Motion Graph for Monocular Dynamic Gaussian Splatting](https://mlsanity.com/paper/2608.31023/) (2026)

## Abstract (arXiv)

Rendering scenes observed in a monocular video from novel viewpoints is a challenging problem. For static scenes the community has studied both scene-specific optimization techniques, which optimize on every test scene, and generalized techniques, which only run a deep net forward pass on a test scene. In contrast, for dynamic scenes, scene-specific optimization techniques exist, but, to our best knowledge, there is currently no generalized method for dynamic novel view synthesis from a given monocular video. To answer whether generalized dynamic novel view synthesis from monocular videos is possible today, we establish an analysis framework based on existing techniques and work toward the generalized approach. We find a pseudo-generalized process without scene-specific appearance optimization is possible, but geometrically and temporally consistent depth estimates are needed. Despite no scene-specific appearance optimization, the pseudo-generalized approach improves upon some scene-specific methods.