mlsanityData to 14 Oct 2026

Novel View Extrapolation with Video Diffusion Priors

arXiv 2411.14208 ↗First posted 21 Nov 2024cs.CVcited by 25

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

The field of novel view synthesis has made significant strides thanks to the development of radiance field methods. However, most radiance field techniques are far better at novel view interpolation than novel view extrapolation where the synthesis novel views are far beyond the observed training views. We design ViewExtrapolator, a novel view synthesis approach that leverages the generative priors of Stable Video Diffusion (SVD) for realistic novel view extrapolation. By redesigning the SVD denoising process, ViewExtrapolator refines the artifact-prone views rendered by radiance fields, greatly enhancing the clarity and realism of the synthesized novel views. ViewExtrapolator is a generic novel view extrapolator that can work with different types of 3D rendering such as views rendered from point clouds when only a single view or monocular video is available. Additionally, ViewExtrapolator requires no fine-tuning of SVD, making it both data-efficient and computation-efficient. Extensive experiments demonstrate the superiority of ViewExtrapolator in novel view extrapolation. Project page: urlhttps://kunhao-liu.github.io/ViewExtrapolator/.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 51 of its 56 references that are in the map. First posted on arXiv 21 Nov 2024.

Builds on

51 of 56 references.

Closest ancestors · 19

Same direction, newest first

  1. 2024ExtraNeRF: Visibility-Aware View Extrapolation of Neural Radiance Fields with Diffusion Models10 Jun
  2. 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer24 May
  3. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Models16 May
  4. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion18 Mar
  5. 2023ReconFusion: 3D Reconstruction with Diffusion Priors5 Dec
  6. 2023ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models3 Dec
  7. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation2 Dec
  8. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  9. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  10. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  11. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  12. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
7 more
  1. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  2. 2023DiffusioNeRF: Regularizing Neural Radiance Fields with Denoising Diffusion Models23 Feb
  3. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  4. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  5. 2022Novel View Synthesis with Diffusion Models6 Oct
  6. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  7. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan

Other references · 32

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  7. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  8. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  9. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  10. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
  11. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  12. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
20 more
  1. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
  2. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 861
  3. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  4. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 746
  5. 2023Scaffold-GS: Structured 3D Gaussians for View-Adaptive Renderingcited by 695
  6. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  7. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  8. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  9. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  10. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 382
  11. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 204
  12. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 180
  13. 2023Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Imagescited by 173
  14. 2023Weakly Supervised 3D Open-vocabulary Segmentationcited by 135
  15. 2024Training-free Camera Control for Video Generationcited by 122
  16. 2023StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fieldscited by 79
  17. 2024StyleGaussian: Instant 3D Style Transfer with Gaussian Splattingcited by 79
  18. 2023Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFscited by 76
  19. 2024ViewDiff: 3D-Consistent Image Generation with Text-to-Image Modelscited by 57
  20. 2022Ray Priors through Reprojection: Improving Neural Radiance Fields for Novel View Extrapolationcited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 9.

25 papers.

Sort:

By area:

  1. 2025Controllable Video Generation: A Surveycited by 98
  2. 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 36
  3. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 27
  4. 2025WorldExplorer: Towards Generating Fully Navigable 3D Scenescited by 26
  5. 2025Viser: Imperative, Web-based 3D Visualization in Pythoncited by 23
  6. 2025Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Controlcited by 18
  7. 2025Zero4D: Training-Free 4D Video Generation From Single Video Using Off-the-Shelf Video Diffusioncited by 16
  8. 2025See4D: Pose-Free 4D Generation via Auto-Regressive Video Inpaintingcited by 12
  9. 2025Vivid4D: Improving 4D Reconstruction from Monocular Video by Video Inpaintingcited by 11
  10. 2025G4Splat: Geometry-Guided Gaussian Splatting with Generative Priorcited by 10
  11. 2025ExploreGS: Explorable 3D Scene Reconstruction with Virtual Camera Samplings and Diffusion Priorscited by 6
  12. 2025ExtraGS: Geometric-Aware Trajectory Extrapolation with Uncertainty-Guided Generative Priorscited by 4
13 more
  1. 2025GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstructioncited by 4
  2. 2026Warp-as-History: Generalizable Camera-Controlled Video Generation from One Training Videocited by 4
  3. 2026TryOnCrafter: Unleashing Camera Trajectories for Realistic Video Virtual Try-on via a Renderable 4D Try-on Proxycited by 3
  4. 2025ExScene: Free-View 3D Scene Reconstruction with Gaussian Splatting from a Single Imagecited by 2
  5. 2025Sketch3DVE: Sketch-based 3D-Aware Scene Video Editingcited by 2
  6. 2025CloseUpShot: Close-up Novel View Synthesis from Sparse-views via Point-conditioned Diffusion Modelcited by 2
  7. 2026FreeFix: Boosting 3D Gaussian Splatting via Fine-Tuning-Free Diffusion Modelscited by 2
  8. 2026Real2SAM2Real: Generative 3D Caches as Complementary Context for Video Diffusioncited by 1
  9. 2026Leveling3D: Leveling Up 3D Reconstruction with Feed-Forward 3D Gaussian Splatting and Geometry-Aware Generationcited by 0
  10. 2026Lifting Unlabeled Internet-level Data for 3D Scene Understandingcited by 0
  11. 2026Efficient 3D Content Reconstruction and Generationcited by 0
  12. 2026Mind the Gap: Standard 3DGS Evaluation Primarily Measures Near-Trajectory Interpolationcited by 0
  13. 2026InceptionGS: Generative Bootstrapping for Large-Scale Gaussian Splatting under Unstructured View Samplingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Models
  2. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis
  3. 2024ViewFusion: Towards Multi-View Consistency via Interpolated Denoising
  4. 2026ArtiFixer: Enhancing and Extending 3D Reconstruction with Auto-Regressive Diffusion Models
  5. 2024ViewFusion: Learning Composable Diffusion Models for Novel View Synthesis
  6. 2025Splatent: Splatting Diffusion Latents for Novel View Synthesis
  7. 2023Optimized View and Geometry Distillation from Multi-view Diffuser
  8. 2023Efficient-3DiM: Learning a Generalizable Single-image Novel-view Synthesizer in One Day