Novel View Synthesis using DDIM Inversion
On the map: Diffusion models → 3D, point cloud and scene generation
Abstract (arXiv)
Single-image novel view synthesis requires viewpoint-dependent structural changes while retaining source-scene information and inferring unobserved content. We formulate this task as a camera-conditioned initialization problem for a frozen pretrained latent diffusion model. At an intermediate DDIM inversion timestep, we decompose the source representation into a structural mean and a source noise residual. A pose-conditioned translation U-Net (TUNet) learns to predict the target-view mean from the source mean. We then introduce a score-guided fusion strategy: the predicted mean and source noise residual form a proposal that is evaluated by the frozen denoiser, whose noise prediction is combined with the translated mean to initialize DDIM sampling. Only TUNet is trained, while the pretrained model synthesizes the final view without explicit reconstruction or rendering of a shared 3D scene. Experiments on MVImgNet and RealEstate10K demonstrate competitive performance. Beyond image-level metrics, Wasserstein comparisons show that the corrected initialization yields trajectory-curvature distributions closer to those of ground-truth target-initialized sampling than raw fusion. These results support structural translation and source-conditioned initialization as a practical approach to reusing frozen diffusion priors for novel view synthesis.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 55 of its 70 references that are in the map. First posted on arXiv 14 Aug 2025.
Builds on
55 of 70 references.
Closest ancestors · 17
Same direction, newest first
- 2024Novel View Synthesis with Pixel-Space Diffusion Models12 Nov
- 2024Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cycle28 Jul
- 2024MultiDiff: Consistent Novel View Synthesis from a Single Image26 Jun
- 2024Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Models18 Jun
- 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Models16 May
- 2024MVDiff: Scalable and Flexible Multi-View Diffusion for 3D Object Reconstruction from Single-View6 May
- 2024Magic-Boost: Boost 3D Generation with Multi-View Conditioned Diffusion9 Apr
- 2024Denoising Diffusion via Image-Based Rendering5 Feb
- 2024BoostDream: Efficient Refining for High-Quality Text-to-3D Generation from Multi-View Diffusion30 Jan
- 2023EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusion11 Dec
- 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
- 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
5 more
- 2023Viewpoint Textual Inversion: Discovering Scene Representations and 3D View Control in 2D Diffusion Models14 Sep
- 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
- 2023Long-Term Photometric Consistent Novel View Synthesis with Diffusion Models21 Apr
- 2023Consistent View Synthesis with Pose-Guided Diffusion Models30 Mar
- 2022DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Models22 Nov
Other references · 38
- 2017Attention Is All You Needcited by 60,361
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
- 2018Deep Retinex Decomposition for Low-Light Enhancementcited by 723
26 more
- 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
- 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 304
- 2020Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancementcited by 303
- 2022Perception Prioritized Training of Diffusion Modelscited by 303
- 2024DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalizationcited by 256
- 2018Stochastic Video Generation with a Learned Priorcited by 245
- 2023Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancementcited by 231
- 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 220
- 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 212
- 2021Neural Rays for Occlusion-aware Image-based Renderingcited by 192
- 2021Low-Light Image Enhancement with Normalizing Flowcited by 185
- 2022Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Methodcited by 174
- 2024ReNoise: Real Image Inversion Through Iterative Noisingcited by 128
- 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 116
- 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
- 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 100
- 2023Global Structure-Aware Diffusion Process for Low-Light Image Enhancementcited by 92
- 2024GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warpingcited by 73
- 2023Multiview Compressive Coding for 3D Reconstructioncited by 71
- 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
- 2021Low-light Image Enhancement via Breaking Down the Darknesscited by 65
- 2025HVI: A New Color Space for Low-light Image Enhancementcited by 63
- 2025A Fourier Space Perspective on Diffusion Modelscited by 52
- 2023NViST: In the Wild New View Synthesis from a Single Image with Transformerscited by 19
- 2024Structure Consistent Gaussian Splatting with Matching Prior for Few-shot Novel View Synthesiscited by 18
- 2025FreeInv: Free Lunch for Improving DDIM Inversioncited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Efficient-3DiM: Learning a Generalizable Single-image Novel-view Synthesizer in One Day
- 2023ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models
- 2022Novel View Synthesis with Diffusion Models
- 2026Inverting the Generation Process of Denoising Diffusion Implicit Models: Empirical Evaluation and a Novel Method
- 2025DT-NVS: Diffusion Transformers for Novel View Synthesis
- 2025Dynamic View Synthesis as an Inverse Problem
- 2024FSViewFusion: Few-Shots View Generation of Novel Objects
- 2025Learning High-Quality Initial Noise for Single-View Synthesis with Diffusion Models