mlsanityData to 7 Oct 2026

Consistent View Synthesis with Pose-Guided Diffusion Models

arXiv 2303.17598 ↗First posted 30 Mar 2023cs.CVcited by 117

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Novel view synthesis from a single image has been a cornerstone problem for many Virtual Reality applications that provide immersive experiences. However, most existing techniques can only synthesize novel views within a limited range of camera motion or fail to generate consistent and high-quality novel views under significant camera movement. In this work, we propose a pose-guided diffusion model to generate a consistent long-term video of novel views from a single image. We design an attention layer that uses epipolar lines as constraints to facilitate the association between different viewpoints. Experimental results on synthetic and real-world datasets demonstrate the effectiveness of the proposed diffusion model against state-of-the-art transformer-based and GAN-based approaches.

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 74 of its 86 references that are in the map. First posted on arXiv 30 Mar 2023.

Builds on

74 of 86 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  2. 2022Novel View Synthesis with Diffusion Models6 Oct
  3. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  4. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan
  5. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 69

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2020Denoising Diffusion Implicit Modelscited by 9,655
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  12. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
38 more
  1. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  2. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  4. 2017Non-local Neural Networkscited by 2,290
  5. 2022Video Diffusion Modelscited by 2,280
  6. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  7. 2021Vision Transformers for Dense Predictioncited by 1,789
  8. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  9. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  10. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  11. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  12. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  13. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  14. 2018Demystifying MMD GANscited by 1,268
  15. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  16. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  17. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 961
  18. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  19. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  20. 2018Self-Attention Generative Adversarial Networkscited by 809
  21. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  22. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  23. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  24. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  25. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  26. 2019Neural Point-Based Graphicscited by 326
  27. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  28. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  29. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  30. 2019DeepView: View Synthesis with Learned Gradient Descentcited by 253
  31. 2021Deblurring via Stochastic Refinementcited by 248
  32. 2018Learning Blind Video Temporal Consistencycited by 247
  33. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  34. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 206
  35. 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
  36. 2019Pushing the Boundaries of View Extrapolation with Multiplane Imagescited by 191
  37. 2015DeepStereo: Learning to Predict New Views from the World's Imagerycited by 185
  38. 2016View Synthesis by Appearance Flowcited by 172

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 29; 2024: 55; 2025: 29; 2026: 4.

117 papers.

Sort:

By area:

  1. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  2. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  3. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  4. 2023ReconFusion: 3D Reconstruction with Diffusion Priorscited by 317
  5. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  6. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 250
  7. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  8. 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generationcited by 163
  9. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  10. 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Controlcited by 117
  11. 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 115
  12. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
38 more
  1. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
  2. 2024MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Viewscited by 111
  3. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
  4. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  5. 2023Expressive Text-to-Image Generation with Rich Textcited by 89
  6. 2024I-Design: Personalized LLM Interior Designercited by 87
  7. 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attentioncited by 87
  8. 2023Consistent123: Improve Consistency for One Image to 3D Object Synthesiscited by 84
  9. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Modelscited by 79
  10. 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizercited by 79
  11. 2023Long-Term Photometric Consistent Novel View Synthesis with Diffusion Modelscited by 76
  12. 2023Free3D: Consistent Novel View Synthesis without 3D Representationcited by 72
  13. 2023EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusioncited by 72
  14. 2024GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warpingcited by 72
  15. 2024MultiDiff: Consistent Novel View Synthesis from a Single Imagecited by 67
  16. 2023ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusioncited by 65
  17. 2024Controlling Space and Time with Diffusion Modelscited by 65
  18. 2024Advances in 3D Generation: A Surveycited by 62
  19. 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
  20. 2024ViewDiff: 3D-Consistent Image Generation with Text-to-Image Modelscited by 57
  21. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 54
  22. 2023Direct2.5: Diverse Text-to-3D Generation via Multi-view 2.5D Diffusioncited by 50
  23. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  24. 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generationcited by 41
  25. 2024A Survey on Long Video Generation: Challenges, Methods, and Prospectscited by 40
  26. 2024SceneCraft: Layout-Guided 3D Scene Generationcited by 40
  27. 2024VistaDream: Sampling multiview consistent images for single-view scene reconstructioncited by 38
  28. 2024DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusioncited by 38
  29. 2024Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learningcited by 34
  30. 2024Put Myself in Your Shoes: Lifting the Egocentric Perspective from Exocentric Videoscited by 32
  31. 20243D-SceneDreamer: Text-Driven 3D-Consistent Scene Generationcited by 29
  32. 2024MagicMan: Generative Novel View Synthesis of Humans with 3D-Aware Diffusion and Iterative Refinementcited by 24
  33. 2024Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attentioncited by 24
  34. 20253D Scene Generation: A Surveycited by 24
  35. 2024CrossViewDiff: A Cross-View Diffusion Model for Satellite-to-Street View Synthesiscited by 23
  36. 2024DriveScape: Towards High-Resolution Controllable Multi-View Driving Video Generationcited by 22
  37. 2024Consolidating Attention Features for Multi-view Image Editingcited by 19
  38. 2024Novel View Synthesis with Pixel-Space Diffusion Modelscited by 16

50 most cited of 117 All 117 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models
  2. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Models
  3. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis
  4. 2025Geometric Consistency Refinement for Single Image Novel View Synthesis via Test-Time Adaptation of Diffusion Models
  5. 2025DT-NVS: Diffusion Transformers for Novel View Synthesis
  6. 2025DreamJourney: Perpetual View Generation with Video Diffusion Models
  7. 2026OrbitNVS: Harnessing Video Diffusion Priors for Novel View Synthesis
  8. 2026UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Models