mlsanityData to 7 Oct 2026

iNVS: Repurposing Diffusion Inpainters for Novel View Synthesis

arXiv 2310.16167 ↗First posted 24 Oct 2023cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

We present a method for generating consistent novel views from a single source image. Our approach focuses on maximizing the reuse of visible pixels from the source image. To achieve this, we use a monocular depth estimator that transfers visible pixels from the source view to the target view. Starting from a pre-trained 2D inpainting diffusion model, we train our method on the large-scale Objaverse dataset to learn 3D object priors. While training we use a novel masking mechanism based on epipolar lines to further improve the quality of our approach. This allows our framework to perform zero-shot novel view synthesis on a variety of objects. We evaluate the zero-shot abilities of our framework on three challenging datasets: Google Scanned Objects, Ray Traced Multiview, and Common Objects in 3D. See our webpage for more details: https://yashkant.github.io/invs/

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 58 of its 77 references that are in the map. First posted on arXiv 24 Oct 2023.

Builds on

58 of 77 references.

References · 58

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  10. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  11. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  12. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
38 more
  1. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  2. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  3. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  4. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  5. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  6. 2020First Order Motion Model for Image Animationcited by 698
  7. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  8. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  9. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  10. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  11. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  12. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  13. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  14. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  15. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  16. 2022Novel View Synthesis with Diffusion Modelscited by 299
  17. 2020Softmax Splatting for Video Frame Interpolationcited by 276
  18. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  19. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  20. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 256
  21. 2021NeRV: Neural Representations for Videoscited by 255
  22. 2018Stochastic Video Generation with a Learned Priorcited by 244
  23. 2019Multi-Garment Net: Learning to Dress 3D People from Imagescited by 244
  24. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  25. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Modelscited by 234
  26. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  27. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  28. 2018Animating Arbitrary Objects via Deep Motion Transfercited by 206
  29. 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
  30. 2022ECON: Explicit Clothed humans Optimized via Normal integrationcited by 199
  31. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179
  32. 2017Decomposing Motion and Content for Natural Video Sequence Predictioncited by 168
  33. 2018Learning to Decompose and Disentangle Representations for Video Predictioncited by 120
  34. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  35. 2018Extreme View Synthesiscited by 94
  36. 2023Anything-3D: Towards Single-view Anything Reconstruction in the Wildcited by 91
  37. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
  38. 2020Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Imagecited by 77

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 8; 2025: 7; 2026: 1.

20 papers.

Sort:

By area:

  1. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
  2. 2023Free3D: Consistent Novel View Synthesis without 3D Representationcited by 72
  3. 2023NeRFiller: Completing Scenes via Generative 3D Inpaintingcited by 62
  4. 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
  5. 2024MegaScenes: Scene-Level View Synthesis at Scalecited by 58
  6. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  7. 2023HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3Dcited by 27
  8. 2025Pippo: High-Resolution Multi-View Humans from a Single Imagecited by 20
  9. 2025Sharp Monocular View Synthesis in Less Than a Secondcited by 19
  10. 2026Vista4D: Video Reshooting with 4D Point Cloudscited by 15
  11. 2024Zero-to-Hero: Enhancing Zero-Shot Novel View Synthesis via Attention Map Filteringcited by 9
  12. 2024HouseCrafter: Lifting Floorplans to 3D Scenes with 2D Diffusion Modelcited by 9
8 more
  1. 2024How to Use Diffusion Priors under Sparse Views?cited by 8
  2. 2025Elevating 3D Models: High-Quality Texture and Geometry Refinement from a Low-Quality Modelcited by 7
  3. 2023VaLID: Variable-Length Input Diffusion for Novel View Synthesiscited by 4
  4. 2024Reconstructing the Image Stitching Pipeline: Integrating Fusion and Rectangling into a Unified Inpainting Modelcited by 4
  5. 2025Appreciate the View: A Task-Aware Evaluation Framework for Novel View Synthesiscited by 4
  6. 2025Fillerbuster: Unified Generative Scene Completion Model for Casual Capturescited by 2
  7. 2025Track, Inpaint, Resplat: Subject-driven 3D and 4D Generation with Progressive Texture Infillingcited by 2
  8. 2025An evaluation of SVBRDF Prediction from Generative Image Models for Appearance Modeling of 3D Scenescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Novel View Synthesis with Pixel-Space Diffusion Models
  2. 2023Inpaint3D: 3D Scene Content Generation using 2D Inpainting Diffusion
  3. 2025Geometry-Aware Diffusion Models for Multiview Scene Inpainting
  4. 2025Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videos
  5. 2024InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior
  6. 2023ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image
  7. 20243D-Consistent Image Inpainting with Diffusion Models
  8. 2024MOVIS: Enhancing Multi-Object Novel View Synthesis for Indoor Scenes