mlsanityData to 7 Oct 2026

PixelSynth: Generating a 3D-Consistent Experience from a Single Image

arXiv 2108.05892 ↗First posted 12 Aug 2021cs.CVcited by 82

Not on the map. Why?

Cited by 28 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advancements in differentiable rendering and 3D reasoning have driven exciting results in novel view synthesis from a single image. Despite realistic results, methods are limited to relatively small view change. In order to synthesize immersive scenes, models must also be able to extrapolate. We present an approach that fuses 3D reasoning with autoregressive modeling to outpaint large view changes in a 3D-consistent manner, enabling scene synthesis. We demonstrate considerable improvement in single image large-angle view synthesis results compared to a variety of methods and possible variants across simulated and real datasets. In addition, we show increased 3D consistency compared to alternative accumulation methods. Project website: https://crockwell.github.io/pixelsynth/

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). Builds on 58 of its 97 references that are in the map. First posted on arXiv 12 Aug 2021.

Builds on

58 of 97 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  12. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
38 more
  1. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  2. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  3. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  4. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  5. 2020Neural Sparse Voxel Fieldscited by 959
  6. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  7. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  8. 2016Pixel Recurrent Neural Networkscited by 814
  9. 2018Self-Attention Generative Adversarial Networkscited by 809
  10. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  11. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  12. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  13. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  14. 2017Neural 3D Mesh Renderercited by 437
  15. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  16. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  17. 2019Neural Point-Based Graphicscited by 326
  18. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  19. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  20. 2020Free View Synthesiscited by 231
  21. 2017Photographic Image Synthesis with Cascaded Refinement Networkscited by 223
  22. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 206
  23. 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
  24. 2019Cross-task weakly supervised learning from instructional videoscited by 199
  25. 2016View Synthesis by Appearance Flowcited by 172
  26. 2020Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Cameracited by 166
  27. 2017PixelSNAIL: An Improved Autoregressive Generative Modelcited by 137
  28. 2019Neural Rerendering in the Wildcited by 121
  29. 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 99
  30. 2016High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesiscited by 89
  31. 2018Layer-structured 3D Scene Inference via View Synthesiscited by 82
  32. 2020Articulation-aware Canonical Surface Mappingcited by 80
  33. 2020Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Imagecited by 77
  34. 2018Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscalingcited by 74
  35. 2017Parallel Multiscale Autoregressive Density Estimationcited by 71
  36. 2019Boundless: Generative Adversarial Networks for Image Extensioncited by 62
  37. 2014Designing Deep Networks for Surface Normal Estimationcited by 59
  38. 2015Multi-view 3D Models from Single Images with a Convolutional Networkcited by 57

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 14; 2023: 31; 2024: 18; 2025: 11; 2026: 4.

82 papers.

Sort:

By area:

  1. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  2. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  3. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  4. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  5. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  6. 2021GeoNeRF: Generalizing NeRF with Geometry Priorscited by 195
  7. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179
  8. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  9. 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fieldscited by 122
  10. 2023Consistent View Synthesis with Pose-Guided Diffusion Modelscited by 117
  11. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
  12. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
38 more
  1. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  2. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
  3. 2021NeRF-SR: High-Quality Neural Radiance Fields using Supersamplingcited by 95
  4. 2023Monocular Depth Estimation using Diffusion Modelscited by 87
  5. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
  6. 2023Long-Term Photometric Consistent Novel View Synthesis with Diffusion Modelscited by 76
  7. 2024GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warpingcited by 72
  8. 2024MultiDiff: Consistent Novel View Synthesis from a Single Imagecited by 67
  9. 2022Removing Objects From Neural Radiance Fieldscited by 66
  10. 2023NeRFiller: Completing Scenes via Generative 3D Inpaintingcited by 62
  11. 2024Advances in 3D Generation: A Surveycited by 62
  12. 2022Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Imagescited by 59
  13. 2022Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Imagecited by 55
  14. 2023Cross-Ray Neural Radiance Fields for Novel-view Synthesis from Unconstrained Image Collectionscited by 54
  15. 2022DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Modelscited by 51
  16. 20213D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Imagecited by 45
  17. 2023Persistent Nature: A Generative Model of Unbounded 3D Worldscited by 44
  18. 2023Text2Immersion: Generative Immersive Scene with 3D Gaussianscited by 43
  19. 2023Neural Volumetric Memory for Visual Locomotion Controlcited by 40
  20. 2022FWD: Real-time Novel View Synthesis with Forward Warping and Depthcited by 39
  21. 2024VistaDream: Sampling multiview consistent images for single-view scene reconstructioncited by 38
  22. 2021Video Autoencoder: self-supervised disentanglement of static 3D structure and motioncited by 34
  23. 2022Simple and Effective Synthesis of Indoor 3D Scenescited by 33
  24. 2025SynCity: Training-Free Generation of 3D Worldscited by 30
  25. 20243D-SceneDreamer: Text-Driven 3D-Consistent Scene Generationcited by 29
  26. 2024StereoDiffusion: Training-Free Stereo Image Generation Using Latent Diffusion Modelscited by 24
  27. 20253D Scene Generation: A Surveycited by 24
  28. 2024Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpaintingcited by 23
  29. 2023Fast View Synthesis of Casual Videos with Soup-of-Planescited by 22
  30. 2025Positional Encoding Fieldcited by 20
  31. 2023Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Imagecited by 19
  32. 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 18
  33. 2023ConsistentNeRF: Enhancing Neural Radiance Fields with 3D Consistency for Sparse View Synthesiscited by 17
  34. 2022Factorized and Controllable Neural Re-Rendering of Outdoor Scene for Photo Extrapolationcited by 16
  35. 20223D Moments from Near-Duplicate Photoscited by 15
  36. 2023MapNeRF: Incorporating Map Priors into Neural Radiance Fields for Driving View Simulationcited by 15
  37. 2024Denoising Diffusion via Image-Based Renderingcited by 15
  38. 2024ExtraNeRF: Visibility-Aware View Extrapolation of Neural Radiance Fields with Diffusion Modelscited by 12

50 most cited of 82 All 82 (JSON)