mlsanityData to 14 Oct 2026

PanoDreamer: Optimization-Based Single Image to 360 3D Scene With Diffusion

arXiv 2412.04827 ↗First posted 6 Dec 2024cs.CV cs.GRcited by 2

Not on the map. Why?

Abstract (arXiv)

In this paper, we present PanoDreamer, a novel method for producing a coherent 360deg 3D scene from a single input image. Unlike existing methods that generate the scene sequentially, we frame the problem as single-image panorama and depth estimation. Once the coherent panoramic image and its corresponding depth are obtained, the scene can be reconstructed by inpainting the small occluded regions and projecting them into 3D space. Our key contribution is formulating single-image panorama and depth estimation as two optimization tasks and introducing alternating minimization strategies to effectively solve their objectives. We demonstrate that our approach outperforms existing techniques in single-image 360deg 3D scene reconstruction in terms of consistency and overall quality.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 42 of its 48 references that are in the map. First posted on arXiv 6 Dec 2024.

Builds on

42 of 48 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  6. 2024Depth Anything V2cited by 1,422
  7. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  8. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
  9. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  10. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
  11. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 406
  12. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
30 more
  1. 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 279
  2. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  3. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  4. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 231
  5. 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
  6. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 202
  7. 2016View Synthesis by Appearance Flowcited by 172
  8. 2023WonderJourney: Going from Anywhere to Everywherecited by 158
  9. 20193D Ken Burns Effect from a Single Imagecited by 128
  10. 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fieldscited by 123
  11. 2023SyncDiffusion: Coherent Montage via Synchronized Joint Diffusionscited by 110
  12. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
  13. 2023DiffCollage: Parallel Generation of Large Content with Diffusion Modelscited by 103
  14. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  15. 2024CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussianscited by 62
  16. 2017Learning to Synthesize a 4D RGBD Light Field from a Single Imagecited by 60
  17. 2023PanoDiffusion: 360-degree Panorama Outpainting via Diffusioncited by 51
  18. 2021SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpaintingcited by 48
  19. 2023Customizing 360-Degree Panoramas through Text-to-Image Diffusion Modelscited by 47
  20. 2020One Shot 3D Photographycited by 45
  21. 2023Text2Immersion: Generative Immersive Scene with 3D Gaussianscited by 43
  22. 2024DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusioncited by 38
  23. 2024Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpaintingcited by 24
  24. 2024HoloDreamer: Holistic 3D Panoramic World Generation from Text Descriptionscited by 24
  25. 2024SceneDreamer360: Text-Driven 3D-Consistent Scene Generation with Panoramic Gaussian Splattingcited by 24
  26. 2024SpotDiffusion: A Fast Approach For Seamless Panorama Generation Over Timecited by 12
  27. 2023SinMPI: Novel View Synthesis from a Single Image with Expanded Multiplane Imagescited by 11
  28. 2024L-MAGIC: Language Model Assisted Generation of Images with Coherencecited by 6
  29. 2023ReShader: View-Dependent Highlights for Single Image View-Synthesiscited by 3
  30. 2023Novel View Synthesis with View-Dependent Effects from a Single Imagecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025PanoLora: Bridging Perspective and Panoramic Video Generation with LoRA Adaptationcited by 3
  2. 2026UniSHARP: Universal Sharp Monocular View Synthesiscited by 2