mlsanityData to 14 Oct 2026

LDM3D-VR: Latent Diffusion Model for 3D VR

arXiv 2311.03226 ↗First posted 6 Nov 2023cs.CV cs.AIcited by 8

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Latent diffusion models have proven to be state-of-the-art in the creation and manipulation of visual outputs. However, as far as we know, the generation of depth maps jointly with RGB is still limited. We introduce LDM3D-VR, a suite of diffusion models targeting virtual reality development that includes LDM3D-pano and LDM3D-SR. These models enable the generation of panoramic RGBD based on textual prompts and the upscaling of low-resolution inputs to high-resolution RGBD, respectively. Our models are fine-tuned from existing pretrained models on datasets containing panoramic/high-resolution RGB images, depth maps and captions. Both models are evaluated in comparison to existing related methods.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 33 of its 46 references that are in the map. First posted on arXiv 6 Nov 2023.

Builds on

33 of 46 references.

References · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  10. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  11. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
  12. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
21 more
  1. 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,552
  2. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  3. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,264
  4. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,254
  5. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,224
  6. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
  7. 2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 903
  8. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 866
  9. 2018Residual Dense Network for Image Super-Resolutioncited by 799
  10. 2022Activating More Pixels in Image Super-Resolution Transformercited by 438
  11. 2015Deeply-Recursive Convolutional Network for Image Super-Resolutioncited by 317
  12. 2020Single Image Super-Resolution via a Holistic Attention Networkcited by 265
  13. 2023Implicit Diffusion Models for Continuous Super-Resolutioncited by 218
  14. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 212
  15. 2023MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimationcited by 174
  16. 2021Transformer for Single Image Super-Resolutioncited by 150
  17. 2021Discrete Cosine Transform Network for Guided Depth Map Super-Resolutioncited by 71
  18. 2021InfinityGAN: Towards Infinite-Pixel Image Synthesiscited by 61
  19. 2023ResDiff: Combining CNN and Diffusion Model for Image Super-Resolutioncited by 53
  20. 2022Text2Light: Zero-Shot Text-Driven HDR Panorama Generationcited by 24
  21. 2021Improving 360 Monocular Depth Estimation via Non-local Dense Prediction Transformer and Joint Supervised and Self-supervised Learningcited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 1.

8 papers.

  1. 2024LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generationcited by 53
  2. 2025DreamCube: 3D Panorama Generation via Multi-plane Synchronizationcited by 12
  3. 2025Vid-CamEdit: Video Camera Trajectory Editing with Generative Rendering from Estimated Geometrycited by 7
  4. 2024EdgeRelight360: Text-Conditioned 360-Degree HDR Image Generation for Real-Time On-Device Video Portrait Relightingcited by 5
  5. 2025TPDiff: Temporal Pyramid Video Diffusion Modelcited by 4
  6. 2024SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blendingcited by 3
  7. 2025DreamAnywhere: Object-Centric Panoramic 3D Scene Generationcited by 2
  8. 2026VS-Splat: Voxel-Selective feed-forward Gaussian Splatting for end-to-end 3D object reconstruction from sparse-viewscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023LDM3D: Latent Diffusion Model for 3D
  2. 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models
  3. 2025ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models
  4. 2025HoloTime: Taming Video Diffusion Models for Panoramic 4D Scene Generation
  5. 2024Diffusion Models in 3D Vision: A Survey
  6. 2025SViM3D: Stable Video Material Diffusion for Single Image 3D Generation
  7. 2024V3D: Video Diffusion Models are Effective 3D Generators
  8. 2026Dual Diffusion Models for Multi-modal Guided 3D Avatar Generation