mlsanityData to 7 Oct 2026

DITTO-NeRF: Diffusion-based Iterative Text To Omni-directional 3D Model

arXiv 2304.02827 ↗First posted 6 Apr 2023cs.CV cs.AIcited by 43

Not on the map. Why?

Cited by 21 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The increasing demand for high-quality 3D content creation has motivated the development of automated methods for creating 3D object models from a single image and/or from a text prompt. However, the reconstructed 3D objects using state-of-the-art image-to-3D methods still exhibit low correspondence to the given image and low multi-view consistency. Recent state-of-the-art text-to-3D methods are also limited, yielding 3D samples with low diversity per prompt with long synthesis time. To address these challenges, we propose DITTO-NeRF, a novel pipeline to generate a high-quality 3D NeRF model from a text prompt or a single image. Our DITTO-NeRF consists of constructing high-quality partial 3D object for limited in-boundary (IB) angles using the given or text-generated 2D image from the frontal view and then iteratively reconstructing the remaining 3D NeRF using inpainting latent diffusion model. We propose progressive 3D object reconstruction schemes in terms of scales (low to high resolution), angles (IB angles initially to outer-boundary (OB) later), and masks (object to background boundary) in our DITTO-NeRF so that high-quality information on IB can be propagated into OB. Our DITTO-NeRF outperforms state-of-the-art methods in terms of fidelity and diversity qualitatively and quantitatively with much faster training times than prior arts on image/text-to-3D such as DreamFusion, and NeuralLift-360.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 55 of its 70 references that are in the map. First posted on arXiv 6 Apr 2023.

Builds on

55 of 70 references.

References · 55

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2022Generative Adversarial Networkscited by 5,603
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  11. 2013Auto-Encoding Variational Bayescited by 3,980
  12. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
38 more
  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  2. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  3. 2016Density estimation using Real NVPcited by 2,029
  4. 2021Vision Transformers for Dense Predictioncited by 1,789
  5. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  6. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  7. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  8. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  9. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  10. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  11. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  12. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  13. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  14. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  15. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  16. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  17. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  18. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  19. 2018Open3D: A Modern Library for 3D Data Processingcited by 717
  20. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  21. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  22. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  23. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  24. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  25. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  26. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  27. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  28. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  29. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  30. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  31. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 487
  32. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  33. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  34. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  35. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
  36. 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
  37. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Viewscited by 148
  38. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 18; 2024: 14; 2025: 10; 2026: 1.

43 papers.

Sort:

By area:

  1. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  2. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  3. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  4. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  5. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 151
  6. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  7. 2024GES: Generalized Exponential Splatting for Efficient Radiance Field Renderingcited by 92
  8. 2023EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusioncited by 72
  9. 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
  10. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  11. 2023T3Bench: Benchmarking Current Progress in Text-to-3D Generationcited by 56
  12. 2023DirecT2V: Large Language Models are Frame-Level Directors for Zero-Shot Text-to-Video Generationcited by 47
31 more
  1. 2023AvatarBooth: High-Quality and Customizable 3D Human Avatar Generationcited by 37
  2. 2023Disentangled Clothed Avatar Generation from Text Descriptionscited by 33
  3. 2023Control4D: Efficient 4D Portrait Editing with Textcited by 32
  4. 2023Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repaintingcited by 25
  5. 2023Efficient-NeRF2NeRF: Streamlining Text-Driven 3D Editing with Multiview Correspondence-Enhanced Diffusion Modelscited by 16
  6. 2024DreamCar: Leveraging Car-specific Prior for in-the-wild 3D Car Reconstructioncited by 15
  7. 2024Envision3D: One Image to 3D with Anchor Views Interpolationcited by 12
  8. 2024TrackNeRF: Bundle Adjusting NeRF from Sparse and Noisy Views via Feature Trackscited by 12
  9. 2024GST: Precise 3D Human Body from a Single Image with Gaussian Splatting Transformerscited by 11
  10. 2024Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Reviewcited by 9
  11. 2025Wonder3D++: Cross-domain Diffusion for High-fidelity 3D Generation from a Single Imagecited by 8
  12. 2024Direct and Explicit 3D Generation from a Single Imagecited by 6
  13. 2023Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representationscited by 5
  14. 2023High-Quality 3D Creation from A Single Image Using Subject-Specific Knowledge Priorcited by 5
  15. 2024X-Diffusion: Generating Detailed 3D MRI Volumes From a Single Image Using Cross-Sectional Diffusion Modelscited by 5
  16. 20234D-Editor: Interactive Object-level Editing in Dynamic Neural Radiance Fields via Semantic Distillationcited by 4
  17. 2025Skrr: Skip and Re-use Text Encoder Layers for Memory Efficient Text-to-Image Generationcited by 4
  18. 2025DMS:Diffusion-Based Multi-Baseline Stereo Generation for Improving Self-Supervised Depth Estimationcited by 3
  19. 2025Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusioncited by 3
  20. 2025Few-step Flow for 3D Generation via Marginal-Data Transport Distillationcited by 2
  21. 2023Detailed Human-Centric Text Description-Driven Large Scene Synthesiscited by 1
  22. 2023Text-Image Conditioned Diffusion for Consistent Text-to-3D Generationcited by 1
  23. 2025CDI3D: Cross-guided Dense-view Interpolation for 3D Reconstructioncited by 1
  24. 2025Efficient Personalization of Quantized Diffusion Model without Backpropagationcited by 1
  25. 2025NeuroVoxel-LM: Language-Aligned 3D Perception via Dynamic Voxelization and Meta-Embeddingcited by 1
  26. 2025MVRoom: Controllable 3D Indoor Scene Generation with Multi-View Diffusion Modelscited by 1
  27. 2024Exploring 3D-aware Latent Spaces for Efficiently Learning Numerous Scenescited by 0
  28. 2024Fused-Planes: Why Train a Thousand Tri-Planes When You Can Share?cited by 0
  29. 2024Fixing the Perspective: A Critical Examination of Zero-1-to-3cited by 0
  30. 2025Scaling LLaNA: Advancing NeRF-Language Understanding Through Large-Scale Trainingcited by 0
  31. 2026Training-free, Perceptually Consistent Low-Resolution Previews with High-Resolution Image for Efficient Workflows of Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model
  2. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation
  3. 2024CorrespondentDream: Enhancing 3D Fidelity of Text-to-3D using Cross-View Correspondences
  4. 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields
  5. 2022NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priors
  6. 2023ED-NeRF: Efficient Text-Guided Editing of 3D Scene with Latent Space NeRF
  7. 2023PI3D: Efficient Text-to-3D Generation with Pseudo-Image Diffusion
  8. 2023IT3D: Improved Text-to-3D Generation with Explicit View Synthesis