mlsanityData to 7 Oct 2026

Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models

arXiv 2303.11989 ↗First posted 21 Mar 2023cs.CVcited by 245

Not on the map. Why?

Cited by 67 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present Text2Room, a method for generating room-scale textured 3D meshes from a given text prompt as input. To this end, we leverage pre-trained 2D text-to-image models to synthesize a sequence of images from different poses. In order to lift these outputs into a consistent 3D scene representation, we combine monocular depth estimation with a text-conditioned inpainting model. The core idea of our approach is a tailored viewpoint selection such that the content of each image can be fused into a seamless, textured 3D mesh. More specifically, we propose a continuous alignment strategy that iteratively fuses scene frames with the existing geometry to create a seamless mesh. Unlike existing works that focus on generating single objects or zoom-out trajectories from text, our method generates complete 3D scenes with multiple objects and explicit 3D geometry. We evaluate our approach using qualitative and quantitative metrics, demonstrating it as the first method to generate room-scale 3D geometry with compelling textures from only text as input.

Key facts. Cited by 245 papers in the map (data through 7 Oct 2026). Builds on 81 of its 98 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

81 of 98 references.

References · 81

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2022Classifier-Free Diffusion Guidancecited by 5,717
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
38 more
  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  9. 2016Improved Techniques for Training GANscited by 2,938
  10. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  12. 2022Video Diffusion Modelscited by 2,280
  13. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  14. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  15. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  16. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  17. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  18. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  19. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  20. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  21. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  22. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  23. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  24. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  25. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  26. 2021Score-based Generative Modeling in Latent Spacecited by 634
  27. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  28. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  29. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  30. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  31. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  32. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  33. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  34. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  35. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  36. 2022Novel View Synthesis with Diffusion Modelscited by 299
  37. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  38. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 40; 2024: 87; 2025: 72; 2026: 46.

245 papers.

Sort:

By area:

  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  2. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 425
  3. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  4. 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 278
  5. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
  6. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  7. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  8. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  9. 2023MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimationcited by 173
  10. 2023WonderJourney: Going from Anywhere to Everywherecited by 157
  11. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  12. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 151
38 more
  1. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  2. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  3. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  4. 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fieldscited by 122
  5. 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Controlcited by 117
  6. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 116
  7. 2023Animate124: Animating One Image to 4D Dynamic Scenecited by 113
  8. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
  9. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
  10. 2024InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Priorcited by 110
  11. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
  12. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
  13. 2024GES: Generalized Exponential Splatting for Efficient Radiance Field Renderingcited by 92
  14. 2024I-Design: Personalized LLM Interior Designercited by 87
  15. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  16. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
  17. 2024DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splattingcited by 81
  18. 2024DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Samplingcited by 72
  19. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
  20. 2024MultiDiff: Consistent Novel View Synthesis from a Single Imagecited by 67
  21. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65
  22. 2025Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generationcited by 65
  23. 2023The Chosen One: Consistent Characters in Text-to-Image Diffusion Modelscited by 62
  24. 2023NeRFiller: Completing Scenes via Generative 3D Inpaintingcited by 62
  25. 2024Advances in 3D Generation: A Surveycited by 62
  26. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  27. 2023Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraintscited by 59
  28. 2024ViewDiff: 3D-Consistent Image Generation with Text-to-Image Modelscited by 57
  29. 2023ControlRoom3D: Room Generation using Semantic Proxy Roomscited by 54
  30. 2023SceneWiz3D: Towards Text-guided 3D Scene Compositioncited by 54
  31. 2023RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texturecited by 53
  32. 2024LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generationcited by 53
  33. 2024LT3SD: Latent Trees for 3D Scene Diffusioncited by 53
  34. 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Textcited by 50
  35. 2024Video-Guided Foley Sound Generation with Multimodal Controlscited by 49
  36. 2024Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understandingcited by 46
  37. 2023SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priorscited by 43
  38. 2023Text2Immersion: Generative Immersive Scene with 3D Gaussianscited by 43

50 most cited of 245 All 245 (JSON)