mlsanityData to 7 Oct 2026

DreamView: Injecting View-specific Text Guidance into Text-to-3D Generation

arXiv 2404.06119 ↗First posted 9 Apr 2024cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Text-to-3D generation, which synthesizes 3D assets according to an overall text description, has significantly progressed. However, a challenge arises when the specific appearances need customizing at designated viewpoints but referring solely to the overall description for generating 3D objects. For instance, ambiguity easily occurs when producing a T-shirt with distinct patterns on its front and back using a single overall text guidance. In this work, we propose DreamView, a text-to-image approach enabling multi-view customization while maintaining overall consistency by adaptively injecting the view-specific and overall text guidance through a collaborative text guidance injection module, which can also be lifted to 3D generation via score distillation sampling. DreamView is trained with large-scale rendered multi-view images and their corresponding view-specific texts to learn to balance the separate content manipulation in each view and the global consistency of the overall object, resulting in a dual achievement of customization and consistency. Consequently, DreamView empowers artists to design 3D objects creatively, fostering the creation of more innovative and diverse 3D assets. Code and model will be released at https://github.com/iSEE-Laboratory/DreamView.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 53 of its 63 references that are in the map. First posted on arXiv 9 Apr 2024.

Builds on

53 of 63 references.

References · 53

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  10. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  11. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  12. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
38 more
  1. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  2. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  3. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  5. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  7. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  8. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  9. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  10. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  11. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  12. 2022On Distillation of Guided Diffusion Modelscited by 721
  13. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  14. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  15. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  16. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  17. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  18. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  19. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  20. 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 285
  21. 2022SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Modelcited by 276
  22. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 262
  23. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
  24. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  25. 2023Text-To-4D Dynamic Scene Generationcited by 204
  26. 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 199
  27. 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 183
  28. 2021GOAL: Generating 4D Whole-Body Motion for Hand-Object Graspingcited by 142
  29. 2023Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyondcited by 139
  30. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3Dcited by 134
  31. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generationcited by 133
  32. 2023DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generationcited by 81
  33. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  34. 2024TAMM: TriAdapter Multi-Modal Learning for 3D Shape Understandingcited by 26
  35. 2022AcroFOD: An Adaptive Method for Cross-domain Few-shot Object Detectioncited by 25
  36. 2021Building-GAN: Graph-Conditioned Architectural Volumetric Design Generationcited by 20
  37. 2023Diversifying Spatial-Temporal Perception for Video Domain Generalizationcited by 19
  38. 2023Sculpting Holistic 3D Representation in Contrastive Language-Image-3D Pre-trainingcited by 17

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 3.

9 papers.

  1. 2024GT23D-Bench: A Comprehensive General Text-to-3D Generation Benchmarkcited by 11
  2. 2026PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual Worldcited by 8
  3. 2024Benchmarking and Learning Multi-Dimensional Quality Evaluator for Text-to-3D Generationcited by 7
  4. 2024SeMv-3D: Towards Concurrency of Semantic and Multi-view Consistency in General Text-to-3D Generationcited by 2
  5. 2025CoherenDream: Boosting Holistic Text Coherence in 3D Generation via Multimodal Large Language Models Feedbackcited by 1
  6. 2025DreamComposer++: Empowering Diffusion Models with Multi-View Conditions for 3D Content Generationcited by 1
  7. 2025DreamLifting: A Plug-in Module Lifting MV Diffusion Models for 3D Asset Generationcited by 0
  8. 2026TIMI: Training-Free Image-to-3D Multi-Instance Generation with Spatial Fidelitycited by 0
  9. 2026Holo-Captioning: Toward the Text Equivalent of 3D Scenescited by 0