mlsanityData to 7 Oct 2026

Text-guided Controllable Mesh Refinement for Interactive 3D Modeling

arXiv 2406.01592 ↗First posted 3 Jun 2024cs.CV cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

We propose a novel technique for adding geometric details to an input coarse 3D mesh guided by a text prompt. Our method is composed of three stages. First, we generate a single-view RGB image conditioned on the input coarse geometry and the input text prompt. This single-view image generation step allows the user to pre-visualize the result and offers stronger conditioning for subsequent multi-view generation. Second, we use our novel multi-view normal generation architecture to jointly generate six different views of the normal images. The joint view generation reduces inconsistencies and leads to sharper details. Third, we optimize our mesh with respect to all views and generate a fine, detailed geometry as output. The resulting method produces an output within seconds and offers explicit user control over the coarse structure, pose, and desired details of the resulting 3D mesh.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 36 of its 52 references that are in the map. First posted on arXiv 3 Jun 2024.

Builds on

36 of 52 references.

References · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Segment Anythingcited by 8,158
  3. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  4. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  6. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  7. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  8. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  9. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  10. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  11. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  12. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
24 more
  1. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  2. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  3. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  4. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  5. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  6. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
  7. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  8. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  9. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  10. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  11. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  12. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  13. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  14. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  15. 2023TextMesh: Generation of Realistic 3D Meshes From Text Promptscited by 135
  16. 2023TextDeformer: Geometry Manipulation using Text Guidancecited by 94
  17. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  18. 2020Deep Geometric Texture Synthesiscited by 51
  19. 2023Direct2.5: Diverse Text-to-3D Generation via Multi-view 2.5D Diffusioncited by 50
  20. 2020Neural Subdivisioncited by 49
  21. 2020DECOR-GAN: 3D Shape Detailization by Conditional Refinementcited by 45
  22. 2023Text-Guided Generation and Editing of Compositional 3D Avatarscited by 39
  23. 2023ShaDDR: Interactive Example-Based Geometry and Texture Generation via 3D Shape Detailization and Differentiable Renderingcited by 13
  24. 2023Neural Progressive Meshescited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025Geometry in Style: 3D Stylization via Surface Normal Deformationcited by 14
  2. 2025CMD: Controllable Multiview Diffusion for 3D Editing and Progressive Generationcited by 14
  3. 2025ART-DECO: Arbitrary Text Guidance for 3D Detailizer Constructioncited by 6
  4. 2025SplatPainter: Interactive Authoring of 3D Gaussians from 2D Edits via Test-Time Trainingcited by 5
  5. 2025GenVDM: Generating Vector Displacement Maps From a Single Imagecited by 1
  6. 2026Copy-Trasform-Paste: Zero-Shot Object-Object Alignment Guided by Vision-Language and Geometric Constraintscited by 1
  7. 2026MeshOn: Intersection-Free Mesh-to-Mesh Compositioncited by 0