mlsanityData to 7 Oct 2026

CLIP-Mesh: Generating textured meshes from text using pretrained image-text models

arXiv 2203.13333 ↗First posted 24 Mar 2022cs.CV cs.GR cs.LGcited by 281

Not on the map. Why?

Cited by 105 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a technique for zero-shot generation of a 3D model using only a target text prompt. Without any 3D supervision our method deforms the control shape of a limit subdivided surface along with its texture map and normal map to obtain a 3D asset that corresponds to the input text prompt and can be easily deployed into games or modeling applications. We rely only on a pre-trained CLIP model that compares the input text prompt with differentiably rendered images of our 3D model. While previous works have focused on stylization or required training of generative models we perform optimization on mesh parameters directly to generate shape, texture or both. To constrain the optimization to produce plausible meshes and textures we introduce a number of techniques using image augmentations and the use of a pretrained prior that generates CLIP image embeddings given a text embedding.

Key facts. Cited by 281 papers in the map (data through 7 Oct 2026). Builds on 17 of its 26 references that are in the map. First posted on arXiv 24 Mar 2022.

Builds on

17 of 26 references.

References · 17

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  6. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  7. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  8. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  9. 2021ImageNet-21K Pretraining for the Massescited by 607
  10. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  11. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  12. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
5 more
  1. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  2. 2018Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddingscited by 169
  3. 2021ClipMatrix: Text-controlled Creation of 3D Textured Meshescited by 40
  4. 2021Appearance-Driven Automatic 3D Model Simplificationcited by 24
  5. 2019Generation High resolution 3D model from natural language by Generative Adversarial Networkcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 17; 2023: 119; 2024: 102; 2025: 31; 2026: 12.

281 papers.

Sort:

By area:

  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  2. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  3. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  4. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  5. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  6. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  7. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  8. 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
  9. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  10. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  11. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  12. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
38 more
  1. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  2. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  3. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  4. 2023Text-to-3D using Gaussian Splattingcited by 299
  5. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  6. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  7. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  8. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Modelscited by 234
  9. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  10. 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 192
  11. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
  12. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  13. 2023DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Modelscited by 154
  14. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 154
  15. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Modelscited by 151
  16. 2023Alpha-CLIP: A CLIP Model Focusing on Wherever You Wantcited by 150
  17. 2023TextMesh: Generation of Realistic 3D Meshes From Text Promptscited by 135
  18. 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fieldscited by 122
  19. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Modelscited by 122
  20. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  21. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidancecited by 109
  22. 2023TADA! Text to Animatable Digital Avatarscited by 109
  23. 2023DreamHuman: Animatable 3D Avatars from Textcited by 102
  24. 2023GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphscited by 96
  25. 2023TextDeformer: Geometry Manipulation using Text Guidancecited by 94
  26. 20233D-GPT: Procedural 3D Modeling with Large Language Modelscited by 91
  27. 2023HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splattingcited by 89
  28. 2023HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generationcited by 88
  29. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 88
  30. 2023ATT3D: Amortized Text-to-3D Object Synthesiscited by 87
  31. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Modelscited by 86
  32. 2023Text-Guided Texturing by Synchronized Multi-View Diffusioncited by 86
  33. 2024EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generationcited by 84
  34. 2022TANGO: Text-driven Photorealistic and Robust 3D Stylization via Lighting Decompositioncited by 83
  35. 2023DreamWaltz: Make a Scene with Complex 3D Animatable Avatarscited by 82
  36. 2023DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generationcited by 81
  37. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
  38. 2023Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenescited by 75

50 most cited of 281 All 281 (JSON)