mlsanityData to 7 Oct 2026

ClipMatrix: Text-controlled Creation of 3D Textured Meshes

arXiv 2109.12922 ↗First posted 27 Sep 2021cs.LG cs.CV cs.GRcited by 40

Not on the map. Why?

Abstract (arXiv)

If a picture is worth thousand words, a moving 3d shape must be worth a million. We build upon the success of recent generative methods that create images fitting the semantics of a text prompt, and extend it to the controlled generation of 3d objects. We present a novel algorithm for the creation of textured 3d meshes, controlled by text prompts. Our method creates aesthetically pleasing high resolution articulated 3d meshes, and opens new possibilities for automation and AI control of 3d assets. We call it "ClipMatrix" because it leverages CLIP text embeddings to breed new digital 3d creatures, a nod to the Latin meaning of the word "matrix" - "mother". See the online gallery for a full impression of our method's capability.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 5 of its 7 references that are in the map. First posted on arXiv 27 Sep 2021.

Builds on

5 of 7 references.

References · 5

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  3. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  4. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  5. 2020Learning to Transfer Texture from Clothing Images to 3D Humanscited by 58

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 10; 2023: 14; 2024: 10; 2025: 2; 2026: 2.

40 papers.

Sort:

By area:

  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  2. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  3. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  4. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  5. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  6. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  7. 2023Text-To-4D Dynamic Scene Generationcited by 204
  8. 20234D-fy: Text-to-4D Generation Using Hybrid Score Distillation Samplingcited by 184
  9. 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 183
  10. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  11. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  12. 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 117
28 more
  1. 2022TANGO: Text-driven Photorealistic and Robust 3D Stylization via Lighting Decompositioncited by 83
  2. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  3. 2023GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learningcited by 63
  4. 2022ClipFace: Text-guided Editing of Textured 3D Morphable Modelscited by 59
  5. 2022AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatarscited by 55
  6. 2023OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllerscited by 47
  7. 2022Being Comes from Not-being: Open-vocabulary Text-to-Motion Generation with Wordless Trainingcited by 45
  8. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 37
  9. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 36
  10. 2022ISS: Image as Stepping Stone for Text-Guided 3D Shape Generationcited by 27
  11. 2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
  12. 2023Surrogate Modeling of Car Drag Coefficient with Depth and Normal Renderingscited by 15
  13. 2024UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with Authenticity Guided Texturescited by 13
  14. 2022Text-Driven Stylization of Video Objectscited by 12
  15. 2023FaceCLIPNeRF: Text-driven 3D Face Manipulation using Deformable Neural Radiance Fieldscited by 12
  16. 2023Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Imagescited by 12
  17. 2024Multimodal Latent Diffusion Model for Complex Sewing Pattern Generationcited by 12
  18. 2024SimAvatar: Simulation-Ready Avatars with Layered Hair and Clothingcited by 9
  19. 2024Fast Text-to-3D-Aware Face Generation and Manipulation via Direct Cross-modal Mapping and Geometric Regularizationcited by 8
  20. 2023HyperStyle3D: Text-Guided 3D Portrait Stylization via Hypernetworkscited by 7
  21. 2023Text-driven Visual Synthesis with Latent Diffusion Priorcited by 6
  22. 2024Magic3DSketch: Create Colorful 3D Models From Sketch-Based 3D Modeling Guided by Text and Language-Image Pre-Trainingcited by 5
  23. 2023DreamStone: Image as Stepping Stone for Text-Guided 3D Shape Generationcited by 4
  24. 2023Directional Texture Editing for 3D Modelscited by 3
  25. 2024PaintScene4D: Consistent 4D Scene Generation from Text Promptscited by 2
  26. 2024Enhancing Texture Generation with High-Fidelity Using Advanced Texture Priorscited by 0
  27. 2025Disentangling 3D from Large Vision-Language Models for Controlled Portrait Generationcited by 0
  28. 2026Advances in Neural 3D Mesh Texturing: A Surveycited by 0