# CRAFT: Designing Creative and Functional 3D Objects

arXiv: https://arxiv.org/abs/2412.03889 · HTML: https://mlsanity.com/paper/2412.03889/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 62 references that are in the map.
- First posted on arXiv 5 Dec 2024.

## Builds on (most cited first, 30 of 43)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [Point-E: A System for Generating 3D Point Clouds from Complex Prompts](https://mlsanity.com/paper/2212.08751/) (2022)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [Zero-Shot Text-Guided Object Generation with Dream Fields](https://mlsanity.com/paper/2112.01455/) (2021)
- [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors](https://mlsanity.com/paper/2306.17843/) (2023)
- [Shap-E: Generating Conditional 3D Implicit Functions](https://mlsanity.com/paper/2305.02463/) (2023)
- [Text2Mesh: Text-Driven Neural Stylization for Meshes](https://mlsanity.com/paper/2112.03221/) (2021)
- [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023)
- [CLIP-Mesh: Generating textured meshes from text using pretrained image-text models](https://mlsanity.com/paper/2203.13333/) (2022)
- [Text2Tex: Text-driven Texture Synthesis via Diffusion Models](https://mlsanity.com/paper/2303.11396/) (2023)
- [MeshDiffusion: Score-based Generative 3D Mesh Modeling](https://mlsanity.com/paper/2303.08133/) (2023)
- [Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings](https://mlsanity.com/paper/1803.08495/) (2018)
- [TextMesh: Generation of Realistic 3D Meshes From Text Prompts](https://mlsanity.com/paper/2304.12439/) (2023)
- [Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models](https://mlsanity.com/paper/2312.13913/) (2023)
- [HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance](https://mlsanity.com/paper/2305.18766/) (2023)
- [Neural Cages for Detail-Preserving 3D Deformations](https://mlsanity.com/paper/1912.06395/) (2019)
- [Art and the science of generative AI: A deeper dive](https://mlsanity.com/paper/2306.04141/) (2023)
- [TextDeformer: Geometry Manipulation using Text Guidance](https://mlsanity.com/paper/2304.13348/) (2023)
- [MIME: Human-Aware 3D Scene Generation](https://mlsanity.com/paper/2212.04360/) (2022)
- [Learning a Shared Shape Space for Multimodal Garment Design](https://mlsanity.com/paper/1806.11335/) (2018)
- [TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion](https://mlsanity.com/paper/2401.09416/) (2024)

## Built on by (most cited first, 2 of 2)

- [Deep Sketch-Based 3D Modeling: A Survey](https://mlsanity.com/paper/2603.03287/) (2026)
- [MeshOn: Intersection-Free Mesh-to-Mesh Composition](https://mlsanity.com/paper/2604.08799/) (2026)

## Abstract (arXiv)

For designing a wide range of everyday objects, the design process should be aware of both the human body and the underlying semantics of the design specification. However, these two objectives present significant challenges to the current AI-based designing tools. In this work, we present a method to synthesize body-aware 3D objects from a base mesh given an input body geometry and either text or image as guidance. The generated objects can be simulated on virtual characters, or fabricated for real-world use. We propose to use a mesh deformation procedure that optimizes for both semantic alignment as well as contact and penetration losses. Using our method, users can generate both virtual or real-world objects from text, image, or sketch, without the need for manual artist intervention. We present both qualitative and quantitative results on various object categories, demonstrating the effectiveness of our approach.