# 3D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillation

arXiv: https://arxiv.org/abs/2311.09571 · HTML: https://mlsanity.com/paper/2311.09571/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 69 references that are in the map.
- First posted on arXiv 16 Nov 2023.

## Builds on (most cited first, 30 of 59)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022)
- [LERF: Language Embedded Radiance Fields](https://mlsanity.com/paper/2303.09553/) (2023)
- [Zero-Shot Text-Guided Object Generation with Dream Fields](https://mlsanity.com/paper/2112.01455/) (2021)
- [Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures](https://mlsanity.com/paper/2211.07600/) (2022)
- [Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions](https://mlsanity.com/paper/2303.12789/) (2023)
- [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022)
- [Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors](https://mlsanity.com/paper/2306.17843/) (2023)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Text2Mesh: Text-Driven Neural Stylization for Meshes](https://mlsanity.com/paper/2112.03221/) (2021)
- [Text2LIVE: Text-Driven Layered Image and Video Editing](https://mlsanity.com/paper/2204.02491/) (2022)
- [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023)
- [CLIP-Mesh: Generating textured meshes from text using pretrained image-text models](https://mlsanity.com/paper/2203.13333/) (2022)
- [Text2Tex: Text-driven Texture Synthesis via Diffusion Models](https://mlsanity.com/paper/2303.11396/) (2023)

## Built on by (most cited first, 30 of 35)

- [CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model](https://mlsanity.com/paper/2403.05034/) (2024)
- [TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts](https://mlsanity.com/paper/2401.14828/) (2024)
- [Instant3dit: Multiview Inpainting for Fast Editing of 3D Objects](https://mlsanity.com/paper/2412.00518/) (2024)
- [LLMs Meet Multimodal Generation and Editing: A Survey](https://mlsanity.com/paper/2405.19334/) (2024)
- [Efficient Diffusion Models: A Survey](https://mlsanity.com/paper/2502.06805/) (2025)
- [PrEditor3D: Fast and Precise 3D Shape Editing](https://mlsanity.com/paper/2412.06592/) (2024)
- [Videoshop: Localized Semantic Video Editing with Noise-Extrapolated Diffusion Inversion](https://mlsanity.com/paper/2403.14617/) (2024)
- [Garment3DGen: 3D Garment Stylization and Texture Generation](https://mlsanity.com/paper/2403.18816/) (2024)
- [LL3M: Large Language 3D Modelers](https://mlsanity.com/paper/2508.08228/) (2025)
- [MeshUp: Multi-Target Mesh Deformation via Blended Score Distillation](https://mlsanity.com/paper/2408.14899/) (2024)
- [iSeg: Interactive 3D Segmentation via Interactive Attention](https://mlsanity.com/paper/2404.03219/) (2024)
- [Geometry in Style: 3D Stylization via Surface Normal Deformation](https://mlsanity.com/paper/2503.23241/) (2025)
- [Diffusion Models in 3D Vision: A Survey](https://mlsanity.com/paper/2410.04738/) (2024)
- [Tinker: Diffusion's Gift to 3D--Multi-View Consistent Editing From Sparse Inputs without Per-Scene Optimization](https://mlsanity.com/paper/2508.14811/) (2025)
- [SplatPainter: Interactive Authoring of 3D Gaussians from 2D Edits via Test-Time Training](https://mlsanity.com/paper/2512.05354/) (2025)
- [ASIA: Adaptive 3D Segmentation using Few Image Annotations](https://mlsanity.com/paper/2509.24288/) (2025)
- [ScanEdit: Hierarchically-Guided Functional 3D Scan Editing](https://mlsanity.com/paper/2504.15049/) (2025)
- [VecSet-Edit: Unleashing Pre-trained LRM for Mesh Editing from Single Image](https://mlsanity.com/paper/2602.04349/) (2026)
- [Artiverse: A Diverse and Physically Grounded Dataset for Articulated Objects](https://mlsanity.com/paper/2605.24403/) (2026)
- [Im2SurfTex: Surface Texture Generation via Neural Backprojection of Multi-View Images](https://mlsanity.com/paper/2502.14006/) (2025)
- [3D PixBrush: Image-Guided Local Texture Synthesis](https://mlsanity.com/paper/2507.03731/) (2025)
- [Voxify3D: Pixel Art Meets Volumetric Rendering](https://mlsanity.com/paper/2512.07834/) (2025)
- [Vinedresser3D: Agentic Text-guided 3D Editing](https://mlsanity.com/paper/2602.19542/) (2026)
- [Consistency^2: Consistent and Fast 3D Painting with Latent Consistency Models](https://mlsanity.com/paper/2406.11202/) (2024)
- [CRAFT: Designing Creative and Functional 3D Objects](https://mlsanity.com/paper/2412.03889/) (2024)
- [TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Masking](https://mlsanity.com/paper/2604.01207/) (2026)
- [Text2VDM: Text to Vector Displacement Maps for Expressive and Interactive 3D Sculpting](https://mlsanity.com/paper/2502.20045/) (2025)
- [Towards Generalized and Training-Free Text-Guided Semantic Manipulation](https://mlsanity.com/paper/2504.17269/) (2025)
- [ScribbleSense: Generative Scribble-Based Texture Editing with Intent Prediction](https://mlsanity.com/paper/2601.22455/) (2026)
- [Best Segmentation Buddies for Image-Shape Correspondence](https://mlsanity.com/paper/2605.18193/) (2026)

## Abstract (arXiv)

In this work we develop 3D Paintbrush, a technique for automatically texturing local semantic regions on meshes via text descriptions. Our method is designed to operate directly on meshes, producing texture maps which seamlessly integrate into standard graphics pipelines. We opt to simultaneously produce a localization map (to specify the edit region) and a texture map which conforms to it. This synergistic approach improves the quality of both the localization and the stylization. To enhance the details and resolution of the textured area, we leverage multiple stages of a cascaded diffusion model to supervise our local editing technique with generative priors learned from images at different resolutions. Our technique, referred to as Cascaded Score Distillation (CSD), simultaneously distills scores at multiple resolutions in a cascaded fashion, enabling control over both the granularity and global understanding of the supervision. We demonstrate the effectiveness of 3D Paintbrush to locally texture a variety of shapes within different semantic regions. Project page: https://threedle.github.io/3d-paintbrush