mlsanityData to 7 Oct 2026

3D Highlighter: Localizing Regions on 3D Shapes via Text Descriptions

arXiv 2212.11263 ↗First posted 21 Dec 2022cs.GR cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

We present 3D Highlighter, a technique for localizing semantic regions on a mesh using text as input. A key feature of our system is the ability to interpret "out-of-domain" localizations. Our system demonstrates the ability to reason about where to place non-obviously related concepts on an input 3D shape, such as adding clothing to a bare 3D animal model. Our method contextualizes the text description using a neural field and colors the corresponding region of the shape using a probability-weighted blend. Our neural optimization is guided by a pre-trained CLIP encoder, which bypasses the need for any 3D datasets or 3D annotations. Thus, 3D Highlighter is highly flexible, general, and capable of producing localizations on a myriad of input shapes. Our code is publicly available at https://github.com/threedle/3DHighlighter.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 30 of its 50 references that are in the map. First posted on arXiv 21 Dec 2022.

Builds on

30 of 50 references.

References · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  6. 2018On the Spectral Bias of Neural Networkscited by 1,321
  7. 2022Language-driven Semantic Segmentationcited by 711
  8. 2021Neural Fields in Visual Computing and Beyondcited by 539
  9. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  10. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  11. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  12. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
18 more
  1. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  2. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  3. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  4. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  5. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
  6. 2016Thingi10K: A Dataset of 10,000 3D-Printing Modelscited by 182
  7. 2019CvxNet: Learnable Convex Decompositioncited by 173
  8. 2020DiffusionNet: Discretization Agnostic Learning on Surfacescited by 163
  9. 2017Visual Attribute Transfer through Deep Image Analogycited by 123
  10. 2019BAE-NET: Branched Autoencoder for Shape Co-Segmentationcited by 89
  11. 2016SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentationcited by 70
  12. 2018MeshCNN: A Network with an Edgecited by 69
  13. 2022ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Modelcited by 62
  14. 2021Subdivision-Based Mesh Convolution Networkscited by 60
  15. 2019PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentationcited by 44
  16. 2020Primal-Dual Mesh Convolutional Neural Networkscited by 44
  17. 2020MeshWalker: Deep Mesh Understanding by Random Walkscited by 32
  18. 20223D Concept Grounding on Neural Fieldscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 10; 2025: 7; 2026: 7.

32 papers.

Sort:

By area:

  1. 2023SATR: Zero-Shot Semantic Segmentation of 3D Shapescited by 62
  2. 2023Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Featurescited by 55
  3. 2024Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databasescited by 43
  4. 2024Segment Any Meshcited by 37
  5. 20233D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillationcited by 34
  6. 2023Zero-Shot 3D Shape Correspondencecited by 33
  7. 2024Videoshop: Localized Semantic Video Editing with Noise-Extrapolated Diffusion Inversioncited by 27
  8. 2024MeshSegmenter: Zero-Shot Mesh Semantic Segmentation via Texture Synthesiscited by 22
  9. 2024PartSTAD: 2D-to-3D Part Segmentation Task Adaptationcited by 18
  10. 2023Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Featurescited by 17
  11. 20243D Part Segmentation via Geometric Aggregation of 2D Visual Featurescited by 15
  12. 2024iSeg: Interactive 3D Segmentation via Interactive Attentioncited by 14
20 more
  1. 2025Geometry in Style: 3D Stylization via Surface Normal Deformationcited by 14
  2. 2023Neural Semantic Surface Mapscited by 13
  3. 2023Style2Fab: Functionality-Aware Segmentation for Fabricating Personalized 3D Models with Generative AIcited by 11
  4. 2024Beyond Complete Shapes: A Benchmark for Quantitative Evaluation of 3D Shape Surface Matching Algorithmscited by 11
  5. 2023DAE-Net: Deforming Auto-Encoder for fine-grained shape co-segmentationcited by 7
  6. 2025ASIA: Adaptive 3D Segmentation using Few Image Annotationscited by 6
  7. 2026PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understandingcited by 5
  8. 20243DStyleGLIP: Part-Tailored Text-Guided 3D Neural Stylizationcited by 4
  9. 2024HaLo-NeRF: Learning Geometry-Guided Semantics for Exploring Unconstrained Photo Collectionscited by 4
  10. 2026Artiverse: A Diverse and Physically Grounded Dataset for Articulated Objectscited by 4
  11. 2025GenAnalysis: Joint Shape Analysis by Learning Man-Made Shape Generators with Deformation Regularizationscited by 2
  12. 20253D PixBrush: Image-Guided Local Texture Synthesiscited by 2
  13. 2026Symmetry Informative and Agnostic Feature Disentanglement for 3D Shapescited by 2
  14. 2025Text2VDM: Text to Vector Displacement Maps for Expressive and Interactive 3D Sculptingcited by 1
  15. 2025MechStyle: Augmenting Generative AI with Mechanical Simulation to Create Stylized and Structurally Viable 3D Modelscited by 1
  16. 2025Hierarchical Transformers for Unsupervised 3D Shape Abstractioncited by 0
  17. 2026MME: Mixture of Mesh Experts with Random Walk Transformer Gatingcited by 0
  18. 2026MeshOn: Intersection-Free Mesh-to-Mesh Compositioncited by 0
  19. 2026Best Segmentation Buddies for Image-Shape Correspondencecited by 0
  20. 2026RADmesh: Remesh-Aware Mesh Deformationcited by 0