mlsanityData to 7 Oct 2026

iSeg: Interactive 3D Segmentation via Interactive Attention

arXiv 2404.03219 ↗First posted 4 Apr 2024cs.CV cs.GRcited by 14

Not on the map. Why?

Abstract (arXiv)

We present iSeg, a new interactive technique for segmenting 3D shapes. Previous works have focused mainly on leveraging pre-trained 2D foundation models for 3D segmentation based on text. However, text may be insufficient for accurately describing fine-grained spatial segmentations. Moreover, achieving a consistent 3D segmentation using a 2D model is highly challenging, since occluded areas of the same semantic region may not be visible together from any 2D view. Thus, we design a segmentation method conditioned on fine user clicks, which operates entirely in 3D. Our system accepts user clicks directly on the shape's surface, indicating the inclusion or exclusion of regions from the desired shape partition. To accommodate various click settings, we propose a novel interactive attention module capable of processing different numbers and types of clicks, enabling the training of a single unified interactive segmentation model. We apply iSeg to a myriad of shapes from different domains, demonstrating its versatility and faithfulness to the user's specifications. Our project page is at https://threedle.github.io/iSeg/.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 34 of its 66 references that are in the map. First posted on arXiv 4 Apr 2024.

Builds on

34 of 66 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Segment Anythingcited by 8,158
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  4. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  5. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  6. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  7. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  8. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 195
  9. 2016Thingi10K: A Dataset of 10,000 3D-Printing Modelscited by 182
  10. 2019CvxNet: Learnable Convex Decompositioncited by 173
  11. 2020DiffusionNet: Discretization Agnostic Learning on Surfacescited by 163
  12. 2023SAM3D: Segment Anything in 3D Scenescited by 155
22 more
  1. 2022Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoderscited by 143
  2. 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 117
  3. 2020Virtual Multi-view Fusion for 3D Semantic Segmentationcited by 99
  4. 2023SAI3D: Segment Any Instance in 3D Scenescited by 90
  5. 2019BAE-NET: Branched Autoencoder for Shape Co-Segmentationcited by 89
  6. 2023OmniSeg3D: Omniversal 3D Segmentation via Hierarchical Contrastive Learningcited by 75
  7. 2022NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenescited by 71
  8. 2016SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentationcited by 70
  9. 2018MeshCNN: A Network with an Edgecited by 69
  10. 20163D Shape Segmentation with Projective Convolutional Networkscited by 63
  11. 2023SATR: Zero-Shot Semantic Segmentation of 3D Shapescited by 62
  12. 2021Subdivision-Based Mesh Convolution Networkscited by 60
  13. 2023FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Modelscited by 47
  14. 2023SAM3D: Zero-Shot 3D Object Detection via Segment Anything Modelcited by 40
  15. 20233D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillationcited by 34
  16. 2023AGILE3D: Attention Guided Interactive Multi-object 3D Segmentationcited by 33
  17. 2023Zero-Shot 3D Shape Correspondencecited by 33
  18. 20223D Highlighter: Localizing Regions on 3D Shapes via Text Descriptionscited by 32
  19. 2023Bridging the Domain Gap: Self-Supervised 3D Scene Understanding with Foundation Modelscited by 26
  20. 2023Interactive Segment Anything NeRF with Feature Imitationcited by 24
  21. 2024PartSTAD: 2D-to-3D Part Segmentation Task Adaptationcited by 18
  22. 20223D Concept Grounding on Neural Fieldscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 5.

14 papers.

Sort:

By area:

  1. 2025PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyondcited by 91
  2. 2025PartSAM: A Scalable Promptable Part Segmentation Model Trained on Native 3D Datacited by 15
  3. 2025iSegMan: Interactive Segment-and-Manipulate 3D Gaussianscited by 11
  4. 2025WildSeg3D: Segment Any 3D Objects in the Wild from 2D Imagescited by 6
  5. 2025ASIA: Adaptive 3D Segmentation using Few Image Annotationscited by 6
  6. 2025GenAnalysis: Joint Shape Analysis by Learning Man-Made Shape Generators with Deformation Regularizationscited by 2
  7. 2025SUM Parts: Benchmarking Part-Level Semantic Segmentation of Urban Meshescited by 1
  8. 2026Semantic Foam: Unifying Spatial and Semantic Scene Decompositioncited by 1
  9. 2025PinPoint3D: Fine-Grained 3D Part Segmentation from a Few Clickscited by 0
  10. 2025HyperDiffusionFields (HyDiF): Diffusion-Guided Hypernetworks for Learning Implicit Molecular Neural Fieldscited by 0
  11. 2026Material Magic Wand: Material-Aware Grouping of 3D Parts in Untextured Meshescited by 0
  12. 2026ClickSeg3D: Few-Click Interactive Segmentation via Semantic Embeddingscited by 0
2 more
  1. 2026Best Segmentation Buddies for Image-Shape Correspondencecited by 0
  2. 2026MeshFM: 2D Features Are All You Need for 3D Shape Understandingcited by 0