# iSeg: Interactive 3D Segmentation via Interactive Attention

arXiv: https://arxiv.org/abs/2404.03219 · HTML: https://mlsanity.com/paper/2404.03219/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 66 references that are in the map.
- First posted on arXiv 4 Apr 2024.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [OpenScene: 3D Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2211.15654/) (2022)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations](https://mlsanity.com/paper/2209.03494/) (2022)
- [Thingi10K: A Dataset of 10,000 3D-Printing Models](https://mlsanity.com/paper/1605.04797/) (2016)
- [CvxNet: Learnable Convex Decomposition](https://mlsanity.com/paper/1909.05736/) (2019)
- [DiffusionNet: Discretization Agnostic Learning on Surfaces](https://mlsanity.com/paper/2012.00888/) (2020)
- [SAM3D: Segment Anything in 3D Scenes](https://mlsanity.com/paper/2306.03908/) (2023)
- [Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoders](https://mlsanity.com/paper/2212.06785/) (2022)
- [PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Models](https://mlsanity.com/paper/2212.01558/) (2022)
- [Virtual Multi-view Fusion for 3D Semantic Segmentation](https://mlsanity.com/paper/2007.13138/) (2020)
- [SAI3D: Segment Any Instance in 3D Scenes](https://mlsanity.com/paper/2312.11557/) (2023)
- [BAE-NET: Branched Autoencoder for Shape Co-Segmentation](https://mlsanity.com/paper/1903.11228/) (2019)
- [OmniSeg3D: Omniversal 3D Segmentation via Hierarchical Contrastive Learning](https://mlsanity.com/paper/2311.11666/) (2023)
- [NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenes](https://mlsanity.com/paper/2209.08776/) (2022)
- [SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation](https://mlsanity.com/paper/1612.00606/) (2016)
- [MeshCNN: A Network with an Edge](https://mlsanity.com/paper/1809.05910/) (2018)
- [3D Shape Segmentation with Projective Convolutional Networks](https://mlsanity.com/paper/1612.02808/) (2016)
- [SATR: Zero-Shot Semantic Segmentation of 3D Shapes](https://mlsanity.com/paper/2304.04909/) (2023)
- [Subdivision-Based Mesh Convolution Networks](https://mlsanity.com/paper/2106.02285/) (2021)
- [FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Models](https://mlsanity.com/paper/2303.12786/) (2023)
- [SAM3D: Zero-Shot 3D Object Detection via Segment Anything Model](https://mlsanity.com/paper/2306.02245/) (2023)
- [3D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillation](https://mlsanity.com/paper/2311.09571/) (2023)
- [AGILE3D: Attention Guided Interactive Multi-object 3D Segmentation](https://mlsanity.com/paper/2306.00977/) (2023)
- [Zero-Shot 3D Shape Correspondence](https://mlsanity.com/paper/2306.03253/) (2023)
- [3D Highlighter: Localizing Regions on 3D Shapes via Text Descriptions](https://mlsanity.com/paper/2212.11263/) (2022)

## Built on by (most cited first, 14 of 14)

- [PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyond](https://mlsanity.com/paper/2504.11451/) (2025)
- [PartSAM: A Scalable Promptable Part Segmentation Model Trained on Native 3D Data](https://mlsanity.com/paper/2509.21965/) (2025)
- [iSegMan: Interactive Segment-and-Manipulate 3D Gaussians](https://mlsanity.com/paper/2505.11934/) (2025)
- [WildSeg3D: Segment Any 3D Objects in the Wild from 2D Images](https://mlsanity.com/paper/2503.08407/) (2025)
- [ASIA: Adaptive 3D Segmentation using Few Image Annotations](https://mlsanity.com/paper/2509.24288/) (2025)
- [GenAnalysis: Joint Shape Analysis by Learning Man-Made Shape Generators with Deformation Regularizations](https://mlsanity.com/paper/2503.00807/) (2025)
- [SUM Parts: Benchmarking Part-Level Semantic Segmentation of Urban Meshes](https://mlsanity.com/paper/2503.15300/) (2025)
- [Semantic Foam: Unifying Spatial and Semantic Scene Decomposition](https://mlsanity.com/paper/2604.26262/) (2026)
- [PinPoint3D: Fine-Grained 3D Part Segmentation from a Few Clicks](https://mlsanity.com/paper/2509.25970/) (2025)
- [HyperDiffusionFields (HyDiF): Diffusion-Guided Hypernetworks for Learning Implicit Molecular Neural Fields](https://mlsanity.com/paper/2510.18122/) (2025)
- [Material Magic Wand: Material-Aware Grouping of 3D Parts in Untextured Meshes](https://mlsanity.com/paper/2603.17370/) (2026)
- [ClickSeg3D: Few-Click Interactive Segmentation via Semantic Embeddings](https://mlsanity.com/paper/2605.08925/) (2026)
- [Best Segmentation Buddies for Image-Shape Correspondence](https://mlsanity.com/paper/2605.18193/) (2026)
- [MeshFM: 2D Features Are All You Need for 3D Shape Understanding](https://mlsanity.com/paper/2607.27592/) (2026)

## Abstract (arXiv)

We present iSeg, a new interactive technique for segmenting 3D shapes. Previous works have focused mainly on leveraging pre-trained 2D foundation models for 3D segmentation based on text. However, text may be insufficient for accurately describing fine-grained spatial segmentations. Moreover, achieving a consistent 3D segmentation using a 2D model is highly challenging, since occluded areas of the same semantic region may not be visible together from any 2D view. Thus, we design a segmentation method conditioned on fine user clicks, which operates entirely in 3D. Our system accepts user clicks directly on the shape's surface, indicating the inclusion or exclusion of regions from the desired shape partition. To accommodate various click settings, we propose a novel interactive attention module capable of processing different numbers and types of clicks, enabling the training of a single unified interactive segmentation model. We apply iSeg to a myriad of shapes from different domains, demonstrating its versatility and faithfulness to the user's specifications. Our project page is at https://threedle.github.io/iSeg/.