# Emergent 3D Instance Segmentation from Self-Supervised Point Transformers

arXiv: https://arxiv.org/abs/2608.15796 · HTML: https://mlsanity.com/paper/2608.15796/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 16 Aug 2026.

## Builds on (most cited first, 24 of 24)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Point Transformer V3: Simpler, Faster, Stronger](https://mlsanity.com/paper/2312.10035/) (2023)
- [Localizing Objects with Self-Supervised Transformers and no Labels](https://mlsanity.com/paper/2109.14279/) (2021)
- [Cut and Learn for Unsupervised Object Detection and Instance Segmentation](https://mlsanity.com/paper/2301.11320/) (2023)
- [Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cut](https://mlsanity.com/paper/2202.11539/) (2022)
- [Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking](https://mlsanity.com/paper/2109.03805/) (2021)
- [Sonata: Self-Supervised Learning of Reliable Point Representations](https://mlsanity.com/paper/2503.16429/) (2025)
- [Motion Inspired Unsupervised Perception and Prediction in Autonomous Driving](https://mlsanity.com/paper/2210.08061/) (2022)
- [Learning to Detect Mobile Objects from LiDAR Scans Without Labels](https://mlsanity.com/paper/2203.15882/) (2022)
- [LISO: Lidar-only Self-Supervised 3D Object Detection](https://mlsanity.com/paper/2403.07071/) (2024)
- [UNION: Unsupervised 3D Object Detection using Object Appearance-based Pseudo-Classes](https://mlsanity.com/paper/2405.15688/) (2024)
- [Utonia: Toward One Encoder for All Point Clouds](https://mlsanity.com/paper/2603.03283/) (2026)
- [Drive&Segment: Unsupervised Semantic Segmentation of Urban Scenes via Cross-modal Distillation](https://mlsanity.com/paper/2203.11160/) (2022)
- [Shelf-Supervised Cross-Modal Pre-Training for 3D Object Detection](https://mlsanity.com/paper/2406.10115/) (2024)
- [UNIT: Unsupervised Online Instance Segmentation through Time](https://mlsanity.com/paper/2409.07887/) (2024)
- [Volume Transformer: Revisiting Vanilla Transformers for 3D Scene Understanding](https://mlsanity.com/paper/2604.19609/) (2026)
- [AutoInst: Automatic Instance-Based Segmentation of LiDAR 3D Scans](https://mlsanity.com/paper/2403.16318/) (2024)
- [TerraSeg: Self-Supervised Ground Segmentation for Any LiDAR](https://mlsanity.com/paper/2603.27344/) (2026)

## Abstract (arXiv)

Unsupervised 3D instance segmentation of outdoor LiDAR scans has traditionally relied on handcrafted geometric priors such as density-based clustering, motion cues, or projected 2D detections. In this work, we investigate whether a frozen, self-supervised point transformer already contains the structural information required to isolate object instances without any handcrafted geometric prior. Using this transformer purely as a feature extractor, we probe its internal representations across the SemanticKITTI, nuScenes, and Waymo Perception datasets. Our analysis yields four core insights: (1) the instance signal concentrates in the attention queries and keys rather than in the values or final output features; (2) output features semantically collapse, merging adjacent same-class objects that the queries and keys keep distinct; (3) this instance signal is bimodal in depth, strongest at the shallowest and deepest encoder stages; and (4) this signal is driven predominantly by the rotary position encoding (RoPE), whose removal collapses its advantage. We put these findings into our method TokenGraph3D, a training-free segmenter that groups points via connected components on a key-similarity graph, using neither density-based clustering nor proximity priors. Under identical prior-free conditions, we substantially outperform output-feature baselines, making the emergent 3D instance structure visible.