# UNIT: Unsupervised Online Instance Segmentation through Time

arXiv: https://arxiv.org/abs/2409.07887 · HTML: https://mlsanity.com/paper/2409.07887/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 40 references that are in the map.
- First posted on arXiv 12 Sep 2024.

## Builds on (most cited first, 30 of 30)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [An End-to-End Transformer Model for 3D Object Detection](https://mlsanity.com/paper/2109.08141/) (2021)
- [Mask3D: Mask Transformer for 3D Semantic Instance Segmentation](https://mlsanity.com/paper/2210.03105/) (2022)
- [PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving](https://mlsanity.com/paper/2112.12610/) (2021)
- [Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking](https://mlsanity.com/paper/2109.03805/) (2021)
- [Train in Germany, Test in The USA: Making 3D Object Detectors Generalize](https://mlsanity.com/paper/2005.08139/) (2020)
- [Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Data](https://mlsanity.com/paper/2203.16258/) (2022)
- [ALSO: Automotive Lidar Self-supervision by Occupancy estimation](https://mlsanity.com/paper/2212.05867/) (2022)
- [4D Panoptic LiDAR Segmentation](https://mlsanity.com/paper/2102.12472/) (2021)
- [MOPT: Multi-Object Panoptic Tracking](https://mlsanity.com/paper/2004.08189/) (2020)
- [Patchwork++: Fast and Robust Ground Segmentation Solving Partial Under-Segmentation Using 3D Point Cloud](https://mlsanity.com/paper/2207.11919/) (2022)
- [Three Pillars improving Vision Foundation Model Distillation for Lidar](https://mlsanity.com/paper/2310.17504/) (2023)
- [UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenes](https://mlsanity.com/paper/2303.14541/) (2023)
- [Patchwork: Concentric Zone-based Region-wise Ground Segmentation with Ground Likelihood Estimation Using a 3D LiDAR Sensor](https://mlsanity.com/paper/2108.05560/) (2021)
- [BEVContrast: Self-Supervision in BEV Space for Automotive Lidar Point Clouds](https://mlsanity.com/paper/2310.17281/) (2023)
- [Spatiotemporal Self-supervised Learning for Point Clouds in the Wild](https://mlsanity.com/paper/2303.16235/) (2023)
- [LISO: Lidar-only Self-Supervised 3D Object Detection](https://mlsanity.com/paper/2403.07071/) (2024)
- [SeMoLi: What Moves Together Belongs Together](https://mlsanity.com/paper/2402.19463/) (2024)
- [AutoInst: Automatic Instance-Based Segmentation of LiDAR 3D Scans](https://mlsanity.com/paper/2403.16318/) (2024)

## Built on by (most cited first, 5 of 5)

- [Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systems](https://mlsanity.com/paper/2512.24385/) (2025)
- [Is clustering enough for LiDAR instance segmentation? A state-of-the-art training-free baseline](https://mlsanity.com/paper/2503.13203/) (2025)
- [UniLiPs: Unified LiDAR Pseudo-Labeling with Geometry-Grounded Dynamic Scene Decomposition](https://mlsanity.com/paper/2601.05105/) (2026)
- [Unsupervised Online 3D Instance Segmentation with Synthetic Sequences and Dynamic Loss](https://mlsanity.com/paper/2509.23194/) (2025)
- [Emergent 3D Instance Segmentation from Self-Supervised Point Transformers](https://mlsanity.com/paper/2608.15796/) (2026)

## Abstract (arXiv)

Online object segmentation and tracking in Lidar point clouds enables autonomous agents to understand their surroundings and make safe decisions. Unfortunately, manual annotations for these tasks are prohibitively costly. We tackle this problem with the task of class-agnostic unsupervised online instance segmentation and tracking. To that end, we leverage an instance segmentation backbone and propose a new training recipe that enables the online tracking of objects. Our network is trained on pseudo-labels, eliminating the need for manual annotations. We conduct an evaluation using metrics adapted for temporal instance segmentation. Computing these metrics requires temporally-consistent instance labels. When unavailable, we construct these labels using the available 3D bounding boxes and semantic labels in the dataset. We compare our method against strong baselines and demonstrate its superiority across two different outdoor Lidar datasets.