# UNION: Unsupervised 3D Object Detection using Object Appearance-based Pseudo-Classes

arXiv: https://arxiv.org/abs/2405.15688 · HTML: https://mlsanity.com/paper/2405.15688/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 38 references that are in the map.
- First posted on arXiv 24 May 2024.

## Builds on (most cited first, 26 of 26)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection](https://mlsanity.com/paper/1908.09492/) (2019)
- [Localizing Objects with Self-Supervised Transformers and no Labels](https://mlsanity.com/paper/2109.14279/) (2021)
- [Cut and Learn for Unsupervised Object Detection and Instance Segmentation](https://mlsanity.com/paper/2301.11320/) (2023)
- [Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cut](https://mlsanity.com/paper/2202.11539/) (2022)
- [FreeSOLO: Learning to Segment Objects without Annotations](https://mlsanity.com/paper/2202.12181/) (2022)
- [Motion Inspired Unsupervised Perception and Prediction in Autonomous Driving](https://mlsanity.com/paper/2210.08061/) (2022)
- [Learning to Detect Mobile Objects from LiDAR Scans Without Labels](https://mlsanity.com/paper/2203.15882/) (2022)
- [ICP-Flow: LiDAR Scene Flow Estimation with ICP](https://mlsanity.com/paper/2402.17351/) (2024)
- [Towards Unsupervised Object Detection From LiDAR Point Clouds](https://mlsanity.com/paper/2311.02007/) (2023)
- [LISO: Lidar-only Self-Supervised 3D Object Detection](https://mlsanity.com/paper/2403.07071/) (2024)
- [Drive&Segment: Unsupervised Semantic Segmentation of Urban Scenes via Cross-modal Distillation](https://mlsanity.com/paper/2203.11160/) (2022)
- [4D Unsupervised Object Discovery](https://mlsanity.com/paper/2210.04801/) (2022)
- [Causal Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2310.07379/) (2023)
- [Reward Finetuning for Faster and More Accurate Unsupervised Object Discovery](https://mlsanity.com/paper/2310.19080/) (2023)
- [Commonsense Prototype for Outdoor Unsupervised 3D Object Detection](https://mlsanity.com/paper/2404.16493/) (2024)
- [Foreground Guidance and Multi-Layer Feature Fusion for Unsupervised Object Discovery with Transformers](https://mlsanity.com/paper/2210.13053/) (2022)

## Built on by (most cited first, 22 of 22)

- [VoteFlow: Enforcing Local Rigidity in Self-Supervised Scene Flow](https://mlsanity.com/paper/2503.22328/) (2025)
- [Harnessing Uncertainty-aware Bounding Boxes for Unsupervised 3D Object Detection](https://mlsanity.com/paper/2408.00619/) (2024)
- [On the Provable Importance of Gradients for Language-Assisted Image Clustering](https://mlsanity.com/paper/2510.16335/) (2025)
- [Learning to Detect Objects from Multi-Agent LiDAR Scans without Manual Labels](https://mlsanity.com/paper/2503.08421/) (2025)
- [OpenBox: Annotate Any Bounding Boxes in 3D](https://mlsanity.com/paper/2512.01352/) (2025)
- [S3PT: Scene Semantics and Structure Guided Clustering to Boost Self-Supervised Pre-Training for Autonomous Driving](https://mlsanity.com/paper/2410.23085/) (2024)
- [DoGFlow: Self-Supervised LiDAR Scene Flow via Cross-Modal Doppler Guidance](https://mlsanity.com/paper/2508.18506/) (2025)
- [SemanticFlow: A Self-Supervised Framework for Joint Scene Flow Prediction and Instance Segmentation in Dynamic Environments](https://mlsanity.com/paper/2503.14837/) (2025)
- [VESPA: Towards un(Human)supervised Open-World Pointcloud Labeling for Autonomous Driving](https://mlsanity.com/paper/2507.20397/) (2025)
- [TerraSeg: Self-Supervised Ground Segmentation for Any LiDAR](https://mlsanity.com/paper/2603.27344/) (2026)
- [Label-Free Model Failure Detection for Lidar-based Point Cloud Segmentation](https://mlsanity.com/paper/2407.14306/) (2024)
- [Enhancing Pseudo-Boxes via Data-Level LiDAR-Camera Fusion for Unsupervised 3D Object Detection](https://mlsanity.com/paper/2508.20530/) (2025)
- [AutoExpert: Automating 3D LiDAR Annotation from Expert-Crafted Guidelines](https://mlsanity.com/paper/2506.02914/) (2025)
- [OWL: Unsupervised 3D Object Detection by Occupancy Guided Warm-up and Large Model Priors Reasoning](https://mlsanity.com/paper/2512.05698/) (2025)
- [FlowCalib: LiDAR-to-Vehicle Miscalibration Detection using Scene Flows](https://mlsanity.com/paper/2601.23107/) (2026)
- [Unified Unsupervised and Sparsely-Supervised 3D Object Detection by Semantic Pseudo-Labeling and Prototype Learning](https://mlsanity.com/paper/2602.21484/) (2026)
- [When the City Teaches the Car: Label-Free 3D Perception from Infrastructure](https://mlsanity.com/paper/2603.16742/) (2026)
- [Unsupervised Multi-agent and Single-agent Perception from Cooperative Views](https://mlsanity.com/paper/2604.05354/) (2026)
- [Open-Vocabulary BEV Segmentation with 3D-Aware Geometric Constraints](https://mlsanity.com/paper/2606.24353/) (2026)
- [CorrelationFlow: A Training-Free Geometric Approach for LiDAR Scene Flow Estimation](https://mlsanity.com/paper/2607.29237/) (2026)
- [Emergent 3D Instance Segmentation from Self-Supervised Point Transformers](https://mlsanity.com/paper/2608.15796/) (2026)
- [RVLoss: Runoff Vote Loss for Self-Supervised LiDAR Scene Flow Estimation](https://mlsanity.com/paper/2608.18864/) (2026)

## Abstract (arXiv)

Unsupervised 3D object detection methods have emerged to leverage vast amounts of data without requiring manual labels for training. Recent approaches rely on dynamic objects for learning to detect mobile objects but penalize the detections of static instances during training. Multiple rounds of self-training are used to add detected static instances to the set of training targets; this procedure to improve performance is computationally expensive. To address this, we propose the method UNION. We use spatial clustering and self-supervised scene flow to obtain a set of static and dynamic object proposals from LiDAR. Subsequently, object proposals' visual appearances are encoded to distinguish static objects in the foreground and background by selecting static instances that are visually similar to dynamic objects. As a result, static and dynamic mobile objects are obtained together, and existing detectors can be trained with a single training. In addition, we extend 3D object discovery to detection by using object appearance-based cluster labels as pseudo-class labels for training object classification. We conduct extensive experiments on the nuScenes dataset and increase the state-of-the-art performance for unsupervised 3D object discovery, i.e. UNION more than doubles the average precision to 39.5. The code is available at github.com/TedLentsch/UNION.