# 4D Unsupervised Object Discovery arXiv: https://arxiv.org/abs/2210.04801 ยท HTML: https://mlsanity.com/paper/2210.04801/ ## Key facts - Cited by 18 papers in the map (data through 14 Oct 2026). - Builds on 36 of its 51 references that are in the map. - First posted on arXiv 10 Oct 2022. ## Builds on (most cited first, 30 of 36) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020) - [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015) - [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019) - [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020) - [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017) - [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021) - [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016) - [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016) - [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015) - [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015) - [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019) - [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019) - [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016) - [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017) - [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019) - [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020) - [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017) - [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018) - [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019) - [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016) - [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019) - [Dense Contrastive Learning for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2011.09157/) (2020) - [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020) - [Localizing Objects with Self-Supervised Transformers and no Labels](https://mlsanity.com/paper/2109.14279/) (2021) - [Embracing Single Stride 3D Object Detector with Sparse Transformer](https://mlsanity.com/paper/2112.06375/) (2021) - [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020) - [FreeSOLO: Learning to Segment Objects without Annotations](https://mlsanity.com/paper/2202.12181/) (2022) - [Neural Scene Flow Prior](https://mlsanity.com/paper/2111.01253/) (2021) ## Built on by (most cited first, 18 of 18) - [ICP-Flow: LiDAR Scene Flow Estimation with ICP](https://mlsanity.com/paper/2402.17351/) (2024) - [Towards Unsupervised Object Detection From LiDAR Point Clouds](https://mlsanity.com/paper/2311.02007/) (2023) - [LISO: Lidar-only Self-Supervised 3D Object Detection](https://mlsanity.com/paper/2403.07071/) (2024) - [UNION: Unsupervised 3D Object Detection using Object Appearance-based Pseudo-Classes](https://mlsanity.com/paper/2405.15688/) (2024) - [Banana: Banach Fixed-Point Network for Pointcloud Segmentation with Inter-Part Equivariance](https://mlsanity.com/paper/2305.16314/) (2023) - [Reward Finetuning for Faster and More Accurate Unsupervised Object Discovery](https://mlsanity.com/paper/2310.19080/) (2023) - [HiMo: High-Speed Objects Motion Compensation in Point Clouds](https://mlsanity.com/paper/2503.00803/) (2025) - [VoteFlow: Enforcing Local Rigidity in Self-Supervised Scene Flow](https://mlsanity.com/paper/2503.22328/) (2025) - [General Geometry-aware Weakly Supervised 3D Object Detection](https://mlsanity.com/paper/2407.13748/) (2024) - [Learning to Detect Objects from Multi-Agent LiDAR Scans without Manual Labels](https://mlsanity.com/paper/2503.08421/) (2025) - [Label-Efficient 3D Object Detection For Road-Side Units](https://mlsanity.com/paper/2404.06256/) (2024) - [Vision-Language Guidance for LiDAR-based Unsupervised 3D Object Detection](https://mlsanity.com/paper/2408.03790/) (2024) - [Weakly Supervised 3D Object Detection with Multi-Stage Generalization](https://mlsanity.com/paper/2306.05418/) (2023) - [MonoSOWA: Scalable monocular 3D Object detector Without human Annotations](https://mlsanity.com/paper/2501.09481/) (2025) - [xMOD: Cross-Modal Distillation for 2D/3D Multi-Object Discovery from 2D motion](https://mlsanity.com/paper/2503.15022/) (2025) - [Object Concepts Emerge from Motion](https://mlsanity.com/paper/2505.21635/) (2025) - [Unsupervised Multi-agent and Single-agent Perception from Cooperative Views](https://mlsanity.com/paper/2604.05354/) (2026) - [Object Concepts Emerge from Motion](https://mlsanity.com/paper/2609.04348/) (2026) ## Abstract (arXiv) Object discovery is a core task in computer vision. While fast progresses have been made in supervised object detection, its unsupervised counterpart remains largely unexplored. With the growth of data volume, the expensive cost of annotations is the major limitation hindering further study. Therefore, discovering objects without annotations has great significance. However, this task seems impractical on still-image or point cloud alone due to the lack of discriminative information. Previous studies underlook the crucial temporal information and constraints naturally behind multi-modal inputs. In this paper, we propose 4D unsupervised object discovery, jointly discovering objects from 4D data -- 3D point clouds and 2D RGB images with temporal information. We present the first practical approach for this task by proposing a ClusterNet on 3D point clouds, which is jointly iteratively optimized with a 2D localization network. Extensive experiments on the large-scale Waymo Open Dataset suggest that the localization network and ClusterNet achieve competitive performance on both class-agnostic 2D object detection and 3D instance segmentation, bridging the gap between unsupervised methods and full supervised ones. Codes and models will be made available at https://github.com/Robertwyq/LSMOL.