Towards Foundation Models for 3D Scene Understanding: Instance-Aware Self-Supervised Learning for Point Clouds
Not on the map. Why?
Abstract (arXiv)
Recent advances in self-supervised learning (SSL) for point clouds have substantially improved 3D scene understanding without human annotations. Existing approaches emphasize semantic awareness by enforcing feature consistency across augmented views or by masked scene modeling. However, the resulting representations transfer poorly to instance localization, and often require full finetuning for strong performance. Instance awareness is a fundamental component of 3D perception, thus bridging this gap is crucial for progressing toward true 3D foundation models that support all downstream tasks on 3D data. In this work, we introduce PointINS, an instance-oriented self-supervised framework that enriches point cloud representations through geometry-aware learning. PointINS employs an orthogonal offset branch to jointly learn high-level semantic understanding and geometric reasoning, yielding instance awareness. We identify two consistent properties essential for robust instance localization and formulate them as complementary regularization strategies, Offset Distribution Regularization (ODR), which aligns predicted offsets with empirically observed geometric priors, and Spatial Clustering Regularization (SCR), which enforces local coherence by regularizing offsets with pseudo-instance masks. Through extensive experiments across five datasets, PointINS achieves on average +3.5% mAP improvement for indoor instance segmentation and +4.1% PQ gain for outdoor panoptic segmentation, paving the way for scalable 3D foundation models.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 42 of its 49 references that are in the map. First posted on arXiv 26 Mar 2026.
Builds on
42 of 49 references.
References · 42
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
- 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,476
- 2020Point Transformercited by 1,470
- 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
- 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
- 2020Center-based 3D Object Detection and Trackingcited by 1,144
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2023Point Transformer V3: Simpler, Faster, Strongercited by 589
30 more
- 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 504
- 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 385
- 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 321
- 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 286
- 2022Language-Grounded Indoor 3D Semantic Segmentation in the Wildcited by 246
- 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
- 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 200
- 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 166
- 2021Hierarchical Aggregation for 3D Instance Segmentationcited by 127
- 2025Sonata: Self-Supervised Learning of Reliable Point Representationscited by 110
- 2021Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentationcited by 107
- 2022Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Cloudscited by 102
- 2022LaserMix for Semi-Supervised LiDAR Semantic Segmentationcited by 93
- 2022Scribble-Supervised LiDAR Semantic Segmentationcited by 69
- 2022Panoptic-PHNet: Towards Real-Time and High-Precision LiDAR Panoptic Segmentation via Clustering Pseudo Heatmapcited by 59
- 2022Waymo Open Dataset: Panoramic Video Panoptic Segmentationcited by 58
- 2022Occupancy-MAE: Self-supervised Pre-training Large-scale LiDAR Point Clouds with Masked Occupancy Autoencoderscited by 58
- 2022GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Cloudscited by 57
- 2023Masked Scene Contrast: A Scalable Framework for Unsupervised 3D Representation Learningcited by 56
- 2023GeoMAE: Masked Geometric Target Prediction for Self-supervised Point Cloud Pre-Trainingcited by 38
- 2022MAELi: Masked Autoencoder for Large-Scale LiDAR Point Cloudscited by 20
- 2023TULIP: Transformer for Upsampling of LiDAR Point Cloudscited by 12
- 2025Multi-Scale Neighborhood Occupancy Masked Autoencoder for Self-Supervised Learning in LiDAR Point Cloudscited by 7
- 2023Center Focusing Network for Real-Time LiDAR Panoptic Segmentationcited by 6
- 2024Unsupervised Point Cloud Registration with Self-Distillationcited by 4
- 2025Masked Scene Modeling: Narrowing the Gap Between Supervised and Self-Supervised Learning in 3D Scene Understandingcited by 4
- 2022Depth Contrast: Self-Supervised Pretraining on 3DPM Images for Mining Material Classificationcited by 3
- 2025FLARES: Fast and Accurate LiDAR Multi-Range Semantic Segmentationcited by 3
- 2025PSA-SSL: Pose and Size-aware Self-Supervised Learning on LiDAR Point Cloudscited by 3
- 2025DOS: Distilling Observable Softmaps of Zipfian Prototypes for Self-Supervised Point Representationcited by 2
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Collaborative Learning for Semi-Supervised LiDAR Semantic Segmentationcited by 1
- 2026GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structurecited by 1
- 2026Data-Efficient Semantic Segmentation of 3D Point Clouds via Open-Vocabulary Image Segmentation-based Pseudo-Labelingcited by 0
- 2026Toward a foundation model for forest point cloudscited by 0
- 2026Toward a foundation model for forest point clouds21 Sep 2026
- 2026GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure14 Aug 2026
- 2026Collaborative Learning for Semi-Supervised LiDAR Semantic Segmentation16 May 2026
- 2026Data-Efficient Semantic Segmentation of 3D Point Clouds via Open-Vocabulary Image Segmentation-based Pseudo-Labeling13 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2023Geometrically-driven Aggregation for Zero-shot 3D Point Cloud Understanding
- 2026Learning Point Cloud Geometry as a Statistical Manifold: Theory and Practice ↗
- 20263D-IDE: 3D Implicit Depth Emergent
- 2024GeoMask3D: Geometrically Informed Mask Selection for Self-Supervised Point Cloud Learning in 3D
- 2025Oh-A-DINO: Understanding and Enhancing Attribute-Level Information in Self-Supervised Object-Centric Representations
- 2026SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
- 2026LaSSM: Efficient Semantic-Spatial Query Decoding via Local Aggregation and State Space Models for 3D Instance Segmentation