mlsanityData to 13 Oct 2026

Sonata: Self-Supervised Learning of Reliable Point Representations

arXiv 2503.16429 ↗First posted 20 Mar 2025cs.CVcited by 109

Not on the map. Why?

Abstract (arXiv)

In this paper, we question whether we have a reliable self-supervised point cloud model that can be used for diverse 3D tasks via simple linear probing, even with limited data and minimal computation. We find that existing 3D self-supervised learning approaches fall short when evaluated on representation quality through linear probing. We hypothesize that this is due to what we term the "geometric shortcut", which causes representations to collapse to low-level spatial features. This challenge is unique to 3D and arises from the sparse nature of point cloud data. We address it through two key strategies: obscuring spatial information and enhancing the reliance on input features, ultimately composing a Sonata of 140k point clouds through self-distillation. Sonata is simple and intuitive, yet its learned representations are strong and reliable: zero-shot visualizations demonstrate semantic grouping, alongside strong spatial reasoning through nearest-neighbor relationships. Sonata demonstrates exceptional parameter and data efficiency, tripling linear probing accuracy (from 21.8% to 72.5%) on ScanNet and nearly doubling performance with only 1% of the data compared to previous approaches. Full fine-tuning further advances SOTA across both 3D indoor and outdoor perception tasks.

Key facts. Cited by 109 papers in the map (data through 13 Oct 2026). Builds on 93 of its 111 references that are in the map. First posted on arXiv 20 Mar 2025.

Builds on

93 of 111 references.

References · 93

  1. 2017Attention Is All You Needcited by 60,106
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,656
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,009
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,857
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,492
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,364
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,991
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,915
  10. 2016Layer Normalizationcited by 5,624
  11. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,231
  12. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,021
38 more
  1. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,730
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,498
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,389
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,096
  5. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,930
  6. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,850
  7. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,440
  8. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,841
  9. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,539
  10. 2020Point Transformercited by 1,465
  11. 2024Depth Anything V2cited by 1,399
  12. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,381
  13. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,353
  14. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,312
  15. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,269
  16. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,205
  17. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,203
  18. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,150
  19. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,064
  20. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,023
  21. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,018
  22. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 895
  23. 2023Vision Transformers Need Registerscited by 850
  24. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 843
  25. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  26. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 752
  27. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 747
  28. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 732
  29. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 676
  30. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  31. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 648
  32. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 585
  33. 2023Point Transformer V3: Simpler, Faster, Strongercited by 584
  34. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 583
  35. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 503
  36. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 457
  37. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 445
  38. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 432

50 most cited of 93 All 93 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 31; 2026: 78.

109 papers.

Sort:

By area:

  1. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 79
  2. 2025SpatialLM: Training Large Language Models for Structured Indoor Modelingcited by 64
  3. 2025Concerto: Joint 2D-3D Self-Supervised Learning Emerges Spatial Representationscited by 43
  4. 2025P3-SAM: Native 3D Part Segmentationcited by 42
  5. 2025DINO in the Room: Leveraging 2D Foundation Models for 3D Segmentationcited by 31
  6. 2025N3D-VLM: Native 3D Grounding Enables Accurate Spatial Reasoning in Vision-Language Modelscited by 28
  7. 2026Utonia: Toward One Encoder for All Point Cloudscited by 19
  8. 2025Reasoning in Space via Grounding in the Worldcited by 18
  9. 2025LitePT: Lighter Yet Stronger Point Transformercited by 17
  10. 2025SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splattingcited by 14
  11. 2026Any3D-VLA: Enhancing VLA Robustness via Diverse Point Cloudscited by 13
  12. 2025Chorus: Multi-Teacher Pretraining for Holistic 3D Gaussian Scene Encodingcited by 9
38 more
  1. 2025Learning 3D-Gaussian Simulators from RGB Videoscited by 8
  2. 2025Register Any Point: Scaling 3D Point Cloud Registration by Flow Matchingcited by 8
  3. 2025Beyond Flatlands: Unlocking Spatial Intelligence by Decoupling 3D Reasoning from Numerical Regressioncited by 7
  4. 2026Mantis: Mamba-native Tuning is Efficient for 3D Point Cloud Foundation Modelscited by 6
  5. 2026Learning Part-Aware Dense 3D Feature Field for Generalizable Articulated Object Manipulationcited by 5
  6. 2025On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentationcited by 4
  7. 2025How Many Tokens Do 3D Point Cloud Transformer Architectures Really Need?cited by 4
  8. 2026Towards Foundation Models for 3D Scene Understanding: Instance-Aware Self-Supervised Learning for Point Cloudscited by 4
  9. 2026TORA: Topological Representation Alignment for 3D Shape Assemblycited by 4
  10. 2026Seed3D 2.0: Advancing High-Fidelity Simulation-Ready 3D Content Generationcited by 4
  11. 2025Pts3D-LLM: Studying the Impact of Token Structure for 3D Scene Understanding With Large Language Modelscited by 2
  12. 2025CitySeg: A 3D Open Vocabulary Semantic Segmentation Foundation Model in City-scale Scenarioscited by 2
  13. 2025Multi-modal Loop Closure Detection with Foundation Models in Severely Unstructured Environmentscited by 2
  14. 2025Panda: Self-distillation of Reusable Sensor-level Representations for High Energy Physicscited by 2
  15. 2025Polar Perspectives: Evaluating 2-D LiDAR Projections for Robust Place Recognition with Visual Foundation Modelscited by 2
  16. 2026R3DPA: Leveraging 3D Representation Alignment and RGB Pretrained Priors for LiDAR Scene Generationcited by 2
  17. 2026OnlineSI: Taming Large Language Model for Online 3D Understanding and Groundingcited by 2
  18. 2026SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMscited by 2
  19. 2026Emerging Extrinsic Dexterity in Cluttered Scenes via Dynamics-aware Policy Learningcited by 2
  20. 2026TerraSeg: Self-Supervised Ground Segmentation for Any LiDARcited by 2
  21. 2026PointTPA: Dynamic Network Parameter Adaptation for 3D Scene Understandingcited by 2
  22. 2026StereoPolicy: Improving Robotic Manipulation Policies via Stereo Perceptioncited by 2
  23. 2025Parameter-Efficient CLIP Adaptation for 3D Understanding via Unified Tokenizationcited by 1
  24. 2025Learning from Random Subspace Exploration: Generalized Test-Time Augmentation with Self-supervised Distillationcited by 1
  25. 2025Distilled-3DGS:Distilled 3D Gaussian Splattingcited by 1
  26. 2025M3DMap: Object-aware Multimodal 3D Mapping for Dynamic Environmentscited by 1
  27. 2026JOPP-3D: Joint Open Vocabulary Semantic Segmentation on Point Clouds and Panoramascited by 1
  28. 2026GeoGuide: Hierarchical Geometric Guidance for Open-Vocabulary 3D Semantic Segmentationcited by 1
  29. 2026Control-DINO: Feature Space Conditioning for Controllable Image-to-Video Diffusioncited by 1
  30. 2026Co-generation of Layout and Shape from Text via Autoregressive 3D Diffusioncited by 1
  31. 2026PointCSP: Cross-Sample Semantic Propagation and Stability Preservation in Self-Supervised Point Cloud Learningcited by 1
  32. 2026Analogical Trajectory Transfercited by 1
  33. 2026OrbiSim: World Models as Differentiable Physics Engines for Embodied Intelligencecited by 1
  34. 2026A Survey on Deep Learning Architectures for Point Cloud Classification and Segmentationcited by 1
  35. 2026ESAM++: Efficient Online 3D Perception on the Edgecited by 1
  36. 2026From Extrinsic to Intrinsic: Geodesic-Guided Representation Learning for 3D Geometric Datacited by 1
  37. 2026PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understandingcited by 1
  38. 20263D Point World Models: Point Completion Enables More Accurate Dynamics Learningcited by 1

50 most cited of 109 All 109 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Geometrically-driven Aggregation for Zero-shot 3D Point Cloud Understanding
  2. 2026Learning Point Cloud Geometry as a Statistical Manifold: Theory and Practice ↗
  3. 2023PointMoment:Mixed-Moment-based Self-Supervised Representation Learning for 3D Point Clouds
  4. 2025StruMamba3D: Exploring Structural Mamba for Self-supervised Point Cloud Representation Learning
  5. 2026A Cookbook of 3D Vision: Data, Learning Paradigms, and Application
  6. 2025Self-Supervised Point Cloud Completion based on Multi-View Augmentations of Single Partial Point Cloud
  7. 2025SM3D: Mitigating Spectral Bias and Semantic Dilution in Point Cloud State Space Models
  8. 2025Point-MoE: Large-Scale Multi-Dataset Training with Mixture-of-Experts for 3D Semantic Segmentation