# Sonata: Self-Supervised Learning of Reliable Point Representations

arXiv: https://arxiv.org/abs/2503.16429 · HTML: https://mlsanity.com/paper/2503.16429/

## Key facts

- Cited by 110 papers in the map (data through 14 Oct 2026).
- Builds on 93 of its 111 references that are in the map.
- First posted on arXiv 20 Mar 2025.

## Builds on (most cited first, 30 of 93)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)

## Built on by (most cited first, 30 of 110)

- [PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation](https://mlsanity.com/paper/2601.03782/) (2026)
- [SpatialLM: Training Large Language Models for Structured Indoor Modeling](https://mlsanity.com/paper/2506.07491/) (2025)
- [Concerto: Joint 2D-3D Self-Supervised Learning Emerges Spatial Representations](https://mlsanity.com/paper/2510.23607/) (2025)
- [P3-SAM: Native 3D Part Segmentation](https://mlsanity.com/paper/2509.06784/) (2025)
- [DINO in the Room: Leveraging 2D Foundation Models for 3D Segmentation](https://mlsanity.com/paper/2503.18944/) (2025)
- [N3D-VLM: Native 3D Grounding Enables Accurate Spatial Reasoning in Vision-Language Models](https://mlsanity.com/paper/2512.16561/) (2025)
- [Utonia: Toward One Encoder for All Point Clouds](https://mlsanity.com/paper/2603.03283/) (2026)
- [Reasoning in Space via Grounding in the World](https://mlsanity.com/paper/2510.13800/) (2025)
- [LitePT: Lighter Yet Stronger Point Transformer](https://mlsanity.com/paper/2512.13689/) (2025)
- [SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting](https://mlsanity.com/paper/2506.08710/) (2025)
- [Any3D-VLA: Enhancing VLA Robustness via Diverse Point Clouds](https://mlsanity.com/paper/2602.00807/) (2026)
- [Chorus: Multi-Teacher Pretraining for Holistic 3D Gaussian Scene Encoding](https://mlsanity.com/paper/2512.17817/) (2025)
- [Register Any Point: Scaling 3D Point Cloud Registration by Flow Matching](https://mlsanity.com/paper/2512.01850/) (2025)
- [Learning 3D-Gaussian Simulators from RGB Videos](https://mlsanity.com/paper/2503.24009/) (2025)
- [Beyond Flatlands: Unlocking Spatial Intelligence by Decoupling 3D Reasoning from Numerical Regression](https://mlsanity.com/paper/2511.11239/) (2025)
- [Mantis: Mamba-native Tuning is Efficient for 3D Point Cloud Foundation Models](https://mlsanity.com/paper/2605.03438/) (2026)
- [Learning Part-Aware Dense 3D Feature Field for Generalizable Articulated Object Manipulation](https://mlsanity.com/paper/2602.14193/) (2026)
- [On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentation](https://mlsanity.com/paper/2505.22444/) (2025)
- [How Many Tokens Do 3D Point Cloud Transformer Architectures Really Need?](https://mlsanity.com/paper/2511.05449/) (2025)
- [Towards Foundation Models for 3D Scene Understanding: Instance-Aware Self-Supervised Learning for Point Clouds](https://mlsanity.com/paper/2603.25165/) (2026)
- [TORA: Topological Representation Alignment for 3D Shape Assembly](https://mlsanity.com/paper/2604.04050/) (2026)
- [Seed3D 2.0: Advancing High-Fidelity Simulation-Ready 3D Content Generation](https://mlsanity.com/paper/2605.13862/) (2026)
- [Pts3D-LLM: Studying the Impact of Token Structure for 3D Scene Understanding With Large Language Models](https://mlsanity.com/paper/2506.05689/) (2025)
- [StereoPolicy: Improving Robotic Manipulation Policies via Stereo Perception](https://mlsanity.com/paper/2605.09989/) (2026)
- [CitySeg: A 3D Open Vocabulary Semantic Segmentation Foundation Model in City-scale Scenarios](https://mlsanity.com/paper/2508.09470/) (2025)
- [Multi-modal Loop Closure Detection with Foundation Models in Severely Unstructured Environments](https://mlsanity.com/paper/2511.05404/) (2025)
- [Panda: Self-distillation of Reusable Sensor-level Representations for High Energy Physics](https://mlsanity.com/paper/2512.01324/) (2025)
- [Polar Perspectives: Evaluating 2-D LiDAR Projections for Robust Place Recognition with Visual Foundation Models](https://mlsanity.com/paper/2512.02897/) (2025)
- [R3DPA: Leveraging 3D Representation Alignment and RGB Pretrained Priors for LiDAR Scene Generation](https://mlsanity.com/paper/2601.07692/) (2026)
- [OnlineSI: Taming Large Language Model for Online 3D Understanding and Grounding](https://mlsanity.com/paper/2601.16538/) (2026)

## Abstract (arXiv)

In this paper, we question whether we have a reliable self-supervised point cloud model that can be used for diverse 3D tasks via simple linear probing, even with limited data and minimal computation. We find that existing 3D self-supervised learning approaches fall short when evaluated on representation quality through linear probing. We hypothesize that this is due to what we term the "geometric shortcut", which causes representations to collapse to low-level spatial features. This challenge is unique to 3D and arises from the sparse nature of point cloud data. We address it through two key strategies: obscuring spatial information and enhancing the reliance on input features, ultimately composing a Sonata of 140k point clouds through self-distillation. Sonata is simple and intuitive, yet its learned representations are strong and reliable: zero-shot visualizations demonstrate semantic grouping, alongside strong spatial reasoning through nearest-neighbor relationships. Sonata demonstrates exceptional parameter and data efficiency, tripling linear probing accuracy (from 21.8% to 72.5%) on ScanNet and nearly doubling performance with only 1% of the data compared to previous approaches. Full fine-tuning further advances SOTA across both 3D indoor and outdoor perception tasks.