# ALSO: Automotive Lidar Self-supervision by Occupancy estimation

arXiv: https://arxiv.org/abs/2212.05867 · HTML: https://mlsanity.com/paper/2212.05867/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 99 references that are in the map.
- First posted on arXiv 12 Dec 2022.

## Builds on (most cited first, 30 of 70)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)

## Built on by (most cited first, 30 of 68)

- [Segment Any Point Cloud Sequences by Distilling Vision Foundation Models](https://mlsanity.com/paper/2306.09347/) (2023)
- [UniPAD: A Universal Pre-training Paradigm for Autonomous Driving](https://mlsanity.com/paper/2310.08370/) (2023)
- [PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm](https://mlsanity.com/paper/2310.08586/) (2023)
- [POP-3D: Open-Vocabulary 3D Occupancy Prediction from Images](https://mlsanity.com/paper/2401.09413/) (2024)
- [FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation](https://mlsanity.com/paper/2312.04484/) (2023)
- [Three Pillars improving Vision Foundation Model Distillation for Lidar](https://mlsanity.com/paper/2310.17504/) (2023)
- [UnO: Unsupervised Occupancy Fields for Perception and Forecasting](https://mlsanity.com/paper/2406.08691/) (2024)
- [UniWorld: Autonomous Driving Pre-training via World Models](https://mlsanity.com/paper/2308.07234/) (2023)
- [4D Contrastive Superflows are Dense 3D Representation Learners](https://mlsanity.com/paper/2407.06190/) (2024)
- [Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2303.01212/) (2023)
- [Multi-Space Alignments Towards Universal LiDAR Segmentation](https://mlsanity.com/paper/2405.01538/) (2024)
- [BEVContrast: Self-Supervision in BEV Space for Automotive Lidar Point Clouds](https://mlsanity.com/paper/2310.17281/) (2023)
- [Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities](https://mlsanity.com/paper/2401.08045/) (2024)
- [SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation](https://mlsanity.com/paper/2304.03251/) (2023)
- [Self-supervised Learning for Pre-Training 3D Point Clouds: A Survey](https://mlsanity.com/paper/2305.04691/) (2023)
- [MAELi: Masked Autoencoder for Large-Scale LiDAR Point Clouds](https://mlsanity.com/paper/2212.07207/) (2022)
- [LiMoE: Mixture of LiDAR Representation Learners from Automotive Scenes](https://mlsanity.com/paper/2501.04004/) (2025)
- [Perspective-Invariant 3D Object Detection](https://mlsanity.com/paper/2507.17665/) (2025)
- [Data-Centric Evolution in Autonomous Driving: A Comprehensive Survey of Big Data System, Data Mining, and Closed-Loop Technologies](https://mlsanity.com/paper/2401.12888/) (2024)
- [OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networks](https://mlsanity.com/paper/2404.14027/) (2024)
- [UniScene: Multi-Camera Unified Pre-training via 3D Scene Reconstruction for Autonomous Driving](https://mlsanity.com/paper/2305.18829/) (2023)
- [VisionPAD: A Vision-Centric Pre-training Paradigm for Autonomous Driving](https://mlsanity.com/paper/2411.14716/) (2024)
- [Self-Supervised Representation Learning with Joint Embedding Predictive Architecture for Automotive LiDAR Object Detection](https://mlsanity.com/paper/2501.04969/) (2025)
- [UniM2AE: Multi-modal Masked Autoencoders with Unified 3D Representation for 3D Perception in Autonomous Driving](https://mlsanity.com/paper/2308.10421/) (2023)
- [An Empirical Study of Training State-of-the-Art LiDAR Segmentation Models](https://mlsanity.com/paper/2405.14870/) (2024)
- [LargeAD: Large-Scale Cross-Sensor Data Pretraining for Autonomous Driving](https://mlsanity.com/paper/2501.04005/) (2025)
- [3EED: Ground Everything Everywhere in 3D](https://mlsanity.com/paper/2511.01755/) (2025)
- [T-MAE: Temporal Masked Autoencoders for Point Cloud Representation Learning](https://mlsanity.com/paper/2312.10217/) (2023)
- [Label-Efficient LiDAR Semantic Segmentation with 2D-3D Vision Transformer Adapters](https://mlsanity.com/paper/2503.03299/) (2025)
- [Building a Strong Pre-Training Baseline for Universal 3D Large-Scale Perception](https://mlsanity.com/paper/2405.07201/) (2024)

## Abstract (arXiv)

We propose a new self-supervised method for pre-training the backbone of deep perception models operating on point clouds. The core idea is to train the model on a pretext task which is the reconstruction of the surface on which the 3D points are sampled, and to use the underlying latent vectors as input to the perception head. The intuition is that if the network is able to reconstruct the scene surface, given only sparse input points, then it probably also captures some fragments of semantic information, that can be used to boost an actual perception task. This principle has a very simple formulation, which makes it both easy to implement and widely applicable to a large range of 3D sensors and deep networks performing semantic segmentation or object detection. In fact, it supports a single-stream pipeline, as opposed to most contrastive learning approaches, allowing training on limited resources. We conducted extensive experiments on various autonomous driving datasets, involving very different kinds of lidars, for both semantic segmentation and object detection. The results show the effectiveness of our method to learn useful representations without any annotation, compared to existing approaches. Code is available at https://github.com/valeoai/ALSO