# SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detection

arXiv: https://arxiv.org/abs/2201.01976 · HTML: https://mlsanity.com/paper/2201.01976/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 31 references that are in the map.
- First posted on arXiv 6 Jan 2022.

## Builds on (most cited first, 27 of 27)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [STD: Sparse-to-Dense 3D Object Detector for Point Cloud](https://mlsanity.com/paper/1907.10471/) (2019)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [3D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detection](https://mlsanity.com/paper/2004.12636/) (2020)
- [Modeling Point Clouds with Self-Attention and Gumbel Subset Sampling](https://mlsanity.com/paper/1904.03375/) (2019)
- [Fast Point R-CNN](https://mlsanity.com/paper/1908.02990/) (2019)
- [Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images](https://mlsanity.com/paper/1511.02300/) (2015)
- [Empowering Things with Intelligence: A Survey of the Progress, Challenges, and Opportunities in Artificial Intelligence of Things](https://mlsanity.com/paper/2011.08612/) (2020)
- [TANet: Robust 3D Object Detection from Point Clouds with Triple Attention](https://mlsanity.com/paper/1912.05163/) (2019)
- [CIA-SSD: Confident IoU-Aware Single-Stage Object Detector From Point Cloud](https://mlsanity.com/paper/2012.03015/) (2020)
- [SampleNet: Differentiable Point Cloud Sampling](https://mlsanity.com/paper/1912.03663/) (2019)
- [Adaptive Hierarchical Down-Sampling for Point Cloud Classification](https://mlsanity.com/paper/1904.08506/) (2019)
- [Progressive LiDAR Adaptation for Road Detection](https://mlsanity.com/paper/1904.01206/) (2019)
- [BirdNet: a 3D Object Detection Framework from LiDAR information](https://mlsanity.com/paper/1805.01195/) (2018)

## Built on by (most cited first, 30 of 52)

- [GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Clouds](https://mlsanity.com/paper/2212.03010/) (2022)
- [Contrastive Boundary Learning for Point Cloud Segmentation](https://mlsanity.com/paper/2203.05272/) (2022)
- [Graph R-CNN: Towards Accurate 3D Object Detection with Semantic-Decorated Local Graph](https://mlsanity.com/paper/2208.03624/) (2022)
- [LION: Linear Group RNN for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/2407.18232/) (2024)
- [SSDA3D: Semi-supervised Domain Adaptation for 3D Object Detection from Point Cloud](https://mlsanity.com/paper/2212.02845/) (2022)
- [OcTr: Octree-based Transformer for 3D Object Detection](https://mlsanity.com/paper/2303.12621/) (2023)
- [DBQ-SSD: Dynamic Ball Query for Efficient 3D Object Detection](https://mlsanity.com/paper/2207.10909/) (2022)
- [A Comprehensive Study of the Robustness for LiDAR-based 3D Object Detectors against Adversarial Attacks](https://mlsanity.com/paper/2212.10230/) (2022)
- [PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer](https://mlsanity.com/paper/2305.06621/) (2023)
- [SEED: A Simple and Effective 3D DETR in Point Clouds](https://mlsanity.com/paper/2407.10749/) (2024)
- [Human-centric Scene Understanding for 3D Large-scale Scenarios](https://mlsanity.com/paper/2307.14392/) (2023)
- [All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation](https://mlsanity.com/paper/2305.15832/) (2023)
- [Unleash the Potential of Image Branch for Cross-modal 3D Object Detection](https://mlsanity.com/paper/2301.09077/) (2023)
- [CP3: Channel Pruning Plug-in for Point-based Networks](https://mlsanity.com/paper/2303.13097/) (2023)
- [UniMamba: Unified Spatial-Channel Representation Learning with Group-Efficient Mamba for LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2503.12009/) (2025)
- [Paint and Distill: Boosting 3D Object Detection with Semantic Passing Network](https://mlsanity.com/paper/2207.05497/) (2022)
- [SparseDet: A Simple and Effective Framework for Fully Sparse LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2406.10907/) (2024)
- [STAL3D: Unsupervised Domain Adaptation for 3D Object Detection via Collaborating Self-Training and Adversarial Learning](https://mlsanity.com/paper/2406.19362/) (2024)
- [PointSee: Image Enhances Point Cloud](https://mlsanity.com/paper/2211.01664/) (2022)
- [Fine-grained Image-to-LiDAR Contrastive Distillation with Visual Foundation Models](https://mlsanity.com/paper/2405.14271/) (2024)
- [GO-N3RDet: Geometry Optimized NeRF-enhanced 3D Object Detector](https://mlsanity.com/paper/2503.15211/) (2025)
- [Multi-Modal 3D Object Detection by Box Matching](https://mlsanity.com/paper/2305.07713/) (2023)
- [PV-RCNN++: Semantical Point-Voxel Feature Interaction for 3D Object Detection](https://mlsanity.com/paper/2208.13414/) (2022)
- [Is Contrastive Distillation Enough for Learning Comprehensive 3D Representations?](https://mlsanity.com/paper/2412.08973/) (2024)
- [Reflectance Prediction-based Knowledge Distillation for Robust 3D Object Detection in Compressed Point Clouds](https://mlsanity.com/paper/2505.17442/) (2025)
- [UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs](https://mlsanity.com/paper/2511.01768/) (2025)
- [OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System](https://mlsanity.com/paper/2303.00501/) (2023)
- [IC-FPS: Instance-Centroid Faster Point Sampling Module for 3D Point-base Object Detection](https://mlsanity.com/paper/2303.17921/) (2023)
- [LiDAR-Based 3D Object Detection via Hybrid 2D Semantic Scene Generation](https://mlsanity.com/paper/2304.01519/) (2023)
- [SEFormer: Structure Embedding Transformer for 3D Object Detection](https://mlsanity.com/paper/2209.01745/) (2022)

## Abstract (arXiv)

Although point-based networks are demonstrated to be accurate for 3D point cloud modeling, they are still falling behind their voxel-based competitors in 3D detection. We observe that the prevailing set abstraction design for down-sampling points may maintain too much unimportant background information that can affect feature learning for detecting objects. To tackle this issue, we propose a novel set abstraction method named Semantics-Augmented Set Abstraction (SASA). Technically, we first add a binary segmentation module as the side output to help identify foreground points. Based on the estimated point-wise foreground scores, we then propose a semantics-guided point sampling algorithm to help retain more important foreground points during down-sampling. In practice, SASA shows to be effective in identifying valuable points related to foreground objects and improving feature learning for point-based 3D detection. Additionally, it is an easy-to-plug-in module and able to boost various point-based detectors, including single-stage and two-stage ones. Extensive experiments on the popular KITTI and nuScenes datasets validate the superiority of SASA, lifting point-based detection models to reach comparable performance to state-of-the-art voxel-based methods.