# Hardness-Aware Scene Synthesis for Semi-Supervised 3D Object Detection

arXiv: https://arxiv.org/abs/2405.17422 · HTML: https://mlsanity.com/paper/2405.17422/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 57 references that are in the map.
- First posted on arXiv 27 May 2024.

## Builds on (most cited first, 30 of 38)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning](https://mlsanity.com/paper/1606.04586/) (2016)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [PIXOR: Real-time 3D Object Detection from Point Clouds](https://mlsanity.com/paper/1902.06326/) (2019)
- [End-to-End Semi-Supervised Object Detection with Soft Teacher](https://mlsanity.com/paper/2106.09018/) (2021)
- [A Simple Semi-Supervised Learning Framework for Object Detection](https://mlsanity.com/paper/2005.04757/) (2020)
- [Semi-Supervised Learning with Ladder Networks](https://mlsanity.com/paper/1507.02672/) (2015)
- [Learning with Pseudo-Ensembles](https://mlsanity.com/paper/1412.4864/) (2014)
- [Multi-Task Multi-Sensor Fusion for 3D Object Detection](https://mlsanity.com/paper/2012.12397/) (2020)
- [SelfOcc: Self-Supervised Vision-Based 3D Occupancy Prediction](https://mlsanity.com/paper/2311.12754/) (2023)
- [LaserMix for Semi-Supervised LiDAR Semantic Segmentation](https://mlsanity.com/paper/2207.00026/) (2022)
- [SESS: Self-Ensembling Semi-Supervised 3D Object Detection](https://mlsanity.com/paper/1912.11803/) (2019)
- [3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection](https://mlsanity.com/paper/2012.04355/) (2020)
- [PointCutMix: Regularization Strategy for Point Cloud Classification](https://mlsanity.com/paper/2101.01461/) (2021)
- [Weakly Supervised 3D Object Detection from Lidar Point Cloud](https://mlsanity.com/paper/2007.11901/) (2020)
- [PointAugment: an Auto-Augmentation Framework for Point Cloud Classification](https://mlsanity.com/paper/2002.10876/) (2020)
- [Complex-YOLO: Real-time 3D Object Detection on Point Clouds](https://mlsanity.com/paper/1803.06199/) (2018)
- [Semi-supervised 3D Object Detection with Proficient Teachers](https://mlsanity.com/paper/2207.12655/) (2022)
- [Augmented LiDAR Simulator for Autonomous Driving](https://mlsanity.com/paper/1811.07112/) (2018)
- [RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection](https://mlsanity.com/paper/2108.07794/) (2021)
- [Mix-Teaching: A Simple, Unified and Effective Semi-Supervised Learning Framework for Monocular 3D Object Detection](https://mlsanity.com/paper/2207.04448/) (2022)
- [AD-PT: Autonomous Driving Pre-Training with Large-scale Point Cloud Dataset](https://mlsanity.com/paper/2306.00612/) (2023)
- [Generative Modeling for Small-Data Object Detection](https://mlsanity.com/paper/1910.07169/) (2019)

## Built on by (most cited first, 7 of 7)

- [AccidentSim: Generating Vehicle Collision Videos with Physically Realistic Collision Trajectories from Real-World Accident Reports](https://mlsanity.com/paper/2503.20654/) (2025)
- [DA-Occ: Direction-Aware 2D Convolution for Efficient and Geometry-Preserving 3D Occupancy Prediction in Autonomous Driving](https://mlsanity.com/paper/2507.23599/) (2025)
- [GeoTeacher: Geometry-Guided Semi-Supervised 3D Object Detection](https://mlsanity.com/paper/2512.23147/) (2025)
- [UniDrive: Towards Universal Driving Perception Across Camera Configurations](https://mlsanity.com/paper/2410.13864/) (2024)
- [Adaptive Multi-Scale Channel-Spatial Attention Aggregation Framework for 3D Indoor Semantic Scene Completion Toward Assisting Visually Impaired](https://mlsanity.com/paper/2602.16385/) (2026)
- [Unified Unsupervised and Sparsely-Supervised 3D Object Detection by Semantic Pseudo-Labeling and Prototype Learning](https://mlsanity.com/paper/2602.21484/) (2026)
- [Synthetic-to-Real Translation for Class-Agnostic Motion Prediction](https://mlsanity.com/paper/2607.06319/) (2026)

## Abstract (arXiv)

3D object detection aims to recover the 3D information of concerning objects and serves as the fundamental task of autonomous driving perception. Its performance greatly depends on the scale of labeled training data, yet it is costly to obtain high-quality annotations for point cloud data. While conventional methods focus on generating pseudo-labels for unlabeled samples as supplements for training, the structural nature of 3D point cloud data facilitates the composition of objects and backgrounds to synthesize realistic scenes. Motivated by this, we propose a hardness-aware scene synthesis (HASS) method to generate adaptive synthetic scenes to improve the generalization of the detection models. We obtain pseudo-labels for unlabeled objects and generate diverse scenes with different compositions of objects and backgrounds. As the scene synthesis is sensitive to the quality of pseudo-labels, we further propose a hardness-aware strategy to reduce the effect of low-quality pseudo-labels and maintain a dynamic pseudo-database to ensure the diversity and quality of synthetic scenes. Extensive experimental results on the widely used KITTI and Waymo datasets demonstrate the superiority of the proposed HASS method, which outperforms existing semi-supervised learning methods on 3D object detection. Code: https://github.com/wzzheng/HASS.