# Boosting Single-Frame 3D Object Detection by Simulating Multi-Frame Point Clouds

arXiv: https://arxiv.org/abs/2207.01030 · HTML: https://mlsanity.com/paper/2207.01030/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 62 references that are in the map.
- First posted on arXiv 3 Jul 2022.

## Builds on (most cited first, 30 of 57)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Similarity-Preserving Knowledge Distillation](https://mlsanity.com/paper/1907.09682/) (2019)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [A Comprehensive Overhaul of Feature Distillation](https://mlsanity.com/paper/1904.01866/) (2019)
- [Categorical Depth Distribution Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.01100/) (2021)
- [Distilling Knowledge via Knowledge Review](https://mlsanity.com/paper/2104.09044/) (2021)
- [STD: Sparse-to-Dense 3D Object Detector for Point Cloud](https://mlsanity.com/paper/1907.10471/) (2019)
- [PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection](https://mlsanity.com/paper/2102.00463/) (2021)
- [PU-Net: Point Cloud Upsampling Network](https://mlsanity.com/paper/1801.06761/) (2018)
- [Distilling Object Detectors with Fine-grained Feature Imitation](https://mlsanity.com/paper/1906.03609/) (2019)
- [3D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detection](https://mlsanity.com/paper/2004.12636/) (2020)
- [GRNet: Gridding Residual Network for Dense Point Cloud Completion](https://mlsanity.com/paper/2006.03761/) (2020)
- [PU-GAN: a Point Cloud Upsampling Adversarial Network](https://mlsanity.com/paper/1907.10844/) (2019)
- [CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2011.04841/) (2020)
- [Offboard 3D Object Detection from Point Cloud Sequences](https://mlsanity.com/paper/2103.05073/) (2021)
- [RangeDet:In Defense of Range View for LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2103.10039/) (2021)
- [Delving into Localization Errors for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.16237/) (2021)
- [Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection](https://mlsanity.com/paper/1903.01864/) (2019)

## Built on by (most cited first, 6 of 6)

- [X3KD: Knowledge Distillation Across Modalities, Tasks and Stages for Multi-Camera 3D Object Detection](https://mlsanity.com/paper/2303.02203/) (2023)
- [A Survey of Label-Efficient Deep Learning for 3D Point Clouds](https://mlsanity.com/paper/2305.19812/) (2023)
- [TASeg: Temporal Aggregation Network for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2407.09751/) (2024)
- [Sunshine to Rainstorm: Cross-Weather Knowledge Distillation for Robust 3D Object Detection](https://mlsanity.com/paper/2402.18493/) (2024)
- [LIX: Implicitly Infusing Spatial Geometric Prior Knowledge into Visual Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2403.08215/) (2024)
- [C2E: Boosting Ego-Only 3D Object Detection via Multi-Teacher Contrastive Knowledge Distillation](https://mlsanity.com/paper/2607.01827/) (2026)

## Abstract (arXiv)

To boost a detector for single-frame 3D object detection, we present a new approach to train it to simulate features and responses following a detector trained on multi-frame point clouds. Our approach needs multi-frame point clouds only when training the single-frame detector, and once trained, it can detect objects with only single-frame point clouds as inputs during the inference. We design a novel Simulated Multi-Frame Single-Stage object Detector (SMF-SSD) framework to realize the approach: multi-view dense object fusion to densify ground-truth objects to generate a multi-frame point cloud; self-attention voxel distillation to facilitate one-to-many knowledge transfer from multi- to single-frame voxels; multi-scale BEV feature distillation to transfer knowledge in low-level spatial and high-level semantic BEV features; and adaptive response distillation to activate single-frame responses of high confidence and accurate localization. Experimental results on the Waymo test set show that our SMF-SSD consistently outperforms all state-of-the-art single-frame 3D object detectors for all object classes of difficulty levels 1 and 2 in terms of both mAP and mAPH.