mlsanityData to 7 Oct 2026

Boosting 3D Object Detection by Simulating Multimodality on Point Clouds

arXiv 2206.14971 ↗First posted 30 Jun 2022cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

This paper presents a new approach to boost a single-modality (LiDAR) 3D object detector by teaching it to simulate features and responses that follow a multi-modality (LiDAR-image) detector. The approach needs LiDAR-image data only when training the single-modality detector, and once well-trained, it only needs LiDAR data at inference. We design a novel framework to realize the approach: response distillation to focus on the crucial response samples and avoid the background samples; sparse-voxel distillation to learn voxel semantics and relations from the estimated crucial voxels; a fine-grained voxel-to-point distillation to better attend to features of small and distant objects; and instance distillation to further enhance the deep-feature consistency. Experimental results on the nuScenes dataset show that our approach outperforms all SOTA LiDAR-only 3D detectors and even surpasses the baseline LiDAR-image detector on the key NDS metric, filling 72% mAP gap between the single- and multi-modality detectors.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 50 of its 57 references that are in the map. First posted on arXiv 30 Jun 2022.

Builds on

50 of 57 references.

References · 50

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  4. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  5. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  6. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  7. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  8. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  9. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  10. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  11. 2019Similarity-Preserving Knowledge Distillationcited by 507
  12. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
38 more
  1. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  2. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 366
  3. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  4. 2019A Comprehensive Overhaul of Feature Distillationcited by 341
  5. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  6. 2021Distilling Knowledge via Knowledge Reviewcited by 285
  7. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  8. 2021Voxel Transformer for 3D Object Detectioncited by 247
  9. 2021PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detectioncited by 232
  10. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  11. 2019Distilling Object Detectors with Fine-grained Feature Imitationcited by 188
  12. 20203D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detectioncited by 176
  13. 2019Stereo R-CNN based 3D Object Detection for Autonomous Drivingcited by 163
  14. 2020CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detectioncited by 159
  15. 2020CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detectioncited by 147
  16. 2021RangeDet:In Defense of Range View for LiDAR-based 3D Object Detectioncited by 140
  17. 2021Delving into Localization Errors for Monocular 3D Object Detectioncited by 138
  18. 2019Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detectioncited by 124
  19. 2021SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloudcited by 119
  20. 2021General Instance Distillation for Object Detectioncited by 113
  21. 2019TANet: Robust 3D Object Detection from Point Clouds with Triple Attentioncited by 108
  22. 2021Improving 3D Object Detection with Channel-wise Transformercited by 108
  23. 2020CIA-SSD: Confident IoU-Aware Single-Stage Object Detector From Point Cloudcited by 107
  24. 2021Distilling Object Detectors via Decoupled Featurescited by 105
  25. 2020DSGN: Deep Stereo Geometry Network for 3D Object Detectioncited by 100
  26. 2021AutoShape: Real-Time Shape-Aware Monocular 3D Object Detectioncited by 87
  27. 2021Geometry-based Distance Decomposition for Monocular 3D Object Detectioncited by 83
  28. 2021Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detectioncited by 79
  29. 2020SSN: Shape Signature Networks for Multi-class Object Detection from Point Cloudscited by 74
  30. 2020A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detectioncited by 70
  31. 2019What You See is What You Get: Exploiting Visibility for 3D Object Detectioncited by 69
  32. 2020LiDAR-based Online 3D Video Object Detection with Graph-based Message Passing and Spatiotemporal Transformer Attentioncited by 65
  33. 20213D-to-2D Distillation for Indoor Scene Parsingcited by 47
  34. 2019Distant Vehicle Detection Using Radar and Visioncited by 44
  35. 2021There is More than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking with Sound by Distilling Multimodal Knowledgecited by 44
  36. 2021M3DSSD: Monocular 3D Single Stage Object Detectorcited by 39
  37. 2020Distilling Object Detectors with Task Adaptive Regularizationcited by 36
  38. 2021Multi-Scale Aligned Distillation for Low-Resolution Detectioncited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 6; 2024: 6; 2025: 4; 2026: 2.

22 papers.

Sort:

By area:

  1. 2023OccDepth: A Depth-Aware Method for 3D Semantic Scene Completioncited by 85
  2. 2023UniDistill: A Universal Cross-Modality Knowledge Distillation Framework for 3D Object Detection in Bird's-Eye Viewcited by 43
  3. 2023GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for Multi-Modal 3D Object Detectioncited by 34
  4. 2024RadarDistill: Boosting Radar-based Object Detection Performance via Knowledge Distillation from LiDAR Featurescited by 27
  5. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  6. 2023Self-supervised Learning for Pre-Training 3D Point Clouds: A Surveycited by 21
  7. 2022MGTANet: Encoding Sequential LiDAR Points Using Long Short-Term Motion-Guided Temporal Attention for 3D Object Detectioncited by 15
  8. 2023SupFusion: Supervised LiDAR-Camera Fusion for 3D Object Detectioncited by 14
  9. 2024VoxelNextFusion: A Simple, Unified and Effective Voxel Fusion Framework for Multi-Modal 3D Object Detectioncited by 14
  10. 2023Robust 3D Object Detection from LiDAR-Radar Point Clouds via Cross-Modal Feature Augmentationcited by 12
  11. 2024Sunshine to Rainstorm: Cross-Weather Knowledge Distillation for Robust 3D Object Detectioncited by 11
  12. 2024LIX: Implicitly Infusing Spatial Geometric Prior Knowledge into Visual Semantic Segmentation for Autonomous Drivingcited by 8
10 more
  1. 2022Boosting Single-Frame 3D Object Detection by Simulating Multi-Frame Point Cloudscited by 6
  2. 2025RCTDistill: Cross-Modal Knowledge Distillation Framework for Radar-Camera 3D Object Detection with Temporal Fusioncited by 3
  3. 2025CMF-IoU: Multi-Stage Cross-Modal Fusion 3D Object Detection with IoU Joint Predictioncited by 2
  4. 2025Towards a 3D Transfer-based Black-box Attack via Critical Feature Guidancecited by 2
  5. 2022D-Align: Dual Query Co-attention Network for 3D Object Detection Based on Multi-frame Point Cloud Sequencecited by 1
  6. 2024Sparse Point Clouds Assisted Learned Image Compressioncited by 1
  7. 2026C2E: Boosting Ego-Only 3D Object Detection via Multi-Teacher Contrastive Knowledge Distillationcited by 1
  8. 2022ImLiDAR: Cross-Sensor Dynamic Message Propagation Network for 3D Object Detectioncited by 0
  9. 2025TransBridge: Boost 3D Object Detection by Scene-Level Completion with Transformer Decodercited by 0
  10. 2026Heterogeneous and Adept Snapshot Distillation for 3D Semantic Segmentationcited by 0