mlsanityData to 14 Oct 2026

itKD: Interchange Transfer-based Knowledge Distillation for 3D Object Detection

arXiv 2205.15531 ↗First posted 31 May 2022cs.CV cs.LGcited by 14

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

Point-cloud based 3D object detectors recently have achieved remarkable progress. However, most studies are limited to the development of network architectures for improving only their accuracy without consideration of the computational efficiency. In this paper, we first propose an autoencoder-style framework comprising channel-wise compression and decompression via interchange transfer-based knowledge distillation. To learn the map-view feature of a teacher network, the features from teacher and student networks are independently passed through the shared autoencoder; here, we use a compressed representation loss that binds the channel-wised compression knowledge from both student and teacher networks as a kind of regularization. The decompressed features are transferred in opposite directions to reduce the gap in the interchange reconstructions. Lastly, we present an head attention loss to match the 3D object detection information drawn by the multi-head self-attention mechanism. Through extensive experiments, we verify that our method can train the lightweight model that is well-aligned with the 3D point cloud detection task and we demonstrate its superiority using the well-known public datasets; e.g., Waymo and nuScenes.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 45 of its 54 references that are in the map. First posted on arXiv 31 May 2022.

Builds on

45 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Towards Efficient 3D Object Detection with Knowledge Distillation30 May

Other references · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  7. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  8. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  9. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  10. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  11. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  12. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
32 more
  1. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  2. 2019Relational Knowledge Distillationcited by 889
  3. 2019Contrastive Representation Distillationcited by 748
  4. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  5. 2015Cyclical Learning Rates for Training Neural Networkscited by 629
  6. 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
  7. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  8. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 368
  9. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  10. 2020Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloudcited by 335
  11. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  12. 2018Paraphrasing Complex Network: Network Compression via Factor Transfercited by 260
  13. 2019Distilling Object Detectors with Fine-grained Feature Imitationcited by 189
  14. 2021Dynamic Head: Unifying Object Detection Heads with Attentionscited by 187
  15. 2019End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Cloudscited by 171
  16. 2021ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detectioncited by 157
  17. 2020Pillar-based Object Detection for Autonomous Drivingcited by 137
  18. 2022Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Cloudscited by 131
  19. 2019Fast Point R-CNNcited by 119
  20. 2021SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloudcited by 119
  21. 2015Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Imagescited by 111
  22. 2021AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Cloudscited by 88
  23. 2020Densely Guided Knowledge Distillation using Multiple Teacher Assistantscited by 83
  24. 2020HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detectioncited by 82
  25. 2016Vote3Deep: Fast Object Detection in 3D Point Clouds Using Efficient Convolutional Neural Networkscited by 72
  26. 2021LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detectorcited by 70
  27. 2020AFDet: Anchor Free One Stage 3D Object Detectioncited by 63
  28. 2021Object DGCNN: 3D Object Detection using Dynamic Graphscited by 58
  29. 20163D Fully Convolutional Network for Vehicle Detection in Point Cloudcited by 57
  30. 2018Complex-YOLO: Real-time 3D Object Detection on Point Cloudscited by 57
  31. 2019StarNet: Targeted Computation for Object Detection in Point Cloudscited by 46
  32. 2021Revisiting Knowledge Distillation: An Inheritance and Exploration Frameworkcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 5; 2025: 4; 2026: 1.

14 papers.

Sort:

By area:

  1. 2024Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillationcited by 43
  2. 2024MapDistill: Boosting Efficient Camera-based HD Map Construction via Camera-LiDAR Fusion Model Distillationcited by 26
  3. 2023Deep Transfer Learning for Intelligent Vehicle Perception: a Surveycited by 10
  4. 2024LabelDistill: Label-guided Cross-modal Knowledge Distillation for Camera-based 3D Object Detectioncited by 9
  5. 2023Geometric-aware Pretraining for Vision-centric 3D Object Detectioncited by 8
  6. 2025CAML: Collaborative Auxiliary Modality Learning for Multi-Agent Systemscited by 6
  7. 2024Weak-to-Strong 3D Object Detection with X-Ray Distillationcited by 5
  8. 2022Structured Knowledge Distillation Towards Efficient and Compact Multi-View 3D Detectioncited by 3
  9. 2025Reflectance Prediction-based Knowledge Distillation for Robust 3D Object Detection in Compressed Point Cloudscited by 3
  10. 2025MMCD: Multi-Modal Collaborative Decision-Making for Connected Autonomy with Knowledge Distillationcited by 2
  11. 2022ORC: Network Group-based Knowledge Distillation using Online Role Changecited by 1
  12. 2024Leveraging Anchor-based LiDAR 3D Object Detection via Point Assisted Sample Selectioncited by 1
2 more
  1. 2025Semantic-Supervised Spatial-Temporal Fusion for LiDAR-based 3D Object Detectioncited by 1
  2. 2026Lightweight 3D Object Detection via Mamba-Based Knowledge Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection
  2. 2023Representation Disparity-aware Distillation for 3D Object Detection
  3. 20223D Point Cloud Pre-training with Knowledge Distillation from 2D Images
  4. 2023AMD: Adaptive Masked Distillation for Object Detection
  5. 2024Distilling Temporal Knowledge with Masked Feature Reconstruction for 3D Object Detection
  6. 2025CLoCKDistill: Consistent Location-and-Context-aware Knowledge Distillation for DETRs
  7. 2024Semi-Supervised 3D Object Detection with Channel Augmentation using Transformation Equivariance
  8. 2022D3ETR: Decoder Distillation for Detection Transformer