mlsanityData to 14 Oct 2026

Bidirectional Knowledge Reconfiguration for Lightweight Point Cloud Analysis

arXiv 2310.05125 ↗First posted 8 Oct 2023cs.CVcited by 0

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

Point cloud analysis faces computational system overhead, limiting its application on mobile or edge devices. Directly employing small models may result in a significant drop in performance since it is difficult for a small model to adequately capture local structure and global shape information simultaneously, which are essential clues for point cloud analysis. This paper explores feature distillation for lightweight point cloud models. To mitigate the semantic gap between the lightweight student and the cumbersome teacher, we propose bidirectional knowledge reconfiguration (BKR) to distill informative contextual knowledge from the teacher to the student. Specifically, a top-down knowledge reconfiguration and a bottom-up knowledge reconfiguration are developed to inherit diverse local structure information and consistent global shape knowledge from the teacher, respectively. However, due to the farthest point sampling in most point cloud models, the intermediate features between teacher and student are misaligned, deteriorating the feature distillation performance. To eliminate it, we propose a feature mover's distance (FMD) loss based on optimal transportation, which can measure the distance between unordered point cloud features effectively. Extensive experiments conducted on shape classification, part segmentation, and semantic segmentation benchmarks demonstrate the universality and superiority of our method.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 46 of its 65 references that are in the map. First posted on arXiv 8 Oct 2023.

Builds on

46 of 65 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Structural and Statistical Texture Knowledge Distillation for Semantic Segmentation6 May
  2. 2022Cross-Image Relational Knowledge Distillation for Semantic Segmentation14 Apr
  3. 2021Knowledge Distillation for Object Detection via Rank Mimicking and Prediction-guided Feature Imitation9 Dec

Other references · 43

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  3. 2017Squeeze-and-Excitation Networkscited by 4,789
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  5. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  6. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  7. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  8. 2020Knowledge Distillation: A Surveycited by 1,603
  9. 2020Point Transformercited by 1,470
  10. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  11. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  12. 2018Path Aggregation Network for Instance Segmentationcited by 798
31 more
  1. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 657
  2. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 585
  3. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 565
  4. 2019Similarity-Preserving Knowledge Distillationcited by 516
  5. 2019Selective Kernel Networkscited by 403
  6. 2022Decoupled Knowledge Distillationcited by 388
  7. 2019A Comprehensive Overhaul of Feature Distillationcited by 348
  8. 2021Distilling Knowledge via Knowledge Reviewcited by 288
  9. 2019Style Transfer by Relaxed Optimal Transport and Self-Similaritycited by 168
  10. 2020Cross-Layer Distillation with Semantic Calibrationcited by 143
  11. 2022Knowledge Distillation with the Reused Teacher Classifiercited by 141
  12. 2019DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processingcited by 104
  13. 2021Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognitioncited by 89
  14. 2019Learning with minibatch Wasserstein : asymptotic and gradient propertiescited by 87
  15. 2021Show, Attend and Distill:Knowledge Distillation via Attention-based Feature Matchingcited by 79
  16. 2019SampleNet: Differentiable Point Cloud Samplingcited by 74
  17. 2021PointMixer: MLP-Mixer for Point Cloud Understandingcited by 71
  18. 2020Distributional Sliced-Wasserstein and Applications to Generative Modelingcited by 69
  19. 2021Adaptive Graph Convolution for Point Cloud Analysiscited by 60
  20. 2019AugFPN: Improving Multi-scale Feature Learning for Object Detectioncited by 55
  21. 2019Dense 3D Point Cloud Reconstruction Using a Deep Pyramid Networkcited by 33
  22. 2022PointConvFormer: Revenge of the Point-based Convolutioncited by 30
  23. 2022Improved Feature Distillation via Projector Ensemblecited by 27
  24. 2021Dual Transformer for Point Cloud Analysiscited by 26
  25. 2021DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover's Distance Improves Out-Of-Distribution Face Identificationcited by 20
  26. 2021On Transportation of Mini-batches: A Hierarchical Approachcited by 16
  27. 2022CDFKD-MFS: Collaborative Data-free Knowledge Distillation via Multi-level Feature Sharingcited by 15
  28. 2022PetsGAN: Rethinking Priors for Single Image Generationcited by 14
  29. 2022One Loss for Quantization: Deep Hashing with Discrete Wasserstein Distributional Matchingcited by 9
  30. 2021A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statisticscited by 6
  31. 2022Exponential Convergence of Sinkhorn Under Regularization Schedulingcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Topology-Guided Knowledge Distillation for Efficient Point Cloud Processing
  2. 2025SRKD: Towards Efficient 3D Point Cloud Segmentation via Structure- and Relation-aware Knowledge Distillation
  3. 2022Distillation with Contrast is All You Need for Self-Supervised Point Cloud Representation Learning
  4. 2025CasPoinTr: Point Cloud Completion with Cascaded Networks and Knowledge Distillation
  5. 2023Multi-to-Single Knowledge Distillation for Point Cloud Semantic Segmentation
  6. 2022PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection
  7. 2024HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation
  8. 2024Efficient Point Cloud Classification via Offline Distillation Framework and Negative-Weight Self-Distillation Technique