mlsanityData to 14 Oct 2026

KD-MVS: Knowledge Distillation Based Self-supervised Learning for Multi-view Stereo

arXiv 2207.10425 ↗First posted 21 Jul 2022cs.CVcited by 18

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

Supervised multi-view stereo (MVS) methods have achieved remarkable progress in terms of reconstruction quality, but suffer from the challenge of collecting large-scale ground-truth depth. In this paper, we propose a novel self-supervised training pipeline for MVS based on knowledge distillation, termed KD-MVS, which mainly consists of self-supervised teacher training and distillation-based student training. Specifically, the teacher model is trained in a self-supervised fashion using both photometric and featuremetric consistency. Then we distill the knowledge of the teacher model to the student model through probabilistic knowledge transferring. With the supervision of validated knowledge, the student model is able to outperform its teacher by a large margin. Extensive experiments performed on multiple datasets show our method can even outperform supervised methods.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 36 of its 47 references that are in the map. First posted on arXiv 21 Jul 2022.

Builds on

36 of 47 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  5. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  6. 2020Knowledge Distillation: A Surveycited by 1,603
  7. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  8. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  9. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
  10. 2019Relational Knowledge Distillationcited by 889
  11. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  12. 2019Contrastive Representation Distillationcited by 748
24 more
  1. 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
  2. 2018Born Again Neural Networkscited by 546
  3. 2019Similarity-Preserving Knowledge Distillationcited by 516
  4. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 490
  5. 2019Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matchingcited by 417
  6. 2019Correlation Congruence for Knowledge Distillationcited by 285
  7. 2019Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inferencecited by 254
  8. 2018Learning Deep Representations with Probabilistic Knowledge Transfercited by 213
  9. 2019Cost Volume Pyramid Based Depth Inference for Multi-View Stereocited by 154
  10. 2019Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awarenesscited by 151
  11. 2021TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformerscited by 117
  12. 2019Multi-Scale Geometric Consistency Guided Multi-View Stereocited by 105
  13. 2020Visibility-aware Multi-view Stereo Networkcited by 98
  14. 2020Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checkingcited by 81
  15. 2021AA-RMVSNet: Adaptive Aggregation Recurrent Multi-view Stereo Networkcited by 71
  16. 2019Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistencycited by 40
  17. 2021Self-supervised Multi-view Stereo via Effective Co-Segmentation and Data-Augmentationcited by 31
  18. 2021Digging into Uncertainty in Self-supervised Multi-view Stereocited by 31
  19. 2019MVS^2: Deep Unsupervised Multi-view Stereo with Multi-View Symmetrycited by 30
  20. 2020M^3VSNet: Unsupervised Multi-metric Multi-view Stereo Networkcited by 29
  21. 2021Self-supervised Learning of Depth Inference for Multi-view Stereocited by 28
  22. 2022WT-MVSNet: Window-based Transformers for Multi-view Stereocited by 20
  23. 2021Deep Multi-View Stereo gone wildcited by 11
  24. 2022Enhancing Multi-view Stereo with Contrastive Matching and Weighted Focal Losscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 7; 2025: 3; 2026: 4.

18 papers.

Sort:

  1. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 220
  2. 2024MoCha-Stereo: Motif Channel Attention Network for Stereo Matchingcited by 48
  3. 2023C2F2NeUS: Cascade Cost Frustum Fusion for High Fidelity and Generalizable Neural Surface Reconstructioncited by 18
  4. 2024Learning-based Multi-View Stereo: A Surveycited by 18
  5. 2024Adapting Fine-Grained Cross-View Localization to Areas without Fine Ground Truthcited by 14
  6. 2024Geometry Field Splatting with Gaussian Surfelscited by 13
  7. 2024GRAPE: Generalizable and Robust Multi-view Facial Capturecited by 6
  8. 2026Distill3R: A Pipeline for Democratizing 3D Foundation Models on Commodity Hardwarecited by 3
  9. 2025RoSe: Robust Self-supervised Stereo Matching under Adverse Weather Conditionscited by 2
  10. 2026SMFormer: Empowering Self-supervised Stereo Matching via Foundation Models and Data Augmentationcited by 2
  11. 2023ES-MVSNet: Efficient Framework for End-to-end Self-supervised Multi-View Stereocited by 1
  12. 2024EV-MGDispNet: Motion-Guided Event-Based Stereo Disparity Estimation Network with Left-Right Consistencycited by 1
6 more
  1. 2023Exploring the Mutual Influence between Self-Supervised Single-Frame and Multi-Frame Depth Estimationcited by 0
  2. 2024Prism: Semi-Supervised Multi-View Stereo with Monocular Structure Priorscited by 0
  3. 2025Boosting Multi-View Stereo with Depth Foundation Model in the Absence of Real-World Labelscited by 0
  4. 2025MAGIC: Learning from Visibility Asymmetry for Unsupervised Stereo Matchingcited by 0
  5. 2026BINO: Encoder Centric Self Supervised Stereo With Native Pair Inputcited by 0
  6. 2026OTT3R: Multi-View 3D Reconstruction and Fast Dataset Generation at 1% Computecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Distilling Stereo Networks for Performant and Efficient Leaner Networks
  2. 2024Stereo-Matching Knowledge Distilled Monocular Depth Estimation Filtered by Multiple Disparity Consistency
  3. 2022Dense Depth Distillation with Out-of-Distribution Simulated Images
  4. 2024Boosting Self-Supervision for Single-View Scene Completion via Knowledge Distillation
  5. 2024TIE-KD: Teacher-Independent and Explainable Knowledge Distillation for Monocular Depth Estimation
  6. 2023Representation Disparity-aware Distillation for 3D Object Detection
  7. 2025DMS:Diffusion-Based Multi-Baseline Stereo Generation for Improving Self-Supervised Depth Estimation
  8. 2024Mono2Stereo: Monocular Knowledge Transfer for Enhanced Stereo Matching