# Multi-view Contrastive Learning for Online Knowledge Distillation

arXiv: https://arxiv.org/abs/2006.04093 · HTML: https://mlsanity.com/paper/2006.04093/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2020.

## Built on by (most cited first, 11 of 11)

- [Cross-Image Relational Knowledge Distillation for Semantic Segmentation](https://mlsanity.com/paper/2204.06986/) (2022)
- [Hierarchical Self-supervised Augmented Knowledge Distillation](https://mlsanity.com/paper/2107.13715/) (2021)
- [Mutual Contrastive Learning for Visual Representation Learning](https://mlsanity.com/paper/2104.12565/) (2021)
- [MixSKD: Self-Knowledge Distillation from Mixup for Image Recognition](https://mlsanity.com/paper/2208.05768/) (2022)
- [Teacher-Student Architecture for Knowledge Distillation: A Survey](https://mlsanity.com/paper/2308.04268/) (2023)
- [Teacher-Student Architecture for Knowledge Learning: A Survey](https://mlsanity.com/paper/2210.17332/) (2022)
- [Knowledge Distillation Using Hierarchical Self-Supervision Augmented Distribution](https://mlsanity.com/paper/2109.03075/) (2021)
- [CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing](https://mlsanity.com/paper/2204.06625/) (2022)
- [Online Policy Distillation with Decision-Attention](https://mlsanity.com/paper/2406.05488/) (2024)
- [Relative Difficulty Distillation for Semantic Segmentation](https://mlsanity.com/paper/2407.03719/) (2024)
- [A General Multiple Data Augmentation Based Framework for Training Deep Neural Networks](https://mlsanity.com/paper/2205.14606/) (2022)

## Abstract (arXiv)

Previous Online Knowledge Distillation (OKD) often carries out mutually exchanging probability distributions, but neglects the useful representational knowledge. We therefore propose Multi-view Contrastive Learning (MCL) for OKD to implicitly capture correlations of feature embeddings encoded by multiple peer networks, which provide various views for understanding the input data instances. Benefiting from MCL, we can learn a more discriminative representation space for classification than previous OKD methods. Experimental results on image classification demonstrate that our MCL-OKD outperforms other state-of-the-art OKD methods by large margins without sacrificing additional inference cost. Codes are available at https://github.com/winycg/MCL-OKD.