mlsanityData to 7 Oct 2026

Mutual Contrastive Learning for Visual Representation Learning

arXiv 2104.12565 ↗First posted 26 Apr 2021cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

We present a collaborative learning method called Mutual Contrastive Learning (MCL) for general visual representation learning. The core idea of MCL is to perform mutual interaction and transfer of contrastive distributions among a cohort of networks. A crucial component of MCL is Interactive Contrastive Learning (ICL). Compared with vanilla contrastive learning, ICL can aggregate cross-network embedding information and maximize the lower bound to the mutual information between two networks. This enables each network to learn extra contrastive knowledge from others, leading to better feature representations for visual recognition tasks. We emphasize that the resulting MCL is conceptually simple yet empirically powerful. It is a generic framework that can be applied to both supervised and self-supervised representation learning. Experimental results on image classification and transfer learning to object detection show that MCL can lead to consistent performance gains, demonstrating that MCL can guide the network to generate better feature representations. Code is available at https://github.com/winycg/MCL.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 26 Apr 2021.

Builds on

43 of 58 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2016Densely Connected Convolutional Networkscited by 6,691
  8. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  9. 2016Wide Residual Networkscited by 3,057
  10. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  11. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  12. 2020Supervised Contrastive Learningcited by 2,803
31 more
  1. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  2. 2016Matching Networks for One Shot Learningcited by 2,246
  3. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  4. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  5. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  6. 2019Contrastive Multiview Codingcited by 1,297
  7. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  8. 2016Colorful Image Colorizationcited by 1,113
  9. 2019Relational Knowledge Distillationcited by 873
  10. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  11. 2017In Defense of the Triplet Loss for Person Re-Identificationcited by 784
  12. 2017Deep Mutual Learningcited by 621
  13. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
  14. 2019Correlation Congruence for Knowledge Distillationcited by 280
  15. 2020Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identificationcited by 214
  16. 2018Knowledge Distillation by On-the-Fly Native Ensemblecited by 186
  17. 2021SEED: Self-supervised Distillation For Visual Representationcited by 132
  18. 2019Online Knowledge Distillation with Diverse Peerscited by 127
  19. 2020Video Playback Rate Perception for Self-supervisedSpatio-Temporal Representation Learningcited by 103
  20. 2020Video Cloze Procedure for Self-Supervised Spatio-Temporal Learningcited by 80
  21. 2019Ranked List Loss for Deep Metric Learningcited by 77
  22. 2018Collaborative Learning for Deep Neural Networkscited by 69
  23. 2020Peer Collaborative Learning for Online Knowledge Distillationcited by 58
  24. 2019MutualNet: Adaptive ConvNet via Mutual Learning from Network Width and Resolutioncited by 36
  25. 2020Knowledge Transfer via Dense Cross-Layer Mutual-Distillationcited by 32
  26. 2020Online Ensemble Model Compression using Knowledge Distillationcited by 16
  27. 2019Gated Convolutional Networks with Hybrid Connectivity for Image Classificationcited by 14
  28. 2018Deep Metric Learning by Online Soft Mining and Class-Aware Attentioncited by 12
  29. 2020Multi-view Contrastive Learning for Online Knowledge Distillationcited by 11
  30. 2020Filter Grafting for Deep Neural Networkscited by 8
  31. 2020Learning to Learn Parameterized Classification Networks for Scalable Input Imagescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 10; 2023: 10; 2024: 14; 2025: 6; 2026: 1.

42 papers.

Sort:

By area:

  1. 2022Cross-Image Relational Knowledge Distillation for Semantic Segmentationcited by 117
  2. 2023CLIP-KD: An Empirical Study of CLIP Model Distillationcited by 87
  3. 2024PromptKD: Unsupervised Prompt Distillation for Vision-Language Modelscited by 72
  4. 2025Frequency-Aligned Knowledge Distillation for Lightweight Spatiotemporal Forecastingcited by 53
  5. 2022Exploring Patch-wise Semantic Relation for Contrastive Learning in Image-to-Image Translation Taskscited by 42
  6. 2022FedX: Unsupervised Federated Learning with Cross Knowledge Distillationcited by 40
  7. 2022Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognitioncited by 36
  8. 2022MixSKD: Self-Knowledge Distillation from Mixup for Image Recognitioncited by 30
  9. 2022PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddingscited by 29
  10. 2023Multi-Mode Online Knowledge Distillation for Self-Supervised Visual Representation Learningcited by 20
  11. 2023Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillationcited by 19
  12. 2024Cascade Prompt Learning for Vision-Language Model Adaptationcited by 19
30 more
  1. 2025Merlin: Multi-View Representation Learning for Robust Multivariate Time Series Forecasting with Unfixed Missing Ratescited by 19
  2. 2021Knowledge Distillation Using Hierarchical Self-Supervision Augmented Distributioncited by 13
  3. 2022Supervised Contrastive Learning with Hard Negative Samplescited by 13
  4. 2025SRKD: Towards Efficient 3D Point Cloud Segmentation via Structure- and Relation-aware Knowledge Distillationcited by 9
  5. 2022miCSE: Mutual Information Contrastive Learning for Low-shot Sentence Embeddingscited by 7
  6. 2024PolyCL: Contrastive Learning for Polymer Representation Learning via Explicit and Implicit Augmentationscited by 7
  7. 2022Global Contrastive Batch Sampling via Optimization on Sample Permutationscited by 5
  8. 2024MutDet: Mutually Optimizing Pre-training for Remote Sensing Object Detectioncited by 5
  9. 2025Test-Time Distillation for Continual Model Adaptationcited by 5
  10. 2023Generalized Category Discovery with Clustering Assignment Consistencycited by 4
  11. 2023Knowledge Graph Error Detection with Contrastive Confidence Adaptioncited by 4
  12. 2024QCS: Feature Refining from Quadruplet Cross Similarity for Facial Expression Recognitioncited by 3
  13. 2023Unsupervised Synthetic Image Refinement via Contrastive Learning and Consistent Semantic-Structural Constraintscited by 2
  14. 2024Distillation Enhanced Time Series Forecasting Network with Momentum Contrastive Learningcited by 2
  15. 2024Online Policy Distillation with Decision-Attentioncited by 2
  16. 2024Relative Difficulty Distillation for Semantic Segmentationcited by 2
  17. 2025TUNI: Unifying Pre-training and Fine-tuning with Modality-Aware Mutual Learning and Rectification for RGB-T Semantic Segmentationcited by 2
  18. 2026Toward Enhancing Representation Learning in Federated Multi-Task Settingscited by 2
  19. 2023Contrastive Learning-based Imputation-Prediction Networks for In-hospital Mortality Risk Modeling using EHRscited by 1
  20. 2024Prototype-Driven Multi-Feature Generation for Visible-Infrared Person Re-identificationcited by 1
  21. 2024Online Multi-level Contrastive Representation Distillation for Cross-Subject fNIRS Emotion Recognitioncited by 1
  22. 2024Combating Semantic Contamination in Learning with Label Noisecited by 1
  23. 2024ECG-guided individual identification via PPGcited by 1
  24. 2022Localizing Semantic Patches for Accelerating Image Classificationcited by 0
  25. 2023MetaMixer: A Regularization Strategy for Online Knowledge Distillationcited by 0
  26. 2023Team AcieLee: Technical Report for EPIC-SOUNDS Audio-Based Interaction Recognition Challenge 2023cited by 0
  27. 2023SemST: Semantically Consistent Multi-Scale Image Translation via Structure-Texture Alignmentcited by 0
  28. 2024FedMID: A Data-Free Method for Using Intermediate Outputs as a Defense Mechanism Against Poisoning Attacks in Federated Learningcited by 0
  29. 2024Keypoint Aware Masked Image Modellingcited by 0
  30. 2025When Better Teachers Don't Make Better Students: Revisiting Knowledge Distillation for CLIP Models in VQAcited by 0