mlsanityData to 14 Oct 2026

Collaborative Learning for Deep Neural Networks

arXiv 1805.11761 ↗First posted 30 May 2018stat.ML cs.CV cs.LGcited by 69

Not on the map. Why?

Abstract (arXiv)

We introduce collaborative learning in which multiple classifier heads of the same network are simultaneously trained on the same training data to improve generalization and robustness to label noise with no extra inference cost. It acquires the strengths from auxiliary training, multi-task learning and knowledge distillation. There are two important mechanisms involved in collaborative learning. First, the consensus of multiple views from different classifier heads on the same example provides supplementary information as well as regularization to each classifier, thereby improving generalization. Second, intermediate-level representation (ILR) sharing with backpropagation rescaling aggregates the gradient flows from all heads, which not only reduces training computational complexity, but also facilitates supervision to the shared layers. The empirical results on CIFAR and ImageNet datasets demonstrate that deep neural networks learned as a group in a collaborative way significantly reduce the generalization error and increase the robustness to label noise.

Key facts. Cited by 69 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 24; 2022: 15; 2023: 12; 2024: 12; 2025: 4; 2026: 2.

69 papers.

Sort:

By area:

  1. 2022Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolutioncited by 54
  2. 2021DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimationcited by 48
  3. 2022Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillationcited by 45
  4. 2021Mutual Contrastive Learning for Visual Representation Learningcited by 42
  5. 2022Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognitioncited by 36
  6. 2023A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentationcited by 33
  7. 2021MixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworkscited by 32
  8. 2023Quality-Agnostic Deepfake Detection with Intra-model Collaborative Learningcited by 32
  9. 2021Student Network Learning via Evolutionary Knowledge Distillationcited by 30
  10. 2021On the Reproducibility of Neural Network Predictionscited by 29
  11. 2022Switchable Online Knowledge Distillationcited by 29
  12. 2021Distilling a Powerful Student Model via Online Knowledge Distillationcited by 22
38 more
  1. 2021Handwritten Mathematical Expression Recognition via Attention Aggregation based Bi-directional Mutual Learningcited by 20
  2. 2023Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillationcited by 20
  3. 2023Improving Plasticity in Online Continual Learning via Collaborative Learningcited by 20
  4. 2022Overcoming Catastrophic Forgetting beyond Continual Learning: Balanced Training for Neural Machine Translationcited by 18
  5. 2021Mutual Adversarial Training: Learning together is better than going alonecited by 17
  6. 2022On the Effects of Different Types of Label Noise in Multi-Label Remote Sensing Image Classificationcited by 17
  7. 2021Revisiting Knowledge Distillation: An Inheritance and Exploration Frameworkcited by 15
  8. 2024Predictive Churn with the Set of Good Modelscited by 15
  9. 2021I3CL:Intra- and Inter-Instance Collaborative Learning for Arbitrary-shaped Scene Text Detectioncited by 14
  10. 2021Knowledge Distillation Using Hierarchical Self-Supervision Augmented Distributioncited by 13
  11. 2024Recurrent Early Exits for Federated Learning with Heterogeneous Clientscited by 12
  12. 2023Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generationcited by 10
  13. 2021Dynamic Encoder Transducer: A Flexible Solution For Trading Off Accuracy For Latencycited by 9
  14. 2023Latent Class-Conditional Noise Modelcited by 9
  15. 2025Vulnerability-Aware Spatio-Temporal Learning for Generalizable Deepfake Video Detectioncited by 9
  16. 2022Co-training 2L Submodels for Visual Recognitioncited by 8
  17. 2024Function Space Diversity for Uncertainty Prediction via Repulsive Last-Layer Ensemblescited by 8
  18. 2021Locally Adaptive Label Smoothing for Predictive Churncited by 7
  19. 2021Collaborative Training of Acoustic Encoders for Speech Recognitioncited by 7
  20. 2022Federated Learning with Privacy-Preserving Ensemble Attention Distillationcited by 6
  21. 2021Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widthscited by 5
  22. 2022Consensus Learning from Heterogeneous Objectives for One-Class Collaborative Filteringcited by 5
  23. 2022Understanding the Tricks of Deep Learning in Medical Image Segmentation: Challenges and Future Directionscited by 5
  24. 2022Collaborative Training of Heterogeneous Reinforcement Learning Agents in Environments with Sparse Rewards: What and When to Share?cited by 4
  25. 2023Test-Time Adaptation for Nighttime Color-Thermal Semantic Segmentationcited by 4
  26. 2021Improved Knowledge Distillation via Adversarial Collaborationcited by 3
  27. 2021Encoding Hierarchical Information in Neural Networks helps in Subpopulation Shiftcited by 3
  28. 2022Deep Combinatorial Aggregationcited by 3
  29. 2024Are EEG Sequences Time Series? EEG Classification with Time Series Models and Joint Subject Trainingcited by 3
  30. 2021Collaborative Label Correction via Entropy Thresholdingcited by 2
  31. 2021Unpaired cross-modality educed distillation (CMEDL) for medical image segmentationcited by 2
  32. 2022Mutual Contrastive Low-rank Learning to Disentangle Whole Slide Image Representations for Glioma Gradingcited by 2
  33. 2023Semi-Supervised Semantic Segmentation of Cell Nuclei via Diffusion-based Large-Scale Pre-Training and Collaborative Learningcited by 2
  34. 2024Knowledge Distillation Approaches for Accurate and Efficient Recommender Systemcited by 2
  35. 2021Intra-Model Collaborative Learning of Neural Networkscited by 1
  36. 2021Cooperative Learning for Noisy Supervisioncited by 1
  37. 2022An Investigation of Smart Contract for Collaborative Machine Learning Model Trainingcited by 1
  38. 2023Heterogeneous-Branch Collaborative Learning for Dialogue Generationcited by 1

50 most cited of 69 All 69 (JSON)