mlsanityData to 14 Oct 2026

Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Experts

arXiv 2402.15505 ↗First posted 23 Feb 2024cs.LG cs.AI cs.CVcited by 28

On the map: Mixture-of-experts → Vision mixture-of-experts

Abstract (arXiv)

Steering the behavior of a strong model pre-trained on internet-scale data can be difficult due to the scarcity of competent supervisors. Recent studies reveal that, despite supervisory noises, a strong student model may surpass its weak teacher when fine-tuned on specific objectives. Yet, the effectiveness of such weak-to-strong generalization remains limited, especially in the presence of large capability gaps. In this paper, we propose to address this challenge by harnessing a diverse set of specialized teachers, instead of a single generalist one, that collectively supervises the strong student. Our approach resembles the classical hierarchical mixture of experts, with two components tailored for co-supervision: (i) we progressively alternate student training and teacher assignment, leveraging the growth of the strong student to identify plausible supervisions; (ii) we conservatively enforce teacher-student and local-global consistency, leveraging their dependencies to reject potential annotation noises. We validate the proposed method through visual recognition tasks on the OpenAI weak-to-strong benchmark and additional multi-domain datasets. Our code is available at urlhttps://github.com/yuejiangliu/csl.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 31 of its 46 references that are in the map. First posted on arXiv 23 Feb 2024.

Builds on

31 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022A Review of Sparse Expert Models in Deep Learning4 Sep
  2. 2021Scaling Vision with Sparse Mixture of Experts10 Jun

Other references · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2017Deep reinforcement learning from human preferencescited by 3,912
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  12. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
17 more
  1. 2020Learning to summarize from human feedbackcited by 2,588
  2. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  3. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  4. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
  5. 2022Mixture-of-Experts with Expert Choice Routingcited by 623
  6. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 467
  7. 2018Scalable agent alignment via reward modeling: a research directioncited by 447
  8. 2018AI safety via debatecited by 400
  9. 2021Ensemble deep learning: A reviewcited by 284
  10. 2023AI Alignment: A Comprehensive Surveycited by 263
  11. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 258
  12. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  13. 2019Understanding and Utilizing Deep Neural Networks Trained with Noisy Labelscited by 209
  14. 2018Supervising strong learners by amplifying weak expertscited by 145
  15. 2023Evaluating Superhuman Models with Consistency Checkscited by 47
  16. 2019Embedded Agencycited by 19
  17. 2023Instance-dependent Noisy-label Learning with Graphical Model Based Noise-rate Estimationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 12; 2025: 14; 2026: 2.

28 papers.

Sort:

By area:

  1. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  2. 2024Weak-to-Strong Reasoningcited by 33
  3. 2024MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimizationcited by 22
  4. 2025Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executorscited by 22
  5. 2024Your Weak LLM is Secretly a Strong Teacher for Alignmentcited by 20
  6. 2024Provable Weak-to-Strong Generalization via Benign Overfittingcited by 16
  7. 2024Improving Weak-to-Strong Generalization with Reliability-Aware Alignmentcited by 14
  8. 2024The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignmentcited by 13
  9. 2025Discrepancies are Virtue: Weak-to-Strong Generalization through Lens of Intrinsic Dimensioncited by 12
  10. 2024ALaRM: Align Language Models via Hierarchical Rewards Modelingcited by 11
  11. 2024Hummer: Towards Limited Competitive Preference Datasetcited by 10
  12. 2024Efficient Model-agnostic Alignment via Bayesian Persuasioncited by 8
16 more
  1. 2025Debate Helps Weak-to-Strong Generalizationcited by 8
  2. 2025On the Mechanisms of Weak-to-Strong Generalization: A Theoretical Perspectivecited by 8
  3. 2025Incentivizing Strong Reasoning from Weak Supervisioncited by 8
  4. 2024Bayesian WeakS-to-Strong from Text Classification to Generationcited by 6
  5. 2025Generalizable Video Quality Assessment via Weak-to-Strong Learningcited by 5
  6. 2025EnsemW2S: Enhancing Weak-to-Strong Generalization with Large Language Model Ensemblescited by 4
  7. 2025Weak-to-Strong Generalization via Bregman Bias-Variance Decompositioncited by 4
  8. 2025Weak-to-Strong Generalization under Distribution Shiftscited by 4
  9. 2025How to Mitigate Overfitting in Weak-to-strong Generalization?cited by 3
  10. 2025The Capabilities and Limitations of Weak-to-Strong Generalization: Generalization and Calibrationcited by 2
  11. 2025Collaborative Learning of On-Device Small Model and Cloud-Based Large Model: Advances and Future Directionscited by 2
  12. 2025Does Weak-to-strong Generalization Happen under Spurious Correlations?cited by 2
  13. 2024EnsemW2S: Enhancing Weak-to-Strong Generalization with Large Language Model Ensemblescited by 1
  14. 2025Research Superalignment Should Advance Now with Alternating Competence and Conformity Optimizationcited by 1
  15. 2026On the Blessing of Pre-training in Weak-to-Strong Generalizationcited by 1
  16. 2026Trust Functions: Near-Lossless Weak-to-Strong Generalization by Learning When to Trust the Weak Teachercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Vision Superalignment: Weak-to-Strong Generalization for Vision Foundation Models
  2. 2017Hard Mixtures of Experts for Large Scale Weakly Supervised Vision
  3. 2025Exploring Weak-to-Strong Generalization for CLIP-based Classification
  4. 2024Improving Weak-to-Strong Generalization with Scalable Oversight and Ensemble Learning
  5. 2025Alice: Proactive Learning with Teacher's Demonstrations for Weak-to-Strong Generalization
  6. 2024UNIC: Universal Classification Models via Multi-teacher Distillation
  7. 2023Active Teacher for Semi-Supervised Object Detection
  8. 2021Humble Teachers Teach Better Students for Semi-Supervised Object Detection