mlsanityData to 14 Oct 2026

Quantifying the Gain in Weak-to-Strong Generalization

arXiv 2405.15116 ↗First posted 24 May 2024cs.LG cs.AIcited by 41

Not on the map. Why?

Abstract (arXiv)

Recent advances in large language models have shown capabilities that are extraordinary and near-superhuman. These models operate with such complexity that reliably evaluating and aligning them proves challenging for humans. This leads to the natural question: can guidance from weak models (like humans) adequately direct the capabilities of strong models? In a recent and somewhat surprising work, Burns et al. (2023) empirically demonstrated that when strong models (like GPT-4) are finetuned using labels generated by weak supervisors (like GPT-2), the strong models outperform their weaker counterparts -- a phenomenon they term weak-to-strong generalization. In this work, we present a theoretical framework for understanding weak-to-strong generalization. Specifically, we show that the improvement in performance achieved by strong models over their weaker counterparts is quantified by the misfit error incurred by the strong model on labels generated by the weaker model. Our theory reveals several curious algorithmic insights. For instance, we can predict the amount by which the strong model will improve over the weak model, and also choose among different weak models to train the strong model, based on its misfit error. We validate our theoretical findings through various empirical assessments.

Key facts. Cited by 41 papers in the map (data through 14 Oct 2026). Builds on 25 of its 40 references that are in the map. First posted on arXiv 24 May 2024.

Builds on

25 of 40 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2017Deep reinforcement learning from human preferencescited by 3,912
  4. 2022Emergent Abilities of Large Language Modelscited by 2,512
  5. 2020Knowledge Distillation: A Surveycited by 1,603
  6. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  7. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  8. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,051
  9. 2019Introduction to Online Convex Optimizationcited by 803
  10. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 739
  11. 2018Born Again Neural Networkscited by 546
  12. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 467
13 more
  1. 2019How does Disagreement Help Generalization against Label Corruption?cited by 390
  2. 2019CamemBERT: a Tasty French Language Modelcited by 368
  3. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 297
  4. 2023AI Alignment: A Comprehensive Surveycited by 263
  5. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  6. 2020Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Datacited by 180
  7. 2020Self-Distillation Amplifies Regularization in Hilbert Spacecited by 173
  8. 2019FlauBERT: Unsupervised Language Model Pre-training for Frenchcited by 153
  9. 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 138
  10. 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 96
  11. 2020Molecular representation learning with language models and domain-relevant auxiliary taskscited by 94
  12. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 79
  13. 2017Theoretical Foundation of Co-Training and Disagreement-Based Algorithmscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 18; 2026: 13.

41 papers.

Sort:

By area:

  1. 2024Weak-to-Strong Reasoningcited by 33
  2. 2024Selective Preference Optimization via Token-Level Reward Function Estimationcited by 25
  3. 2024Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalizationcited by 23
  4. 2024High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Lawscited by 22
  5. 2025Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executorscited by 22
  6. 2024Weak-to-Strong Generalization Through the Data-Centric Lenscited by 21
  7. 2024Provable Weak-to-Strong Generalization via Benign Overfittingcited by 16
  8. 2025Weak-to-Strong Generalization Even in Random Feature Networks, Provablycited by 15
  9. 2024A transfer learning framework for weak-to-strong generalizationcited by 14
  10. 2024Improving Weak-to-Strong Generalization with Reliability-Aware Alignmentcited by 14
  11. 2024The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignmentcited by 13
  12. 2025Discrepancies are Virtue: Weak-to-Strong Generalization through Lens of Intrinsic Dimensioncited by 12
29 more
  1. 2025Representations Shape Weak-to-Strong Generalization: Theoretical Insights and Empirical Predictionscited by 10
  2. 2025Debate Helps Weak-to-Strong Generalizationcited by 8
  3. 2025On the Mechanisms of Weak-to-Strong Generalization: A Theoretical Perspectivecited by 8
  4. 2025CrowdSelect: Synthetic Instruction Data Selection with Multi-LLM Wisdomcited by 6
  5. 2026Weak Critics Make Strong Learners: On-Policy Critique Distillation for Scalable Oversightcited by 6
  6. 2025EnsemW2S: Enhancing Weak-to-Strong Generalization with Large Language Model Ensemblescited by 4
  7. 2025Weak-to-Strong Generalization via Bregman Bias-Variance Decompositioncited by 4
  8. 2025Weak-to-Strong Generalization under Distribution Shiftscited by 4
  9. 2025How to Mitigate Overfitting in Weak-to-strong Generalization?cited by 3
  10. 2025Weak-to-Strong Generalization with Failure Trajectories: A Tree-based Approach to Elicit Optimal Policy in Strong Modelscited by 3
  11. 2025The Capabilities and Limitations of Weak-to-Strong Generalization: Generalization and Calibrationcited by 2
  12. 2025Synergistic Weak-Strong Collaboration by Aligning Preferencescited by 2
  13. 2025Does Weak-to-strong Generalization Happen under Spurious Correlations?cited by 2
  14. 2026Weak-to-Strong Generalization is Nearly Inevitable (in Linear Models)cited by 2
  15. 2026Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillationcited by 2
  16. 2024EnsemW2S: Enhancing Weak-to-Strong Generalization with Large Language Model Ensemblescited by 1
  17. 2025Limitations of refinement methods for weak to strong generalizationcited by 1
  18. 2026Why Self-Training Helps and Hurts: Denoising vs. Signal Forgettingcited by 1
  19. 2026On the Blessing of Pre-training in Weak-to-Strong Generalizationcited by 1
  20. 2026The Mechanism of Weak-to-Strong Generalization: Feature Elicitation from Latent Knowledgecited by 1
  21. 2025Denoising Mutual Knowledge Distillation in Bi-Directional Multiple Instance Learningcited by 0
  22. 2025Fluent Alignment with Disfluent Judges: Post-training for Lower-resource Languagescited by 0
  23. 2026Evaluating Risks in Weak-to-Strong Alignment: A Bias-Variance Perspectivecited by 0
  24. 2026When In-Distribution Gains Fail: Evaluating Weak-to-Strong Reward Models under Preference Shiftcited by 0
  25. 2026Trust Functions: Near-Lossless Weak-to-Strong Generalization by Learning When to Trust the Weak Teachercited by 0
  26. 2026What Makes a Strong Model? A Unified Spectral Analysis of Knowledge Transfer over High-dimensional Linear Regressioncited by 0
  27. 2026CritICL: Inference-Time Weak-to-Strong Generalization from Small Language Model Failure Modescited by 0
  28. 2026ABC-Align: Prediction-Powered Alignment with Adaptive Bias Controlcited by 0
  29. 2026Gradient Routing Localizes and Removes Unintended Behaviors in RLcited by 0