mlsanityData to 14 Oct 2026

Weak-to-Strong Generalization under Distribution Shifts

arXiv 2510.21332 ↗First posted 24 Oct 2025cs.LG cs.AI stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

As future superhuman models become increasingly complex, accurately supervising their behavior may exceed human capabilities. Recent works have demonstrated that in such scenarios, weak models can effectively supervise strong models, a phenomenon known as weak-to-strong generalization. However, we find that naive weak-to-strong generalization fails under distribution shifts, often leading to worse performance of the strong model than its weak supervisors. To address this, we propose RAVEN, a robust weak-to-strong generalization framework that dynamically learns the optimal combinations of weak models in addition to parameters of the strong model. We demonstrate the effectiveness of RAVEN on image classification, text classification, and preference alignment tasks. RAVEN outperforms alternative baselines by over 30% on out-of-distribution tasks while matching or surpassing existing methods on in-distribution tasks. Moreover, our results show that RAVEN assigns higher weights to more accurate weak models, demonstrating its ability to automatically identify trustworthy supervision.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 60 of its 70 references that are in the map. First posted on arXiv 24 Oct 2025.

Builds on

60 of 70 references.

References · 60

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  9. 2024Qwen2.5 Technical Reportcited by 4,942
  10. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  11. 2017Deep reinforcement learning from human preferencescited by 3,912
  12. 2023Let's Verify Step by Stepcited by 3,789
38 more
  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  2. 2020Learning to summarize from human feedbackcited by 2,588
  3. 2016Concrete Problems in AI Safetycited by 1,642
  4. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  5. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  6. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  7. 2020In Search of Lost Domain Generalizationcited by 1,054
  8. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 732
  9. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543
  10. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 467
  11. 2018Scalable agent alignment via reward modeling: a research directioncited by 447
  12. 2018AI safety via debatecited by 400
  13. 2022Self-critiquing models for assisting human evaluatorscited by 380
  14. 2020What is being transferred in transfer learning?cited by 368
  15. 2017Functional Map of the Worldcited by 338
  16. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 274
  17. 2024RLHF Workflow: From Reward Modeling to Online RLHFcited by 267
  18. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 266
  19. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  20. 2018Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shiftcited by 243
  21. 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 225
  22. 2021Trustworthy AI: From Principles to Practicescited by 207
  23. 2021A Fine-Grained Analysis on Distribution Shiftcited by 171
  24. 2018Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objectscited by 143
  25. 2015Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networkscited by 139
  26. 2021Goal Misgeneralization in Deep Reinforcement Learningcited by 123
  27. 2018Dark Model Adaptation: Semantic Image Segmentation from Daytime to Nighttimecited by 122
  28. 2024On scalable oversight with weak LLMs judging strong LLMscited by 99
  29. 2015Webly Supervised Learning of Convolutional Networkscited by 95
  30. 2018Unsupervised domain adaptation for medical imaging segmentation with self-ensemblingcited by 94
  31. 2021The iWildCam 2021 Competition Datasetcited by 77
  32. 2024VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignmentcited by 58
  33. 2024Theoretical Analysis of Weak-to-Strong Generalizationcited by 48
  34. 2020Hidden Incentives for Auto-Induced Distributional Shiftcited by 43
  35. 2024Quantifying the Gain in Weak-to-Strong Generalizationcited by 41
  36. 2024Vision Superalignment: Weak-to-Strong Generalization for Vision Foundation Modelscited by 28
  37. 2024Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Expertscited by 28
  38. 2021An Effective Baseline for Robustness to Distributional Shiftcited by 25

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Learning Stable Predictors from Weak Supervision under Distribution Shiftcited by 1
  2. 2026On the Blessing of Pre-training in Weak-to-Strong Generalizationcited by 1
  3. 2026J-Zero: Unified Challenger--Solver--Judge Self-Evolution from Zero Datacited by 1
  4. 2026When Weak LLMs Speak with Confidence, Preference Alignment Gets Strongercited by 0