mlsanityData to 7 Oct 2026

Improving Weak-to-Strong Generalization with Scalable Oversight and Ensemble Learning

arXiv 2402.00667 ↗First posted 1 Feb 2024cs.CLcited by 17

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

This paper presents a follow-up study to OpenAI's recent superalignment work on Weak-to-Strong Generalization (W2SG). Superalignment focuses on ensuring that high-level AI systems remain consistent with human values and intentions when dealing with complex, high-risk tasks. The W2SG framework has opened new possibilities for empirical research in this evolving field. Our study simulates two phases of superalignment under the W2SG framework: the development of general superhuman models and the progression towards superintelligence. In the first phase, based on human supervision, the quality of weak supervision is enhanced through a combination of scalable oversight and ensemble learning, reducing the capability gap between weak teachers and strong students. In the second phase, an automatic alignment evaluator is employed as the weak supervisor. By recursively updating this auto aligner, the capabilities of the weak teacher models are synchronously enhanced, achieving weak-to-strong supervision over stronger student models.We also provide an initial validation of the proposed approach for the first phase. Using the SciQ task as example, we explore ensemble learning for weak teacher models through bagging and boosting. Scalable oversight is explored through two auxiliary settings: human-AI interaction and AI-AI debate. Additionally, the paper discusses the impact of improved weak supervision on enhancing weak-to-strong generalization based on in-context learning. Experiment code and dataset will be released at https://github.com/ADaM-BJTU/W2SG.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 9 of its 13 references that are in the map. First posted on arXiv 1 Feb 2024.

Builds on

9 of 13 references.

References · 9

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  4. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  5. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  6. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  7. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  8. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242
  9. 2023OpenICL: An Open-Source Framework for In-context Learningcited by 51

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 2.

17 papers.

Sort:

By area:

  1. 2024Weak-to-Strong Reasoningcited by 32
  2. 2024The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignmentcited by 11
  3. 2025Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safetycited by 7
  4. 2024Bayesian WeakS-to-Strong from Text Classification to Generationcited by 6
  5. 2025Generalizable Video Quality Assessment via Weak-to-Strong Learningcited by 5
  6. 2024Weak-to-Strong Generalization beyond Accuracy: a Pilot Study in Safety, Toxicity, and Legal Reasoningcited by 4
  7. 2025EnsemW2S: Enhancing Weak-to-Strong Generalization with Large Language Model Ensemblescited by 4
  8. 2025Weak-to-Strong Generalization under Distribution Shiftscited by 4
  9. 2025Weak-to-Strong Generalization via Bregman Bias-Variance Decompositioncited by 3
  10. 2025Weak-to-Strong Generalization with Failure Trajectories: A Tree-based Approach to Elicit Optimal Policy in Strong Modelscited by 3
  11. 2025The Capabilities and Limitations of Weak-to-Strong Generalization: Generalization and Calibrationcited by 2
  12. 2025A Benchmark for Scalable Oversight Protocolscited by 2
5 more
  1. 2024EnsemW2S: Enhancing Weak-to-Strong Generalization with Large Language Model Ensemblescited by 1
  2. 2025Research Superalignment Should Advance Now with Alternating Competence and Conformity Optimizationcited by 1
  3. 2025Alice: Proactive Learning with Teacher's Demonstrations for Weak-to-Strong Generalizationcited by 1
  4. 2026On the Blessing of Pre-training in Weak-to-Strong Generalizationcited by 0
  5. 2026AdaStop: Cost-Aware Early Stopping for DNN Test Selectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Selective Weak-to-Strong Generalization
  2. 2024Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalization
  3. 2025How to Mitigate Overfitting in Weak-to-strong Generalization?
  4. 2024The Superalignment of Superhuman Intelligence with Large Language Models
  5. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
  6. 2024Vision Superalignment: Weak-to-Strong Generalization for Vision Foundation Models
  7. 2024Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Experts
  8. 2024Improving Weak-to-Strong Generalization with Reliability-Aware Alignment