# EnsemW2S: Enhancing Weak-to-Strong Generalization with Large Language Model Ensembles

arXiv: https://arxiv.org/abs/2410.04571 · HTML: https://mlsanity.com/paper/2410.04571/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 6 Oct 2024.

## Builds on (most cited first, 22 of 22)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models](https://mlsanity.com/paper/2404.18796/) (2024)
- [Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences](https://mlsanity.com/paper/2404.03715/) (2024)
- [Controlled Decoding from Language Models](https://mlsanity.com/paper/2310.17022/) (2023)
- [Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision](https://mlsanity.com/paper/2403.09472/) (2024)
- [QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questions](https://mlsanity.com/paper/1909.03553/) (2019)
- [Learning to Decode Collaboratively with Multiple Language Models](https://mlsanity.com/paper/2403.03870/) (2024)
- [Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling](https://mlsanity.com/paper/2408.16737/) (2024)
- [Learning to Generate Better Than Your LLM](https://mlsanity.com/paper/2306.11816/) (2023)
- [Theoretical Analysis of Weak-to-Strong Generalization](https://mlsanity.com/paper/2405.16043/) (2024)
- [Quantifying the Gain in Weak-to-Strong Generalization](https://mlsanity.com/paper/2405.15116/) (2024)
- [The Unreasonable Effectiveness of Easy Training Data for Hard Tasks](https://mlsanity.com/paper/2401.06751/) (2024)
- [Vision Superalignment: Weak-to-Strong Generalization for Vision Foundation Models](https://mlsanity.com/paper/2402.03749/) (2024)
- [Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Experts](https://mlsanity.com/paper/2402.15505/) (2024)
- [Easy2Hard-Bench: Standardized Difficulty Labels for Profiling LLM Performance and Generalization](https://mlsanity.com/paper/2409.18433/) (2024)
- [Transcendence: Generative Models Can Outperform The Experts That Train Them](https://mlsanity.com/paper/2406.11741/) (2024)
- [Improving Weak-to-Strong Generalization with Scalable Oversight and Ensemble Learning](https://mlsanity.com/paper/2402.00667/) (2024)
- [Collaborative decoding of critical tokens for boosting factuality of large language models](https://mlsanity.com/paper/2402.17982/) (2024)

## Built on by (most cited first, 1 of 1)

- [Uncertainty-Aware Answer Selection for Improved Reasoning in Multi-LLM Systems](https://mlsanity.com/paper/2510.02377/) (2025)

## Abstract (arXiv)

With Large Language Models (LLMs) rapidly approaching and potentially surpassing human-level performance, it has become imperative to develop approaches capable of effectively supervising and enhancing these powerful models using smaller, human-level models exposed to only human-level data. We address this critical weak-to-strong (W2S) generalization challenge by proposing a novel method aimed at improving weak experts, by training on the same limited human-level data, enabling them to generalize to complex, super-human-level tasks. Our approach, called **EnsemW2S**, employs a token-level ensemble strategy that iteratively combines multiple weak experts, systematically addressing the shortcomings identified in preceding iterations. By continuously refining these weak models, we significantly enhance their collective ability to supervise stronger student models. We extensively evaluate the generalization performance of both the ensemble of weak experts and the subsequent strong student model across in-distribution (ID) and out-of-distribution (OOD) datasets. For OOD, we specifically introduce question difficulty as an additional dimension for defining distributional shifts. Our empirical results demonstrate notable improvements, achieving 4%, and 3.2% improvements on ID datasets and, upto 6% and 2.28% on OOD datasets for experts and student models respectively, underscoring the effectiveness of our proposed method in advancing W2S generalization.