# Selective Weak-to-Strong Generalization

arXiv: https://arxiv.org/abs/2511.14166 · HTML: https://mlsanity.com/paper/2511.14166/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2025.

## Built on by (most cited first, 3 of 3)

- [DenoiseRL: Bootstrapping Reasoning Models to Recover from Noisy Prefixes](https://mlsanity.com/paper/2605.28421/) (2026)
- [Emergent Alignment](https://mlsanity.com/paper/2606.19527/) (2026)
- [Sharding Prevents LLM Oversight Failures and Adversarial Exploitation](https://mlsanity.com/paper/2608.06422/) (2026)

## Abstract (arXiv)

Future superhuman models will surpass the ability of humans and humans will only be able to textitweakly supervise superhuman models. To alleviate the issue of lacking high-quality data for model alignment, some works on weak-to-strong generalization (W2SG) finetune a strong pretrained model with a weak supervisor so that it can generalize beyond weak supervision. However, the invariable use of weak supervision in existing methods exposes issues in robustness, with a proportion of weak labels proving harmful to models. In this paper, we propose a selective W2SG framework to avoid using weak supervision when unnecessary. We train a binary classifier P(IK) to identify questions that a strong model can answer and use its self-generated labels for alignment. We further refine weak labels with a graph smoothing method. Extensive experiments on three benchmarks show that our method consistently outperforms competitive baselines. Further analyses show that P(IK) can generalize across tasks and difficulties, which indicates selective W2SG can help superalignment.