mlsanityData to 7 Oct 2026

Representations Shape Weak-to-Strong Generalization: Theoretical Insights and Empirical Predictions

arXiv 2502.00620 ↗First posted 2 Feb 2025cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Weak-to-Strong Generalization (W2SG), where a weak model supervises a stronger one, serves as an important analogy for understanding how humans might guide superhuman intelligence in the future. Promising empirical results revealed that a strong model can surpass its weak supervisor. While recent work has offered theoretical insights into this phenomenon, a clear understanding of the interactions between weak and strong models that drive W2SG remains elusive. We investigate W2SG through a theoretical lens and show that it can be characterized using kernels derived from the principal components of weak and strong models' internal representations. These kernels can be used to define a space that, at a high level, captures what the weak model is unable to learn but is learnable by the strong model. The projection of labels onto this space quantifies how much the strong model falls short of its full potential due to weak supervision. This characterization also provides insights into how certain errors in weak supervision can be corrected by the strong model, regardless of overfitting. Our theory has significant practical implications, providing a representation-based metric that predicts W2SG performance trends without requiring labels, as shown in experiments on molecular predictions with transformers and 5 NLP tasks involving 52 LLMs.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 38 of its 55 references that are in the map. First posted on arXiv 2 Feb 2025.

Builds on

38 of 55 references.

References · 38

  1. 2023Qwen Technical Reportcited by 3,273
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  3. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  5. 2017MoleculeNet: A Benchmark for Molecular Machine Learningcited by 1,037
  6. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  7. 2020Aligning AI With Shared Human Valuescited by 706
  8. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  9. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  10. 2019Benign Overfitting in Linear Regressioncited by 544
  11. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
  12. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
26 more
  1. 2015An Introduction to Matrix Concentration Inequalitiescited by 391
  2. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  3. 2018GuacaMol: Benchmarking Models for De Novo Molecular Designcited by 294
  4. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 292
  5. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 281
  6. 2019SGD on Neural Networks Learns Functions of Increasing Complexitycited by 203
  7. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  8. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 150
  9. 2021Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learningcited by 119
  10. 2022A Kernel-Based View of Language Model Fine-Tuningcited by 118
  11. 2020Classification vs regression in overparameterized regimes: Does the loss function matter?cited by 97
  12. 2020Molecular representation learning with language models and domain-relevant auxiliary taskscited by 91
  13. 2022Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Datacited by 74
  14. 2020Implicit Regularization of Random Feature Modelscited by 54
  15. 2021The Power of Contrast for Feature Learning: A Theoretical Analysiscited by 53
  16. 2016Convolutional Neural Networks Analyzed via Convolutional Sparse Codingcited by 51
  17. 2014Sparse Modeling for Image and Vision Processingcited by 48
  18. 2021Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularizationcited by 48
  19. 2024Theoretical Analysis of Weak-to-Strong Generalizationcited by 46
  20. 2023Understanding Multimodal Contrastive Learning and Incorporating Unpaired Datacited by 43
  21. 2021Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolationcited by 41
  22. 2024Quantifying the Gain in Weak-to-Strong Generalizationcited by 40
  23. 2023Which Features are Learnt by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppressioncited by 30
  24. 2021Multi-scale Feature Learning Dynamics: Insights for Double Descentcited by 24
  25. 2024Weak-to-Strong Generalization Through the Data-Centric Lenscited by 21
  26. 2024Provable Weak-to-Strong Generalization via Benign Overfittingcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 5.

9 papers.

  1. 2025Discrepancies are Virtue: Weak-to-Strong Generalization through Lens of Intrinsic Dimensioncited by 10
  2. 2025On the Mechanisms of Weak-to-Strong Generalization: A Theoretical Perspectivecited by 7
  3. 2025Weak-to-Strong Generalization via Bregman Bias-Variance Decompositioncited by 3
  4. 2026Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillationcited by 2
  5. 2025Does Weak-to-strong Generalization Happen under Spurious Correlations?cited by 1
  6. 2026The Mechanism of Weak-to-Strong Generalization: Feature Elicitation from Latent Knowledgecited by 1
  7. 2026Why Self-Training Helps and Hurts: Denoising vs. Signal Forgettingcited by 0
  8. 2026DC-W2S: Dual-Consensus Weak-to-Strong Training for Reliable Process Reward Modeling in Biological Reasoningcited by 0
  9. 2026On the Blessing of Pre-training in Weak-to-Strong Generalizationcited by 0