# Strength from Weakness: Fast Learning Using Weak Supervision

arXiv: https://arxiv.org/abs/2002.08483 · HTML: https://mlsanity.com/paper/2002.08483/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2020.

## Built on by (most cited first, 18 of 18)

- [Can contrastive learning avoid shortcut solutions?](https://mlsanity.com/paper/2106.11230/) (2021)
- [Theoretical Analysis of Weak-to-Strong Generalization](https://mlsanity.com/paper/2405.16043/) (2024)
- [Weakly Supervised Data Augmentation Through Prompting for Dialogue Understanding](https://mlsanity.com/paper/2210.14169/) (2022)
- [An Instance-Dependent Simulation Framework for Learning with Label Noise](https://mlsanity.com/paper/2107.11413/) (2021)
- [On Label Granularity and Object Localization](https://mlsanity.com/paper/2207.10225/) (2022)
- [Leveraging Large Language Models and Weak Supervision for Social Media data annotation: an evaluation using COVID-19 self-reported vaccination tweets](https://mlsanity.com/paper/2309.06503/) (2023)
- [Ground Truth Inference for Weakly Supervised Entity Matching](https://mlsanity.com/paper/2211.06975/) (2022)
- [Few-Shot Upsampling for Protest Size Detection](https://mlsanity.com/paper/2105.11260/) (2021)
- [Blessing of Class Diversity in Pre-training](https://mlsanity.com/paper/2209.03447/) (2022)
- [Unsupervised domain adaptation by learning using privileged information](https://mlsanity.com/paper/2303.09350/) (2023)
- [Free-Grained Hierarchical Visual Recognition](https://mlsanity.com/paper/2510.14737/) (2025)
- [Fine-grained Classes and How to Find Them](https://mlsanity.com/paper/2406.11070/) (2024)
- [XPASC: Measuring Generalization in Weak Supervision by Explainability and Association](https://mlsanity.com/paper/2206.01444/) (2022)
- [Prospector Heads: Generalized Feature Attribution for Large Models & Data](https://mlsanity.com/paper/2402.11729/) (2024)
- [Predicting Perceived Gloss: Do Weak Labels Suffice?](https://mlsanity.com/paper/2403.17672/) (2024)
- [Coupled Training with Privileged Information and Unlabeled Data](https://mlsanity.com/paper/2605.23268/) (2026)
- [Informed Learning by Wide Neural Networks: Convergence, Generalization and Sampling Complexity](https://mlsanity.com/paper/2207.00751/) (2022)
- [Enhancing Instance-Level Image Classification with Set-Level Labels](https://mlsanity.com/paper/2311.05659/) (2023)

## Abstract (arXiv)

We study generalization properties of weakly supervised learning. That is, learning where only a few "strong" labels (the actual target of our prediction) are present but many more "weak" labels are available. In particular, we show that having access to weak labels can significantly accelerate the learning rate for the strong task to the fast rate of O(nicefrac1n), where n denotes the number of strongly labeled data points. This acceleration can happen even if by itself the strongly labeled data admits only the slower O(nicefrac1√n) rate. The actual acceleration depends continuously on the number of weak labels available, and on the relation between the two tasks. Our theoretical results are reflected empirically across a range of tasks and illustrate how weak labels speed up learning on the strong task.