mlsanityData to 7 Oct 2026

A General Framework for Learning from Weak Supervision

arXiv 2402.01922 ↗First posted 2 Feb 2024cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Weakly supervised learning generally faces challenges in applicability to various scenarios with diverse weak supervision and in scalability due to the complexity of existing algorithms, thereby hindering the practical deployment. This paper introduces a general framework for learning from weak supervision (GLWS) with a novel algorithm. Central to GLWS is an Expectation-Maximization (EM) formulation, adeptly accommodating various weak supervision sources, including instance partial labels, aggregate statistics, pairwise observations, and unlabeled data. We further present an advanced algorithm that significantly simplifies the EM computational demands using a Non-deterministic Finite Automaton (NFA) along with a forward-backward algorithm, which effectively reduces time complexity from quadratic or factorial often required in existing solutions to linear scale. The problem of learning from arbitrary weak supervision is therefore converted to the NFA modeling of them. GLWS not only enhances the scalability of machine learning models but also demonstrates superior performance and versatility across 11 weak supervision scenarios. We hope our work paves the way for further advancements and practical deployment in this field.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 52 of its 87 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

52 of 87 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  7. 2016Wide Residual Networkscited by 3,057
  8. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  9. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  10. 2018Attention-based Deep Multiple Instance Learningcited by 1,077
  11. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  12. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
38 more
  1. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  2. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
  3. 2022DeiT III: Revenge of the ViTcited by 417
  4. 2021ResNet strikes back: An improved training procedure in timmcited by 365
  5. 2022FreeMatch: Self-adaptive Thresholding for Semi-supervised Learningcited by 232
  6. 2017Positive-Unlabeled Learning with Non-Negative Risk Estimatorcited by 217
  7. 2023SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learningcited by 133
  8. 2020Progressive Identification of True Labels for Partial-Label Learningcited by 102
  9. 2019Multi-class Classification without Multi-class Labelscited by 100
  10. 2022USB: A Unified Semi-supervised Learning Benchmark for Classificationcited by 95
  11. 2020Provably Consistent Partial-Label Learningcited by 92
  12. 2015A Deeper Look at Dataset Biascited by 86
  13. 2020Privacy in Deep Learning: A Surveycited by 70
  14. 2021Leveraged Weighted Loss for Partial Label Learningcited by 66
  15. 2018Complementary-Label Learning for Arbitrary Losses and Modelscited by 57
  16. 2019Learning with Multiple Complementary Labelscited by 53
  17. 2017Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labelscited by 52
  18. 2021Mixture Proportion Estimation and PU Learning: A Modern Approachcited by 52
  19. 2018On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Datacited by 43
  20. 2014On Learning from Label Proportionscited by 38
  21. 2022Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspectivecited by 37
  22. 2018Classification from Pairwise Similarity and Unlabeled Datacited by 36
  23. 2019Learning from Label Proportions with Consistency Regularizationcited by 34
  24. 2022Learning from Label Proportions by Learning with Label Noisecited by 30
  25. 2022Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classificationcited by 24
  26. 2017Binary Classification from Positive-Confidence Datacited by 23
  27. 2020Learning from Positive and Unlabeled Data with Arbitrary Positive Shiftcited by 23
  28. 2019Classification from Pairwise Similarities/Dissimilarities and Unlabeled Data via Empirical Risk Minimizationcited by 21
  29. 2020Pointwise Binary Classification with Pairwise Confidence Comparisonscited by 20
  30. 2021Learning from Similarity-Confidence Datacited by 19
  31. 2023Binary Classification with Confidence Differencecited by 13
  32. 2020Pairwise Supervision Can Provably Elicit a Decision Boundarycited by 9
  33. 2023Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurationscited by 9
  34. 2023A Unified Approach to Count-Based Weakly-Supervised Learningcited by 9
  35. 2019Classification from Triplet Comparison Datacited by 6
  36. 2022Multi-class Classification from Multiple Unlabeled Datasets with Partial Risk Regularizationcited by 6
  37. 2022Data Privacy and Trustworthy Machine Learningcited by 6
  38. 2023A Universal Unbiased Method for Classification from Aggregate Observationscited by 5

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2025Weak-to-Strong Generalization with Failure Trajectories: A Tree-based Approach to Elicit Optimal Policy in Strong Modelscited by 3
  2. 2025Accessible, Realistic, and Fair Evaluation of Positive-Unlabeled Learning Algorithmscited by 1
  3. 2026Learning Stable Predictors from Weak Supervision under Distribution Shiftcited by 1
  4. 2026FastBUS: A Fast Bayesian Framework for Unified Weakly-Supervised Learningcited by 0
  5. 2026Learning Demographic-Conditioned Mobility Trajectories with Aggregate Supervisioncited by 0
  6. 2026BiTro: Bidirectional Transfer Learning Enhances Bulk and Spatial Transcriptomics Prediction in Cancer Pathological Imagescited by 0
  7. 2026Risk-Consistent Multiclass Learning from Random Label-Subset Membership Queriescited by 0