mlsanityData to 14 Oct 2026

On the Minimal Supervision for Training Any Binary Classifier from Only Unlabeled Data

arXiv 1808.10585 ↗First posted 31 Aug 2018stat.ML cs.LGcited by 43

Not on the map. Why?

Abstract (arXiv)

Empirical risk minimization (ERM), with proper loss function and regularization, is the common practice of supervised classification. In this paper, we study training arbitrary (from linear to deep) binary classifier from only unlabeled (U) data by ERM. We prove that it is impossible to estimate the risk of an arbitrary binary classifier in an unbiased manner given a single set of U data, but it becomes possible given two sets of U data with different class priors. These two facts answer a fundamental question---what the minimal supervision is for training any binary classifier from only U data. Following these findings, we propose an ERM-based learning method from two sets of U data, and then prove it is consistent. Experiments demonstrate the proposed method could train deep models and outperform state-of-the-art methods for learning from two sets of U data.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 12; 2022: 6; 2023: 6; 2024: 3; 2025: 12; 2026: 3.

43 papers.

Sort:

By area:

  1. 2021Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularizationcited by 62
  2. 2021Instance-Dependent Partial Label Learningcited by 49
  3. 2020Robust Imitation Learning from Noisy Demonstrationscited by 26
  4. 2022Is the Performance of My Deep Network Too Good to Be True? A Direct Approach to Estimating the Bayes Error in Binary Classificationcited by 26
  5. 2021On the Robustness of Average Losses for Partial-Label Learningcited by 20
  6. 2023Easy Learning from Label Proportionscited by 20
  7. 2021Two-stage Training for Learning from Label Proportionscited by 19
  8. 2021Can Less be More? When Increasing-to-Balancing Label Noise Rates Considered Beneficialcited by 13
  9. 2023Binary Classification with Confidence Differencecited by 13
  10. 2021Binary Classification from Multiple Unlabeled Datasets via Surrogate Set Classificationcited by 12
  11. 2023Weakly Supervised AUC Optimization: A Unified Partial AUC Approachcited by 11
  12. 2021CCMN: A General Framework for Learning with Class-Conditional Multi-Label Noisecited by 9
31 more
  1. 2023Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurationscited by 9
  2. 2021Learning with Proper Partial Labelscited by 8
  3. 2021Positive-Unlabeled Classification under Class-Prior Shift: A Prior-invariant Approach Based on Density Ratio Estimationcited by 7
  4. 2022Reduction from Complementary-Label Learning to Probability Estimatescited by 7
  5. 2024A General Framework for Learning from Weak Supervisioncited by 7
  6. 2022Multi-class Classification from Multiple Unlabeled Datasets with Partial Risk Regularizationcited by 6
  7. 2023Unified Risk Analysis for Weakly Supervised Learningcited by 5
  8. 2023Making Binary Classification from Multiple Unlabeled Datasets Almost Free of Supervisioncited by 4
  9. 2025Nearly Optimal Sample Complexity for Learning with Label Proportionscited by 4
  10. 2025On Symmetric Losses for Robust Policy Optimization with Noisy Preferencescited by 4
  11. 2021Multi-Class Classification from Single-Class Data with Confidencescited by 3
  12. 2021Labels, Information, and Computation: Efficient Learning Using Sufficient Labelscited by 2
  13. 2022Learning from Label Proportions with Instance-wise Consistencycited by 2
  14. 2024Learning from Aggregate responses: Instance Level versus Bag Level Loss Functionscited by 2
  15. 2025Self Iterative Label Refinement via Robust Unlabeled Learningcited by 2
  16. 2022Learning from Positive and Unlabeled Data with Augmented Classescited by 1
  17. 2025Optimal Learning from Label Proportions with General Loss Functionscited by 1
  18. 2021A Symmetric Loss Perspective of Reliable Machine Learningcited by 0
  19. 2022Decompositional Generation Process for Instance-Dependent Partial Label Learningcited by 0
  20. 2024Learning from Concealed Labelscited by 0
  21. 2025Reduction of Supervision for Biomedical Knowledge Discoverycited by 0
  22. 2025Learning from M-Tuple Dominant Positive and Unlabeled Datacited by 0
  23. 2025A Unified Empirical Risk Minimization Framework for Flexible N-Tuples Weak Supervisioncited by 0
  24. 2025Learning from Similarity-Confidence and Confidence-Differencecited by 0
  25. 2025Learning from Uncertain Similarity and Unlabeled Datacited by 0
  26. 2025Learning from N-Tuple Data with M Positive Instances: Unbiased Risk Estimation and Theoretical Guaranteescited by 0
  27. 2025Cost-Sensitive Unbiased Risk Estimation for Multi-Class Positive-Unlabeled Learningcited by 0
  28. 2025A Unified and Stable Risk Minimization Framework for Weakly Supervised Learning with Theoretical Guaranteescited by 0
  29. 2026Binary Classification from Coupled Pairwise Labelscited by 0
  30. 2026Mixture Proportion Estimation and Weakly-supervised Kernel Test for Conditional Independencecited by 0
  31. 2026Importance Weighting for Unlabeled-unlabeled Learning under Distribution Shiftcited by 0