Confident Sinkhorn Allocation for Pseudo-Labeling
Not on the map. Why?
Abstract (arXiv)
Semi-supervised learning is a critical tool in reducing machine learning's dependence on labeled data. It has been successfully applied to structured data, such as images and natural language, by exploiting the inherent spatial and semantic structure therein with pretrained models or data augmentation. These methods are not applicable, however, when the data does not have the appropriate structure, or invariances. Due to their simplicity, pseudo-labeling (PL) methods can be widely used without any domain assumptions. However, the greedy mechanism in PL is sensitive to a threshold and can perform poorly if wrong assignments are made due to overconfidence. This paper studies theoretically the role of uncertainty to pseudo-labeling and proposes Confident Sinkhorn Allocation (CSA), which identifies the best pseudo-label allocation via optimal transport to only samples with high confidence scores. CSA outperforms the current state-of-the-art in this practically important area of semi-supervised learning. Additionally, we propose to use the Integral Probability Metrics to extend and improve the existing PACBayes bound which relies on the Kullback-Leibler (KL) divergence, for ensemble models. Our code is publicly available at https://github.com/amzn/confident-sinkhorn-allocation.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 38 of its 79 references that are in the map. First posted on arXiv 13 Jun 2022.
Builds on
38 of 79 references.
References · 38
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
- 2016Wide Residual Networkscited by 3,070
- 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
- 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
- 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
- 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
- 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
26 more
- 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 540
- 2021Tabular Data: Deep Learning is Not All You Needcited by 535
- 2019Unlabeled Data Improves Adversarial Robustnesscited by 416
- 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
- 2020Rethinking the Value of Labels for Improving Class-Imbalanced Learningcited by 216
- 2020ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learningcited by 209
- 2021Dash: Semi-Supervised Learning with Dynamic Thresholdingcited by 148
- 2018Computational Optimal Transport: Complexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithmcited by 112
- 2019Overcoming Catastrophic Forgetting with Unlabeled Data in the Wildcited by 111
- 2022Retrieval Augmented Classification for Long-Tail Visual Recognitioncited by 106
- 2019Learning with minibatch Wasserstein : asymptotic and gradient propertiescited by 87
- 2021RETRIEVE: Coreset Selection for Efficient and Robust Semi-Supervised Learningcited by 74
- 2020Uncertainty in Gradient Boosting via Ensemblescited by 56
- 2020Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learningcited by 53
- 2021Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Datacited by 50
- 2015Explaining the Success of AdaBoost and Random Forests as Interpolating Classifierscited by 45
- 2016A Unified View of Multi-Label Performance Measurescited by 36
- 2020Second Order PAC-Bayesian Bounds for the Weighted Majority Votecited by 34
- 2021Sinkhorn Label Allocation: Semi-Supervised Classification via Annealed Self-Trainingcited by 32
- 2021Multiview Pseudo-Labeling for Semi-supervised Learning from Videocited by 32
- 2021DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samplescited by 18
- 2020Deep Reinforcement Learning for Efficient Measurement of Quantum Devicescited by 15
- 2020Quantum device fine-tuning using unsupervised embedding learningcited by 14
- 2020Adaptive XGBoost for Evolving Data Streamscited by 8
- 2020Semi-Supervised Learning with Meta-Gradientcited by 7
- 2021Label Propagation Through Optimal Transportcited by 4
Built on by
Citing papers per year: 2023: 1; 2024: 2.
3 papers.