mlsanityData to 7 Oct 2026

SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning

arXiv 2301.10921 ↗First posted 26 Jan 2023cs.LG cs.AI cs.CVcited by 133

Not on the map. Why?

Abstract (arXiv)

The critical challenge of Semi-Supervised Learning (SSL) is how to effectively leverage the limited labeled data and massive unlabeled data to improve the model's generalization performance. In this paper, we first revisit the popular pseudo-labeling methods via a unified sample weighting formulation and demonstrate the inherent quantity-quality trade-off problem of pseudo-labeling with thresholding, which may prohibit learning. To this end, we propose SoftMatch to overcome the trade-off by maintaining both high quantity and high quality of pseudo-labels during training, effectively exploiting the unlabeled data. We derive a truncated Gaussian function to weight samples based on their confidence, which can be viewed as a soft version of the confidence threshold. We further enhance the utilization of weakly-learned classes by proposing a uniform alignment approach. In experiments, SoftMatch shows substantial improvements across a wide variety of benchmarks, including image, text, and imbalanced classification.

Key facts. Cited by 133 papers in the map (data through 7 Oct 2026). Builds on 37 of its 55 references that are in the map. First posted on arXiv 26 Jan 2023.

Builds on

37 of 55 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2016Wide Residual Networkscited by 3,057
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  9. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  10. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  11. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  12. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
25 more
  1. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  2. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  3. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  4. 2021FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labelingcited by 537
  5. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 507
  6. 2021Semi-Supervised Semantic Segmentation with Cross Pseudo Supervisioncited by 462
  7. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  8. 2020Meta Pseudo Labelscited by 372
  9. 2021End-to-End Semi-Supervised Object Detection with Soft Teachercited by 315
  10. 2019Label Propagation for Deep Semi-supervised Learningcited by 287
  11. 2021CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learningcited by 174
  12. 2020CoMatch: Semi-supervised Learning with Contrastive Graph Regularizationcited by 167
  13. 2021Dash: Semi-Supervised Learning with Dynamic Thresholdingcited by 147
  14. 2022SimMatch: Semi-supervised Learning with Similarity Matchingcited by 118
  15. 2020Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learningcited by 105
  16. 2021AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptationcited by 104
  17. 2021Semi-Supervised Semantic Segmentation via Adaptive Equalization Learningcited by 101
  18. 2022USB: A Unified Semi-supervised Learning Benchmark for Classificationcited by 95
  19. 2022Debiased Learning from Naturally Imbalanced Pseudo-Labelscited by 85
  20. 2021ABC: Auxiliary Balanced Classifier for Class-imbalanced Semi-supervised Learningcited by 65
  21. 2020Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learningcited by 53
  22. 2021CoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learningcited by 41
  23. 2021Revisiting Consistency Regularization for Semi-Supervised Learningcited by 34
  24. 2021Sinkhorn Label Allocation: Semi-Supervised Classification via Annealed Self-Trainingcited by 32
  25. 2022ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency Regularizationcited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 26; 2024: 49; 2025: 31; 2026: 27.

133 papers.

Sort:

By area:

  1. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  2. 2024DreamDA: Generative Data Augmentation with Diffusion Modelscited by 24
  3. 2023LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learningcited by 20
  4. 2023FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learningcited by 20
  5. 2024Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learningcited by 19
  6. 2024SAM as the Guide: Mastering Pseudo-Label Refinement in Semi-Supervised Referring Expression Segmentationcited by 17
  7. 2024Towards the Uncharted: Density-Descending Feature Perturbation for Semi-supervised Semantic Segmentationcited by 16
  8. 2023SemiReward: A General Reward Model for Semi-supervised Learningcited by 14
  9. 2023All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentationcited by 13
  10. 2023InstanT: Semi-supervised Learning with Instance-dependent Thresholdscited by 13
  11. 2023Segment Together: A Versatile Paradigm for Semi-Supervised Medical Image Segmentationcited by 12
  12. 2024CamoTeacher: Dual-Rotation Consistency Learning for Semi-Supervised Camouflaged Object Detectioncited by 12
38 more
  1. 2025STiL: Semi-supervised Tabular-Image Learning for Comprehensive Task-Relevant Information Exploration in Multimodal Classificationcited by 12
  2. 2025CGMatch: A Different Perspective of Semi-supervised Learningcited by 11
  3. 2023EEGMatch: Learning with Incomplete Labels for Semi-Supervised EEG-based Cross-Subject Emotion Recognitioncited by 10
  4. 2023Semi-Supervised Dual-Stream Self-Attentive Adversarial Graph Contrastive Learning for Cross-Subject EEG-based Emotion Recognitioncited by 10
  5. 2024CDMAD: Class-Distribution-Mismatch-Aware Debiasing for Class-Imbalanced Semi-Supervised Learningcited by 10
  6. 2024Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognitioncited by 10
  7. 2025MOS: Modeling Object-Scene Associations in Generalized Category Discoverycited by 10
  8. 2025CustomKD: Customizing Large Vision Foundation for Edge Model Improvement via Knowledge Distillationcited by 10
  9. 2023Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurationscited by 9
  10. 2024Foster Adaptivity and Balance in Learning with Noisy Labelscited by 9
  11. 2023RelationMatch: Matching In-batch Relationships for Semi-supervised Learningcited by 8
  12. 2024Towards Generalizing to Unseen Domains with Few Labelscited by 8
  13. 2024Noisy Label Processing for Classification: A Surveycited by 8
  14. 2023Flexible Distribution Alignment: Towards Long-tailed Semi-supervised Learning with Proper Calibrationcited by 7
  15. 2024A General Framework for Learning from Weak Supervisioncited by 7
  16. 2025A Square Peg in a Square Hole: Meta-Expert for Long-Tailed Semi-Supervised Learningcited by 7
  17. 2023SEAL: Simultaneous Label Hierarchy Exploration And Learningcited by 6
  18. 2023Semi-Supervised Learning for Multi-Label Cardiovascular Diseases Prediction:A Multi-Dataset Studycited by 6
  19. 2023OTMatch: Improving Semi-Supervised Learning with Optimal Transportcited by 6
  20. 2024Learning with Imbalanced Noisy Data by Preventing Bias in Sample Selectioncited by 6
  21. 2024BEM: Balanced and Entropy-based Mix for Long-Tailed Semi-Supervised Learningcited by 6
  22. 2024InfoMatch: Entropy Neural Estimation for Semi-Supervised Image Classificationcited by 6
  23. 2024Boosting Single Positive Multi-label Classification with Generalized Robust Losscited by 6
  24. 2024RankUp: Boosting Semi-Supervised Regression with an Auxiliary Ranking Classifiercited by 6
  25. 2025Revisiting semi-supervised learning in the era of foundation modelscited by 6
  26. 2025When Confidence Fails: Revisiting Pseudo-Label Selection in Semi-supervised Semantic Segmentationcited by 6
  27. 2023High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Samplingcited by 5
  28. 2024ScholarChemQA: Unveiling the Power of Language Models in Chemical Research Question Answeringcited by 5
  29. 2024Domain-Guided Weight Modulation for Semi-Supervised Domain Generalizationcited by 5
  30. 2024Towards Understanding Why FixMatch Generalizes Better Than Supervised Learningcited by 5
  31. 2024(FL)2: Overcoming Few Labels in Federated Semi-Supervised Learningcited by 5
  32. 2025Variational Supervised Contrastive Learningcited by 5
  33. 2023Self-Evolving Neural Radiance Fieldscited by 4
  34. 2023FAGC:Feature Augmentation on Geodesic Curve in the Pre-Shape Spacecited by 4
  35. 2023UP-DP: Unsupervised Prompt Learning for Data Pre-Selection with Vision-Language Modelscited by 3
  36. 2023UCE-FID: Using Large Unlabeled, Medium Crowdsourced-Labeled, and Small Expert-Labeled Tweets for Foodborne Illness Detectioncited by 3
  37. 2024SIAVC: Semi-Supervised Framework for Industrial Accident Video Classificationcited by 3
  38. 2024Generalized Semi-Supervised Learning via Self-Supervised Feature Adaptationcited by 3

50 most cited of 133 All 133 (JSON)