# FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learning

arXiv: https://arxiv.org/abs/2310.16412 · HTML: https://mlsanity.com/paper/2310.16412/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 85 references that are in the map.
- First posted on arXiv 25 Oct 2023.

## Builds on (most cited first, 30 of 53)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Unsupervised Data Augmentation for Consistency Training](https://mlsanity.com/paper/1904.12848/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [Exploring Generalization in Deep Learning](https://mlsanity.com/paper/1706.08947/) (2017)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [SWAD: Domain Generalization by Seeking Flat Minima](https://mlsanity.com/paper/2102.08604/) (2021)
- [Meta Pseudo Labels](https://mlsanity.com/paper/2003.10580/) (2020)
- [Entropy-SGD: Biasing Gradient Descent Into Wide Valleys](https://mlsanity.com/paper/1611.01838/) (2016)
- [ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring](https://mlsanity.com/paper/1911.09785/) (2019)
- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks](https://mlsanity.com/paper/2102.11600/) (2021)
- [FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning](https://mlsanity.com/paper/2205.07246/) (2022)
- [Open-World Semi-Supervised Learning](https://mlsanity.com/paper/2102.03526/) (2021)
- [Dash: Semi-Supervised Learning with Dynamic Thresholding](https://mlsanity.com/paper/2109.00650/) (2021)
- [Towards Understanding Sharpness-Aware Minimization](https://mlsanity.com/paper/2206.06232/) (2022)

## Built on by (most cited first, 20 of 20)

- [Is Gradient Ascent Really Necessary? Memorize to Forget for Machine Unlearning](https://mlsanity.com/paper/2602.06441/) (2026)
- [(FL)2: Overcoming Few Labels in Federated Semi-Supervised Learning](https://mlsanity.com/paper/2410.23227/) (2024)
- [Multimodal Classification via Modal-Aware Interactive Enhancement](https://mlsanity.com/paper/2407.04587/) (2024)
- [STS MICCAI 2023 Challenge: Grand challenge on 2D and 3D semi-supervised tooth segmentation](https://mlsanity.com/paper/2407.13246/) (2024)
- [Re-Evaluating the Impact of Unseen-Class Unlabeled Data on Semi-Supervised Learning Model](https://mlsanity.com/paper/2503.00884/) (2025)
- [Cycle Training with Semi-Supervised Domain Adaptation: Bridging Accuracy and Efficiency for Real-Time Mobile Scene Detection](https://mlsanity.com/paper/2504.09297/) (2025)
- [i-PhysGaussian: Implicit Physical Simulation for 3D Gaussian Splatting](https://mlsanity.com/paper/2602.17117/) (2026)
- [AdaptGCD: Multi-Expert Adapter Tuning for Generalized Category Discovery](https://mlsanity.com/paper/2410.21705/) (2024)
- [Decouple then Converge: Handling Unknown Unlabeled Distributions in Long-Tailed Semi-Supervised Learning](https://mlsanity.com/paper/2406.13187/) (2024)
- [Sharpness-aware Dynamic Anchor Selection for Generalized Category Discovery](https://mlsanity.com/paper/2512.12925/) (2025)
- [InterLUDE: Interactions between Labeled and Unlabeled Data to Enhance Semi-Supervised Learning](https://mlsanity.com/paper/2403.10658/) (2024)
- [Smooth Pseudo-Labeling](https://mlsanity.com/paper/2405.14313/) (2024)
- [LayerMatch: Do Pseudo-labels Benefit All Layers?](https://mlsanity.com/paper/2406.14207/) (2024)
- [Decorrelating Structure via Adapters Makes Ensemble Learning Practical for Semi-supervised Learning](https://mlsanity.com/paper/2408.04150/) (2024)
- [Uncertainty-aware Long-tailed Weights Model the Utility of Pseudo-labels for Semi-supervised Learning](https://mlsanity.com/paper/2503.09974/) (2025)
- [An Adaptor for Triggering Semi-Supervised Learning to Out-of-Box Serve Deep Image Clustering](https://mlsanity.com/paper/2509.20976/) (2025)
- [SpikeMatch: Semi-Supervised Learning with Temporal Dynamics of Spiking Neural Networks](https://mlsanity.com/paper/2509.22581/) (2025)
- [Theoretical Analysis of Measure Consistency Regularization for Partially Observed Data](https://mlsanity.com/paper/2602.01437/) (2026)
- [BrokenBind: Universal Modality Exploration beyond Dataset Boundaries](https://mlsanity.com/paper/2602.06451/) (2026)
- [JEPAMatch: Geometric Representation Shaping for Semi-Supervised Learning](https://mlsanity.com/paper/2604.21046/) (2026)

## Abstract (arXiv)

Semi-Supervised Learning (SSL) has been an effective way to leverage abundant unlabeled data with extremely scarce labeled data. However, most SSL methods are commonly based on instance-wise consistency between different data transformations. Therefore, the label guidance on labeled data is hard to be propagated to unlabeled data. Consequently, the learning process on labeled data is much faster than on unlabeled data which is likely to fall into a local minima that does not favor unlabeled data, leading to sub-optimal generalization performance. In this paper, we propose FlatMatch which minimizes a cross-sharpness measure to ensure consistent learning performance between the two datasets. Specifically, we increase the empirical risk on labeled data to obtain a worst-case model which is a failure case that needs to be enhanced. Then, by leveraging the richness of unlabeled data, we penalize the prediction difference (i.e., cross-sharpness) between the worst-case model and the original model so that the learning direction is beneficial to generalization on unlabeled data. Therefore, we can calibrate the learning process without being limited to insufficient label information. As a result, the mismatched learning performance can be mitigated, further enabling the effective exploitation of unlabeled data and improving SSL performance. Through comprehensive validation, we show FlatMatch achieves state-of-the-art results in many SSL settings.