# Smooth Neighbors on Teacher Graphs for Semi-supervised Learning

arXiv: https://arxiv.org/abs/1711.00258 · HTML: https://mlsanity.com/paper/1711.00258/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2017.

## Built on by (most cited first, 30 of 48)

- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [Augmentation Strategies for Learning with Noisy Labels](https://mlsanity.com/paper/2103.02130/) (2021)
- [ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning](https://mlsanity.com/paper/2101.00407/) (2021)
- [Semi-Supervised and Unsupervised Deep Visual Learning: A Survey](https://mlsanity.com/paper/2208.11296/) (2022)
- [Semi-supervised Semantic Segmentation with Prototype-based Consistency Regularization](https://mlsanity.com/paper/2210.04388/) (2022)
- [ConDA: Unsupervised Domain Adaptation for LiDAR Segmentation via Regularized Domain Concatenation](https://mlsanity.com/paper/2111.15242/) (2021)
- [CoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2112.04564/) (2021)
- [Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels](https://mlsanity.com/paper/2302.10586/) (2023)
- [Revisiting Consistency Regularization for Semi-Supervised Learning](https://mlsanity.com/paper/2112.05825/) (2021)
- [Pseudo-supervised Deep Subspace Clustering](https://mlsanity.com/paper/2104.03531/) (2021)
- [CiPR: An Efficient Framework with Cross-instance Positive Relations for Generalized Category Discovery](https://mlsanity.com/paper/2304.06928/) (2023)
- [All-Around Real Label Supervision: Cyclic Prototype Consistency Learning for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2109.13930/) (2021)
- [Image Data Augmentation Approaches: A Comprehensive Survey and Future directions](https://mlsanity.com/paper/2301.02830/) (2023)
- [Semi-supervised Multi-task Learning for Semantics and Depth](https://mlsanity.com/paper/2110.07197/) (2021)
- [Conservative-Progressive Collaborative Learning for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2211.16701/) (2022)
- [Fast Batch Nuclear-norm Maximization and Minimization for Robust Domain Adaptation](https://mlsanity.com/paper/2107.06154/) (2021)
- [Seminar Learning for Click-Level Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2108.13393/) (2021)
- [On Data-Augmentation and Consistency-Based Semi-Supervised Learning](https://mlsanity.com/paper/2101.06967/) (2021)
- [Estimating Soft Labels for Out-of-Domain Intent Detection](https://mlsanity.com/paper/2211.05561/) (2022)
- [Virtual Category Learning: A Semi-Supervised Learning Method for Dense Prediction with Extremely Limited Labels](https://mlsanity.com/paper/2312.01169/) (2023)
- [Flipped Classroom: Aligning Teacher Attention with Student in Generalized Category Discovery](https://mlsanity.com/paper/2409.19659/) (2024)
- [Semi-supervised Deep Learning for Image Classification with Distribution Mismatch: A Survey](https://mlsanity.com/paper/2203.00190/) (2022)
- [SemPPL: Predicting pseudo-labels for better contrastive representations](https://mlsanity.com/paper/2301.05158/) (2023)
- [LaplaceNet: A Hybrid Graph-Energy Neural Network for Deep Semi-Supervised Classification](https://mlsanity.com/paper/2106.04527/) (2021)
- [Semi-Supervised Building Footprint Generation with Feature and Output Consistency Training](https://mlsanity.com/paper/2205.08416/) (2022)
- [SLiDE: Self-supervised LiDAR De-snowing through Reconstruction Difficulty](https://mlsanity.com/paper/2208.04043/) (2022)
- [Unsupervised Domain Adaptation via Distilled Discriminative Clustering](https://mlsanity.com/paper/2302.11984/) (2023)
- [Meta Co-Training: Two Views are Better than One](https://mlsanity.com/paper/2311.18083/) (2023)
- [Predicting Out-of-Domain Generalization with Neighborhood Invariance](https://mlsanity.com/paper/2207.02093/) (2022)
- [Semi-Supervised Learning with Pseudo-Negative Labels for Image Classification](https://mlsanity.com/paper/2301.03976/) (2023)

## Abstract (arXiv)

The recently proposed self-ensembling methods have achieved promising results in deep semi-supervised learning, which penalize inconsistent predictions of unlabeled data under different perturbations. However, they only consider adding perturbations to each single data point, while ignoring the connections between data samples. In this paper, we propose a novel method, called Smooth Neighbors on Teacher Graphs (SNTG). In SNTG, a graph is constructed based on the predictions of the teacher model, i.e., the implicit self-ensemble of models. Then the graph serves as a similarity measure with respect to which the representations of "similar" neighboring points are learned to be smooth on the low-dimensional manifold. We achieve state-of-the-art results on semi-supervised learning benchmarks. The error rates are 9.89%, 3.99% for CIFAR-10 with 4000 labels, SVHN with 500 labels, respectively. In particular, the improvements are significant when the labels are fewer. For the non-augmented MNIST with only 20 labels, the error rate is reduced from previous 4.81% to 1.36%. Our method also shows robustness to noisy labels.