# Exploring Self-Supervised Regularization for Supervised and Semi-Supervised Learning

arXiv: https://arxiv.org/abs/1906.10343 · HTML: https://mlsanity.com/paper/1906.10343/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jun 2019.

## Built on by (most cited first, 4 of 4)

- [Quadratic Residual Networks: A New Class of Neural Networks for Solving Forward and Inverse Problems in Physics Involving PDEs](https://mlsanity.com/paper/2101.08366/) (2021)
- [SSLayout360: Semi-Supervised Indoor Layout Estimation from 360-Degree Panorama](https://mlsanity.com/paper/2103.13696/) (2021)
- [Understanding the Role of Self-Supervised Learning in Out-of-Distribution Detection Task](https://mlsanity.com/paper/2110.13435/) (2021)
- [Semi-Supervised 360 Layout Estimation with Panoramic Collaborative Perturbations](https://mlsanity.com/paper/2503.01114/) (2025)

## Abstract (arXiv)

Recent advances in semi-supervised learning have shown tremendous potential in overcoming a major barrier to the success of modern machine learning algorithms: access to vast amounts of human-labeled training data. Previous algorithms based on consistency regularization can harness the abundance of unlabeled data to produce impressive results on a number of semi-supervised benchmarks, approaching the performance of strong supervised baselines using only a fraction of the available labeled data. In this work, we challenge the long-standing success of consistency regularization by introducing self-supervised regularization as the basis for combining semantic feature representations from unlabeled data. We perform extensive comparative experiments to demonstrate the effectiveness of self-supervised regularization for supervised and semi-supervised image classification on SVHN, CIFAR-10, and CIFAR-100 benchmark datasets. We present two main results: (1) models augmented with self-supervised regularization significantly improve upon traditional supervised classifiers without the need for unlabeled data; (2) together with unlabeled data, our models yield semi-supervised performance competitive with, and in many cases exceeding, prior state-of-the-art consistency baselines. Lastly, our models have the practical utility of being efficiently trained end-to-end and require no additional hyper-parameters to tune for optimal performance beyond the standard set for training neural networks. Reference code and data are available at https://github.com/vuptran/sesemi