# Universum Prescription: Regularization using Unlabeled Data

arXiv: https://arxiv.org/abs/1511.03719 · HTML: https://mlsanity.com/paper/1511.03719/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2015.

## Built on by (most cited first, 16 of 16)

- [SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation](https://mlsanity.com/paper/2104.14812/) (2021)
- [Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions](https://mlsanity.com/paper/2104.03829/) (2021)
- [Taxonomy of Machine Learning Safety: A Survey and Primer](https://mlsanity.com/paper/2106.04823/) (2021)
- [Being a Bit Frequentist Improves Bayesian Neural Networks](https://mlsanity.com/paper/2106.10065/) (2021)
- [Correct-by-Construction Runtime Enforcement in AI -- A Survey](https://mlsanity.com/paper/2208.14426/) (2022)
- [DOC3-Deep One Class Classification using Contradictions](https://mlsanity.com/paper/2105.07636/) (2021)
- [Towards Consistent Predictive Confidence through Fitted Ensembles](https://mlsanity.com/paper/2106.12070/) (2021)
- [Class-Aware Universum Inspired Re-Balance Learning for Long-Tailed Recognition](https://mlsanity.com/paper/2207.12808/) (2022)
- [All Beings Are Equal in Open Set Recognition](https://mlsanity.com/paper/2401.17654/) (2024)
- [Universum-inspired Supervised Contrastive Learning](https://mlsanity.com/paper/2204.10695/) (2022)
- [Automatic Large Scale Detection of Red Palm Weevil Infestation using Aerial and Street View Images](https://mlsanity.com/paper/2104.02598/) (2021)
- [Universum GANs: Improving GANs through contradictions](https://mlsanity.com/paper/2106.09946/) (2021)
- [One Representation to Rule Them All: Identifying Out-of-Support Examples in Few-shot Learning with Generic Representations](https://mlsanity.com/paper/2106.01423/) (2021)
- [On Feature Diversity in Energy-based Models](https://mlsanity.com/paper/2306.01489/) (2023)
- [Defensive Perception: Estimation and Monitoring of Neural Network Performance under Deployment](https://mlsanity.com/paper/2308.06299/) (2023)
- [Preventing Arbitrarily High Confidence on Far-Away Data in Point-Estimated Discriminative Neural Networks](https://mlsanity.com/paper/2311.03683/) (2023)

## Abstract (arXiv)

This paper shows that simply prescribing "none of the above" labels to unlabeled data has a beneficial regularization effect to supervised learning. We call it universum prescription by the fact that the prescribed labels cannot be one of the supervised labels. In spite of its simplicity, universum prescription obtained competitive results in training deep convolutional networks for CIFAR-10, CIFAR-100, STL-10 and ImageNet datasets. A qualitative justification of these approaches using Rademacher complexity is presented. The effect of a regularization parameter -- probability of sampling from unlabeled data -- is also studied empirically.