# Learning Bounds for Open-Set Learning

arXiv: https://arxiv.org/abs/2106.15792 · HTML: https://mlsanity.com/paper/2106.15792/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 59 references that are in the map.
- First posted on arXiv 30 Jun 2021.

## Builds on (most cited first, 30 of 30)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)
- [Reducing Network Agnostophobia](https://mlsanity.com/paper/1811.04110/) (2018)
- [Generative OpenMax for Multi-Class Open Set Classification](https://mlsanity.com/paper/1707.07418/) (2017)
- [Adversarial Reciprocal Points Learning for Open Set Recognition](https://mlsanity.com/paper/2103.00953/) (2021)
- [Classification-Reconstruction Learning for Open-Set Recognition](https://mlsanity.com/paper/1812.04246/) (2018)
- [C2AE: Class Conditioned Auto-Encoder for Open-set Recognition](https://mlsanity.com/paper/1904.01198/) (2019)
- [Learning Deep Kernels for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2002.09116/) (2020)
- [Learning Open Set Network with Discriminative Reciprocal Points](https://mlsanity.com/paper/2011.00178/) (2020)
- [Conditional Gaussian Distribution Learning for Open Set Recognition](https://mlsanity.com/paper/2003.08823/) (2020)
- [Rethinking Importance Weighting for Deep Learning under Distribution Shift](https://mlsanity.com/paper/2006.04662/) (2020)
- [The Extreme Value Machine](https://mlsanity.com/paper/1506.06112/) (2015)
- [Invertible Denoising Network: A Light Solution for Real Noise Removal](https://mlsanity.com/paper/2104.10546/) (2021)
- [Progressive Graph Learning for Open-Set Domain Adaptation](https://mlsanity.com/paper/2006.12087/) (2020)
- [What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation](https://mlsanity.com/paper/2004.11500/) (2020)
- [Open Set Domain Adaptation: Theoretical Bound and Algorithm](https://mlsanity.com/paper/1907.08375/) (2019)
- [Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning](https://mlsanity.com/paper/1603.03130/) (2016)
- [Open Category Detection with PAC Guarantees](https://mlsanity.com/paper/1808.00529/) (2018)
- [Open Set Domain Adaptation for Image and Action Recognition](https://mlsanity.com/paper/1907.12865/) (2019)
- [Adversarial Bipartite Graph Learning for Video Domain Adaptation](https://mlsanity.com/paper/2007.15829/) (2020)
- [Rethinking Softmax with Cross-Entropy: Neural Network Classifier as Mutual Information Estimator](https://mlsanity.com/paper/1911.10688/) (2019)
- [CSCL: Critical Semantic-Consistent Learning for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2008.10464/) (2020)
- [How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches?](https://mlsanity.com/paper/2101.01104/) (2020)
- [Semantic-Transferable Weakly-Supervised Endoscopic Lesions Segmentation](https://mlsanity.com/paper/1908.07669/) (2019)
- [Butterfly: One-step Approach towards Wildly Unsupervised Domain Adaptation](https://mlsanity.com/paper/1905.07720/) (2019)
- [Bridging the Theoretical Bound and Deep Algorithms for Open Set Domain Adaptation](https://mlsanity.com/paper/2006.13022/) (2020)
- [Weakly-Supervised Cross-Domain Adaptation for Endoscopic Lesions Segmentation](https://mlsanity.com/paper/2012.04170/) (2020)

## Built on by (most cited first, 27 of 27)

- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Federated Class-Incremental Learning](https://mlsanity.com/paper/2203.11473/) (2022)
- [Is Out-of-Distribution Detection Learnable?](https://mlsanity.com/paper/2210.14707/) (2022)
- [Learning to Augment Distributions for Out-of-Distribution Detection](https://mlsanity.com/paper/2311.01796/) (2023)
- [Understanding the Feature Norm for Out-of-Distribution Detection](https://mlsanity.com/paper/2310.05316/) (2023)
- [Open Set Action Recognition via Multi-Label Evidential Learning](https://mlsanity.com/paper/2303.12698/) (2023)
- [Selective Knowledge Sharing for Privacy-Preserving Federated Distillation without A Good Teacher](https://mlsanity.com/paper/2304.01731/) (2023)
- [OpenTAL: Towards Open Set Temporal Action Localization](https://mlsanity.com/paper/2203.05114/) (2022)
- [Towards Accurate Open-Set Recognition via Background-Class Regularization](https://mlsanity.com/paper/2207.10287/) (2022)
- [Meta OOD Learning for Continuously Adaptive OOD Detection](https://mlsanity.com/paper/2309.11705/) (2023)
- [Conditional Extreme Value Theory for Open Set Video Domain Adaptation](https://mlsanity.com/paper/2109.00522/) (2021)
- [Adaptive Hierarchical Graph Cut for Multi-granularity Out-of-distribution Detection](https://mlsanity.com/paper/2412.15668/) (2024)
- [A Survey on Open-Set Image Recognition](https://mlsanity.com/paper/2312.15571/) (2023)
- [A Novel Data Augmentation Technique for Out-of-Distribution Sample Detection using Compounded Corruptions](https://mlsanity.com/paper/2207.13916/) (2022)
- [A Unified Data Representation Learning for Non-parametric Two-sample Testing](https://mlsanity.com/paper/2412.00613/) (2024)
- [Mitigating Group Bias in Federated Learning for Heterogeneous Devices](https://mlsanity.com/paper/2309.07085/) (2023)
- [GearNet: Stepwise Dual Learning for Weakly Supervised Domain Adaptation](https://mlsanity.com/paper/2201.06001/) (2022)
- [Rethinking Guidance Information to Utilize Unlabeled Samples:A Label Encoding Perspective](https://mlsanity.com/paper/2406.02862/) (2024)
- [Hierarchical Visual Categories Modeling: A Joint Representation Learning and Density Estimation Framework for Out-of-Distribution Detection](https://mlsanity.com/paper/2408.15580/) (2024)
- [Generalizable Prompt Learning of CLIP: A Brief Overview](https://mlsanity.com/paper/2503.01263/) (2025)
- [Understanding Open-Set Recognition by Jacobian Norm and Inter-Class Separation](https://mlsanity.com/paper/2209.11436/) (2022)
- [Latent Space Energy-based Model for Fine-grained Open Set Recognition](https://mlsanity.com/paper/2309.10711/) (2023)
- [Advancing Prompt Learning through an External Layer](https://mlsanity.com/paper/2407.19674/) (2024)
- [Exploring Open-world Continual Learning with Knowns-Unknowns Knowledge Transfer](https://mlsanity.com/paper/2502.20124/) (2025)
- [Contrastive variational information bottleneck for aspect-based sentiment analysis](https://mlsanity.com/paper/2303.02846/) (2023)
- [Dynamic Against Dynamic: An Open-set Self-learning Framework](https://mlsanity.com/paper/2404.17830/) (2024)
- [How Out-of-Distribution Detection Learning Theory Enhances Transformer: Learnability and Reliability](https://mlsanity.com/paper/2406.12915/) (2024)

## Abstract (arXiv)

Traditional supervised learning aims to train a classifier in the closed-set world, where training and test samples share the same label space. In this paper, we target a more challenging and realistic setting: open-set learning (OSL), where there exist test samples from the classes that are unseen during training. Although researchers have designed many methods from the algorithmic perspectives, there are few methods that provide generalization guarantees on their ability to achieve consistent performance on different training samples drawn from the same distribution. Motivated by the transfer learning and probably approximate correct (PAC) theory, we make a bold attempt to study OSL by proving its generalization error-given training samples with size n, the estimation error will get close to order Op(1/√n). This is the first study to provide a generalization bound for OSL, which we do by theoretically investigating the risk of the target classifier on unknown classes. According to our theory, a novel algorithm, called auxiliary open-set risk (AOSR) is proposed to address the OSL problem. Experiments verify the efficacy of AOSR. The code is available at github.com/Anjin-Liu/OpensetLearningAOSR.