# Class-Specific Semantic Reconstruction for Open Set Recognition

arXiv: https://arxiv.org/abs/2207.02158 · HTML: https://mlsanity.com/paper/2207.02158/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 56 references that are in the map.
- First posted on arXiv 5 Jul 2022.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [Scaling Out-of-Distribution Detection for Real-World Settings](https://mlsanity.com/paper/1911.11132/) (2019)
- [CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances](https://mlsanity.com/paper/2007.08176/) (2020)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)
- [Deep Anomaly Detection Using Geometric Transformations](https://mlsanity.com/paper/1805.10917/) (2018)
- [SSD: A Unified Framework for Self-Supervised Outlier Detection](https://mlsanity.com/paper/2103.12051/) (2021)
- [Towards Open World Recognition](https://mlsanity.com/paper/1412.5687/) (2014)
- [OCGAN: One-class Novelty Detection Using GANs with Constrained Latent Representations](https://mlsanity.com/paper/1903.08550/) (2019)
- [Contrastive Training for Improved Out-of-Distribution Detection](https://mlsanity.com/paper/2007.05566/) (2020)
- [Reducing Network Agnostophobia](https://mlsanity.com/paper/1811.04110/) (2018)
- [Adversarial Reciprocal Points Learning for Open Set Recognition](https://mlsanity.com/paper/2103.00953/) (2021)
- [Classification-Reconstruction Learning for Open-Set Recognition](https://mlsanity.com/paper/1812.04246/) (2018)
- [C2AE: Class Conditioned Auto-Encoder for Open-set Recognition](https://mlsanity.com/paper/1904.01198/) (2019)
- [OpenGAN: Open-Set Recognition via Open Data Generation](https://mlsanity.com/paper/2104.02939/) (2021)
- [Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifiers](https://mlsanity.com/paper/1809.03576/) (2018)
- [Learning Placeholders for Open-Set Recognition](https://mlsanity.com/paper/2103.15086/) (2021)

## Built on by (most cited first, 28 of 28)

- [Unified Entropy Optimization for Open-Set Test-Time Adaptation](https://mlsanity.com/paper/2404.06065/) (2024)
- [ProtoGCD: Unified and Unbiased Prototype Learning for Generalized Category Discovery](https://mlsanity.com/paper/2504.03755/) (2025)
- [WiOpen: A Robust Wi-Fi-based Open-set Gesture Recognition Framework](https://mlsanity.com/paper/2402.00822/) (2024)
- [Towards Trustworthy Dataset Distillation](https://mlsanity.com/paper/2307.09165/) (2023)
- [Handling Out-of-Distribution Data: A Survey](https://mlsanity.com/paper/2507.21160/) (2025)
- [Open-Set Biometrics: Beyond Good Closed-Set Models](https://mlsanity.com/paper/2407.16133/) (2024)
- [Advancing Open-Set Domain Generalization Using Evidential Bi-Level Hardest Domain Scheduler](https://mlsanity.com/paper/2409.17555/) (2024)
- [LMC: Large Model Collaboration with Cross-assessment for Training-Free Open-Set Object Recognition](https://mlsanity.com/paper/2309.12780/) (2023)
- [Unified Classification and Rejection: A One-versus-All Framework](https://mlsanity.com/paper/2311.13355/) (2023)
- [A Survey on Open-Set Image Recognition](https://mlsanity.com/paper/2312.15571/) (2023)
- [Open Set Classification of GAN-based Image Manipulations via a ViT-based Hybrid Architecture](https://mlsanity.com/paper/2304.05212/) (2023)
- [BOSC: A Backdoor-based Framework for Open Set Synthetic Image Attribution](https://mlsanity.com/paper/2405.11491/) (2024)
- [Mitigating Label Noise using Prompt-Based Hyperbolic Meta-Learning in Open-Set Domain Generalization](https://mlsanity.com/paper/2412.18342/) (2024)
- [Embrace Limited and Imperfect Training Datasets: Opportunities and Challenges in Plant Disease Recognition Using Deep Learning](https://mlsanity.com/paper/2305.11533/) (2023)
- [Revealing the Two Sides of Data Augmentation: An Asymmetric Distillation-based Win-Win Solution for Open-Set Recognition](https://mlsanity.com/paper/2404.19527/) (2024)
- [BackMix: Regularizing Open Set Recognition by Removing Underlying Fore-Background Priors](https://mlsanity.com/paper/2503.17717/) (2025)
- [Understanding Open-Set Recognition by Jacobian Norm and Inter-Class Separation](https://mlsanity.com/paper/2209.11436/) (2022)
- [OpenSlot: Mixed Open-Set Recognition with Object-Centric Learning](https://mlsanity.com/paper/2407.02386/) (2024)
- [Exploring Open-world Continual Learning with Knowns-Unknowns Knowledge Transfer](https://mlsanity.com/paper/2502.20124/) (2025)
- [Dynamic Against Dynamic: An Open-set Self-learning Framework](https://mlsanity.com/paper/2404.17830/) (2024)
- [SphOR: A Representation Learning Perspective on Open-set Recognition for Identifying Unknown Classes in Deep Learning Models](https://mlsanity.com/paper/2503.08049/) (2025)
- [Spatial-Temporal Attention Network for Open-Set Fine-Grained Image Recognition](https://mlsanity.com/paper/2211.13940/) (2022)
- [Complementary Frequency-Varying Awareness Network for Open-Set Fine-Grained Image Recognition](https://mlsanity.com/paper/2307.07214/) (2023)
- [Recursive Counterfactual Deconfounding for Object Recognition](https://mlsanity.com/paper/2309.13924/) (2023)
- [Towards Open-set Gesture Recognition via Feature Activation Enhancement and Orthogonal Prototype Learning](https://mlsanity.com/paper/2312.02535/) (2023)
- [Multi-method Integration with Confidence-based Weighting for Zero-shot Image Classification](https://mlsanity.com/paper/2405.02155/) (2024)
- [COSTARR: Consolidated Open Set Technique with Attenuation for Robust Recognition](https://mlsanity.com/paper/2508.01087/) (2025)
- [SmoothOperator: Enhancing Representations for Fine-grained Open-set Recognition via Modulated Label Smoothing](https://mlsanity.com/paper/2610.00851/) (2026)

## Abstract (arXiv)

Open set recognition enables deep neural networks (DNNs) to identify samples of unknown classes, while maintaining high classification accuracy on samples of known classes. Existing methods basing on auto-encoder (AE) and prototype learning show great potential in handling this challenging task. In this study, we propose a novel method, called Class-Specific Semantic Reconstruction (CSSR), that integrates the power of AE and prototype learning. Specifically, CSSR replaces prototype points with manifolds represented by class-specific AEs. Unlike conventional prototype-based methods, CSSR models each known class on an individual AE manifold, and measures class belongingness through AE's reconstruction error. Class-specific AEs are plugged into the top of the DNN backbone and reconstruct the semantic representations learned by the DNN instead of the raw image. Through end-to-end learning, the DNN and the AEs boost each other to learn both discriminative and representative information. The results of experiments conducted on multiple datasets show that the proposed method achieves outstanding performance in both close and open set recognition and is sufficiently simple and flexible to incorporate into existing frameworks.