BackMix: Regularizing Open Set Recognition by Removing Underlying Fore-Background Priors
Not on the map. Why?
Abstract (arXiv)
Open set recognition (OSR) requires models to classify known samples while detecting unknown samples for real-world applications. Existing studies show impressive progress using unknown samples from auxiliary datasets to regularize OSR models, but they have proved to be sensitive to selecting such known outliers. In this paper, we discuss the aforementioned problem from a new perspective: Can we regularize OSR models without elaborately selecting auxiliary known outliers? We first empirically and theoretically explore the role of foregrounds and backgrounds in open set recognition and disclose that: 1) backgrounds that correlate with foregrounds would mislead the model and cause failures when encounters 'partially' known images; 2) Backgrounds unrelated to foregrounds can serve as auxiliary known outliers and provide regularization via global average pooling. Based on the above insights, we propose a new method, Background Mix (BackMix), that mixes the foreground of an image with different backgrounds to remove the underlying fore-background priors. Specifically, BackMix first estimates the foreground with class activation maps (CAMs), then randomly replaces image patches with backgrounds from other images to obtain mixed images for training. With backgrounds de-correlated from foregrounds, the open set recognition performance is significantly improved. The proposed method is quite simple to implement, requires no extra operation for inferences, and can be seamlessly integrated into almost all of the existing frameworks. The code is released on https://github.com/Vanixxz/BackMix.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 59 of its 86 references that are in the map. First posted on arXiv 22 Mar 2025.
Builds on
59 of 86 references.
References · 59
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
- 2015SSD: Single Shot MultiBox Detectorcited by 3,130
- 2016Wide Residual Networkscited by 3,070
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
38 more
- 2015Learning Deep Features for Discriminative Localizationcited by 2,436
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
- 2021Learning to Prompt for Vision-Language Modelscited by 2,281
- 2013Describing Textures in the Wildcited by 2,125
- 2022Visual Prompt Tuningcited by 1,681
- 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
- 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
- 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
- 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
- 2020Energy-based Out-of-distribution Detectioncited by 1,299
- 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
- 2018Deep Anomaly Detection with Outlier Exposurecited by 942
- 2014Show and Tell: A Neural Image Caption Generatorcited by 879
- 2013Network In Networkcited by 605
- 2021Domain Generalization with MixStylecited by 566
- 2015Towards Open Set Deep Networkscited by 538
- 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 514
- 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
- 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 353
- 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 343
- 2020Environment Inference for Invariant Learningcited by 306
- 2017Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-supervised Object and Action Localizationcited by 207
- 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
- 2015TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Trackingcited by 165
- 2018Reducing Network Agnostophobiacited by 165
- 2021Adversarial Reciprocal Points Learning for Open Set Recognitioncited by 153
- 2018Classification-Reconstruction Learning for Open-Set Recognitioncited by 146
- 2019C2AE: Class Conditioned Auto-Encoder for Open-set Recognitioncited by 142
- 2021OpenGAN: Open-Set Recognition via Open Data Generationcited by 135
- 2020GridMask Data Augmentationcited by 123
- 2021Learning Placeholders for Open-Set Recognitioncited by 121
- 2020Learning Open Set Network with Discriminative Reciprocal Pointscited by 115
- 2023LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learningcited by 107
- 2020Conditional Gaussian Distribution Learning for Open Set Recognitioncited by 104
- 2020Are Labels Always Necessary for Classifier Accuracy Evaluation?cited by 104
- 2021On the Impact of Spurious Correlation for Out-of-distribution Detectioncited by 68
- 2022MaskTune: Mitigating Spurious Correlations by Forcing to Explorecited by 66
- 2020Robustness to Spurious Correlations via Human Annotationscited by 62
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.