# Unexplored Faces of Robustness and Out-of-Distribution: Covariate Shifts in Environment and Sensor Domains

arXiv: https://arxiv.org/abs/2404.15882 · HTML: https://mlsanity.com/paper/2404.15882/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 47 references that are in the map.
- First posted on arXiv 24 Apr 2024.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [Learning Robust Global Representations by Penalizing Local Predictive Power](https://mlsanity.com/paper/1905.13549/) (2019)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Out-of-Distribution Detection with Deep Nearest Neighbors](https://mlsanity.com/paper/2204.06507/) (2022)
- [ReAct: Out-of-distribution Detection With Rectified Activations](https://mlsanity.com/paper/2111.12797/) (2021)
- [Scaling Out-of-Distribution Detection for Real-World Settings](https://mlsanity.com/paper/1911.11132/) (2019)
- [Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Data](https://mlsanity.com/paper/2002.11297/) (2020)
- [ViM: Out-Of-Distribution with Virtual-logit Matching](https://mlsanity.com/paper/2203.10807/) (2022)
- [Open-Set Recognition: a Good Closed-Set Classifier is All You Need?](https://mlsanity.com/paper/2110.06207/) (2021)

## Built on by (most cited first, 22 of 22)

- [Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey](https://mlsanity.com/paper/2407.21794/) (2024)
- [COME: Test-time adaption by Conservatively Minimizing Entropy](https://mlsanity.com/paper/2410.10894/) (2024)
- [textttBATCLIP: Bimodal Online Test-Time Adaptation for CLIP](https://mlsanity.com/paper/2412.02837/) (2024)
- [AdaSCALE: Adaptive Scaling for OOD Detection](https://mlsanity.com/paper/2503.08023/) (2025)
- [Adaptive Camera Sensor for Vision Models](https://mlsanity.com/paper/2503.02170/) (2025)
- [SenseShift6D: Multimodal RGB-D Benchmarking for Robust 6D Pose Estimation across Environment and Sensor Variations](https://mlsanity.com/paper/2507.05751/) (2025)
- [AI Should Sense Better, Not Just Scale Bigger: Adaptive Sensing as a Paradigm Shift](https://mlsanity.com/paper/2507.07820/) (2025)
- [Processing and acquisition traces in visual encoders: What does CLIP know about your camera?](https://mlsanity.com/paper/2508.10637/) (2025)
- [TULiP: Test-time Uncertainty Estimation via Linearization and Weight Perturbation](https://mlsanity.com/paper/2505.16923/) (2025)
- [Uniformity First: Uniformity-aware Test-time Adaptation of Vision-language Models against Image Corruption](https://mlsanity.com/paper/2505.12912/) (2025)
- [SNAP: A Benchmark for Testing the Effects of Capture Conditions on Fundamental Vision Tasks](https://mlsanity.com/paper/2505.15628/) (2025)
- [Self-Ensemble Post Learning for Noisy Domain Generalization](https://mlsanity.com/paper/2512.10818/) (2025)
- [Measurement Plasticity: Sensor-Level Adaptation for Vision-Language Models](https://mlsanity.com/paper/2512.12571/) (2025)
- [Predictive Spectral Calibration for Source-Free Test-Time Regression](https://mlsanity.com/paper/2603.09338/) (2026)
- [Tracking the Discriminative Axis: Dual Prototypes for Test-Time OOD Detection Under Covariate Shift](https://mlsanity.com/paper/2603.15213/) (2026)
- [[Emerging Ideas] Artificial Tripartite Intelligence: A Bio-Inspired, Sensor-First Architecture for Physical AI](https://mlsanity.com/paper/2604.13959/) (2026)
- [CURE-OOD: Benchmarking Out-of-Distribution Detection for Survival Prediction](https://mlsanity.com/paper/2605.00350/) (2026)
- [MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalization](https://mlsanity.com/paper/2605.01967/) (2026)
- [Learning Hyperspherical Time-Frequency Representations for Time-Series Out-of-Distribution Detection](https://mlsanity.com/paper/2605.31155/) (2026)
- [Sparse Autoencoders for Interpretable Out-of-Distribution Detection](https://mlsanity.com/paper/2607.12094/) (2026)
- [G-MAD: A Game-Based Data Generation Framework for Multi-View RGB-T Aerial Object Detection](https://mlsanity.com/paper/2607.19942/) (2026)
- [What Paired Evaluations Reveal under Visual Perturbations](https://mlsanity.com/paper/2609.35583/) (2026)

## Abstract (arXiv)

Computer vision applications predict on digital images acquired by a camera from physical scenes through light. However, conventional robustness benchmarks rely on perturbations in digitized images, diverging from distribution shifts occurring in the image acquisition process. To bridge this gap, we introduce a new distribution shift dataset, ImageNet-ES, comprising variations in environmental and camera sensor factors by directly capturing 202k images with a real camera in a controllable testbed. With the new dataset, we evaluate out-of-distribution (OOD) detection and model robustness. We find that existing OOD detection methods do not cope with the covariate shifts in ImageNet-ES, implying that the definition and detection of OOD should be revisited to embrace real-world distribution shifts. We also observe that the model becomes more robust in both ImageNet-C and -ES by learning environment and sensor variations in addition to existing digital augmentations. Lastly, our results suggest that effective shift mitigation via camera sensor control can significantly improve performance without increasing model size. With these findings, our benchmark may aid future research on robustness, OOD, and camera sensor control for computer vision. Our code and dataset are available at https://github.com/Edw2n/ImageNet-ES.