# A Critical Evaluation of Open-World Machine Learning

arXiv: https://arxiv.org/abs/2007.04391 · HTML: https://mlsanity.com/paper/2007.04391/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in World model security and GUI agents, one of 11 directions in World models.
- Builds on 5 of its 16 references that are in the map.
- First posted on arXiv 8 Jul 2020.

## Placed in

- World models / [World model security and GUI agents](https://mlsanity.com/wm/world-model-security-and-gui-agents/)

## Builds on (most cited first, 5 of 5)

- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks](https://mlsanity.com/paper/1807.03888/) (2018)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)

## Built on by (most cited first, 6 of 6)

- [Open-world Machine Learning: Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2105.13448/) (2021)
- [L2Explorer: A Lifelong Reinforcement Learning Assessment Environment](https://mlsanity.com/paper/2203.07454/) (2022)
- [Margin-Aware Intra-Class Novelty Identification for Medical Images](https://mlsanity.com/paper/2108.00117/) (2021)
- [Deciphering the Definition of Adversarial Robustness for post-hoc OOD Detectors](https://mlsanity.com/paper/2406.15104/) (2024)
- [A Comprehensive Assessment Benchmark for Rigorously Evaluating Deep Learning Image Classifiers](https://mlsanity.com/paper/2308.04137/) (2023)
- [Enhancing Open-World Bacterial Raman Spectra Identification by Feature Regularization for Improved Resilience against Unknown Classes](https://mlsanity.com/paper/2310.13723/) (2023)

## Abstract (arXiv)

Open-world machine learning (ML) combines closed-world models trained on in-distribution data with out-of-distribution (OOD) detectors, which aim to detect and reject OOD inputs. Previous works on open-world ML systems usually fail to test their reliability under diverse, and possibly adversarial conditions. Therefore, in this paper, we seek to understand how resilient are state-of-the-art open-world ML systems to changes in system components? With our evaluation across 6 OOD detectors, we find that the choice of in-distribution data, model architecture and OOD data have a strong impact on OOD detection performance, inducing false positive rates in excess of 70%. We further show that OOD inputs with 22 unintentional corruptions or adversarial perturbations render open-world ML systems unusable with false positive rates of up to 100%. To increase the resilience of open-world ML, we combine robust classifiers with OOD detection techniques and uncover a new trade-off between OOD detection and robustness.