# Policy Entropy for Out-of-Distribution Classification

arXiv: https://arxiv.org/abs/2005.12069 · HTML: https://mlsanity.com/paper/2005.12069/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2020.

## Built on by (most cited first, 11 of 11)

- [Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles](https://mlsanity.com/paper/2207.12380/) (2022)
- [Guaranteeing Out-Of-Distribution Detection in Deep RL via Transition Estimation](https://mlsanity.com/paper/2503.05238/) (2025)
- [A Distance-based Anomaly Detection Framework for Deep Reinforcement Learning](https://mlsanity.com/paper/2109.09889/) (2021)
- [Task-Driven Detection of Distribution Shifts with Statistical Guarantees for Robot Learning](https://mlsanity.com/paper/2106.13703/) (2021)
- [Quantifying Multimodality in World Models](https://mlsanity.com/paper/2112.07263/) (2021)
- [OIL-AD: An Anomaly Detection Framework for Sequential Decision Sequences](https://mlsanity.com/paper/2402.04567/) (2024)
- [Energy-Based Transfer for Reinforcement Learning](https://mlsanity.com/paper/2506.16590/) (2025)
- [Mahalanobis-Guided Latent OOD Detection for Hybrid ES-DRL Control in Time-Varying Systems](https://mlsanity.com/paper/2606.11474/) (2026)
- [Multi-view Deep One-class Classification: A Systematic Exploration](https://mlsanity.com/paper/2104.13000/) (2021)
- [Distributed Detection of Adversarial Attacks in Multi-Agent Reinforcement Learning with Continuous Action Space](https://mlsanity.com/paper/2508.15764/) (2025)
- [Selective Agent Guidance via Entropy: Learning Autonomous Policies from Imperfect VLM Teachers](https://mlsanity.com/paper/2609.01567/) (2026)

## Abstract (arXiv)

One critical prerequisite for the deployment of reinforcement learning systems in the real world is the ability to reliably detect situations on which the agent was not trained. Such situations could lead to potential safety risks when wrong predictions lead to the execution of harmful actions. In this work, we propose PEOC, a new policy entropy based out-of-distribution classifier that reliably detects unencountered states in deep reinforcement learning. It is based on using the entropy of an agent's policy as the classification score of a one-class classifier. We evaluate our approach using a procedural environment generator. Results show that PEOC is highly competitive against state-of-the-art one-class classification algorithms on the evaluated environments. Furthermore, we present a structured process for benchmarking out-of-distribution classification in reinforcement learning.