# Meta-AAD: Active Anomaly Detection with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2009.07415 · HTML: https://mlsanity.com/paper/2009.07415/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2020.

## Built on by (most cited first, 30 of 34)

- [ADBench: Anomaly Detection Benchmark](https://mlsanity.com/paper/2206.09426/) (2022)
- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions](https://mlsanity.com/paper/2201.00382/) (2022)
- [Data-centric AI: Perspectives and Challenges](https://mlsanity.com/paper/2301.04819/) (2023)
- [Weakly Supervised Anomaly Detection: A Survey](https://mlsanity.com/paper/2302.04549/) (2023)
- [AutoShard: Automated Embedding Table Sharding for Recommender Systems](https://mlsanity.com/paper/2208.06399/) (2022)
- [DreamShard: Generalizable Embedding Table Placement for Recommender Systems](https://mlsanity.com/paper/2210.02023/) (2022)
- [Weakly Supervised Anomaly Detection via Knowledge-Data Alignment](https://mlsanity.com/paper/2402.03785/) (2024)
- [Towards Automated Imbalanced Learning with Deep Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2208.12433/) (2022)
- [Deep Anomaly Detection under Labeling Budget Constraints](https://mlsanity.com/paper/2302.07832/) (2023)
- [Learning Disentangled Representations for Time Series](https://mlsanity.com/paper/2105.08179/) (2021)
- [Context-aware Domain Adaptation for Time Series Anomaly Detection](https://mlsanity.com/paper/2304.07453/) (2023)
- [UADB: Unsupervised Anomaly Detection Booster](https://mlsanity.com/paper/2306.01997/) (2023)
- [Multitask Active Learning for Graph Anomaly Detection](https://mlsanity.com/paper/2401.13210/) (2024)
- [AutoVideo: An Automated Video Action Recognition System](https://mlsanity.com/paper/2108.04212/) (2021)
- [Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models](https://mlsanity.com/paper/2305.01868/) (2023)
- [OptIForest: Optimal Isolation Forest for Anomaly Detection](https://mlsanity.com/paper/2306.12703/) (2023)
- [Active anomaly detection based on deep one-class classification](https://mlsanity.com/paper/2309.09465/) (2023)
- [GRAM: An Interpretable Approach for Graph Anomaly Detection using Gradient Attention Maps](https://mlsanity.com/paper/2311.06153/) (2023)
- [Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AI](https://mlsanity.com/paper/2508.21101/) (2025)
- [Causality-informed Anomaly Detection in Partially Observable Sensor Networks: Moving beyond Correlations](https://mlsanity.com/paper/2507.09742/) (2025)
- [Outlier Detection using AI: A Survey](https://mlsanity.com/paper/2112.00588/) (2021)
- [Towards Personalized Preprocessing Pipeline Search](https://mlsanity.com/paper/2302.14329/) (2023)
- [An Automated Machine Learning Approach for Detecting Anomalous Peak Patterns in Time Series Data from a Research Watershed in the Northeastern United States Critical Zone](https://mlsanity.com/paper/2309.07992/) (2023)
- [Synergizing Large Language Models and Task-specific Models for Time Series Anomaly Detection](https://mlsanity.com/paper/2501.05675/) (2025)
- [Memorize Early, Then Query: Inlier-Memorization-Guided Active Outlier Detection](https://mlsanity.com/paper/2601.10993/) (2026)
- [Interactive System-wise Anomaly Detection](https://mlsanity.com/paper/2304.10704/) (2023)
- [Tackling Diverse Minorities in Imbalanced Classification](https://mlsanity.com/paper/2308.14838/) (2023)
- [Towards Automated Self-Supervised Learning for Truly Unsupervised Graph Anomaly Detection](https://mlsanity.com/paper/2501.14694/) (2025)
- [Label-Informed Outlier Detection Based on Granule Density](https://mlsanity.com/paper/2512.18774/) (2025)

## Abstract (arXiv)

High false-positive rate is a long-standing challenge for anomaly detection algorithms, especially in high-stake applications. To identify the true anomalies, in practice, analysts or domain experts will be employed to investigate the top instances one by one in a ranked list of anomalies identified by an anomaly detection system. This verification procedure generates informative labels that can be leveraged to re-rank the anomalies so as to help the analyst to discover more true anomalies given a time budget. Some re-ranking strategies have been proposed to approximate the above sequential decision process. Specifically, existing strategies have been focused on making the top instances more likely to be anomalous based on the feedback. Then they greedily select the top-1 instance for query. However, these greedy strategies could be sub-optimal since some low-ranked instances could be more helpful in the long-term. In this work, we propose Active Anomaly Detection with Meta-Policy (Meta-AAD), a novel framework that learns a meta-policy for query selection. Specifically, Meta-AAD leverages deep reinforcement learning to train the meta-policy to select the most proper instance to explicitly optimize the number of discovered anomalies throughout the querying process. Meta-AAD is easy to deploy since a trained meta-policy can be directly applied to any new datasets without further tuning. Extensive experiments on 24 benchmark datasets demonstrate that Meta-AAD significantly outperforms the state-of-the-art re-ranking strategies and the unsupervised baseline. The empirical analysis shows that the trained meta-policy is transferable and inherently achieves a balance between long-term and short-term rewards.