# Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning

arXiv: https://arxiv.org/abs/2504.02999 · HTML: https://mlsanity.com/paper/2504.02999/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 6 of its 30 references that are in the map.
- First posted on arXiv 3 Apr 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 6 of 6)

- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection](https://mlsanity.com/paper/1607.00148/) (2016)
- [Time-Series Anomaly Detection Service at Microsoft](https://mlsanity.com/paper/1906.03821/) (2019)
- [RLAD: Time Series Anomaly Detection through Reinforcement Learning and Active Learning](https://mlsanity.com/paper/2104.00543/) (2021)
- [Object-sensitive Deep Reinforcement Learning](https://mlsanity.com/paper/1809.06064/) (2018)
- [Emotion Detection in Twitter Messages Using Combination of Long Short-Term Memory and Convolutional Deep Neural Networks](https://mlsanity.com/paper/2503.20163/) (2025)

## Built on by (most cited first, 3 of 3)

- [LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detection](https://mlsanity.com/paper/2601.02511/) (2026)
- [DRTA: Dynamic Reward Scaling for Reinforcement Learning in Time Series Anomaly Detection](https://mlsanity.com/paper/2508.18474/) (2025)
- [Dynamic Reward Scaling for Multivariate Time Series Anomaly Detection: A VAE-Enhanced Reinforcement Learning Approach](https://mlsanity.com/paper/2511.12351/) (2025)

## Abstract (arXiv)

A novel approach to detecting anomalies in time series data is presented in this paper. This approach is pivotal in domains such as data centers, sensor networks, and finance. Traditional methods often struggle with manual parameter tuning and cannot adapt to new anomaly types. Our method overcomes these limitations by integrating Deep Reinforcement Learning (DRL) with a Variational Autoencoder (VAE) and Active Learning. By incorporating a Long Short-Term Memory (LSTM) network, our approach models sequential data and its dependencies effectively, allowing for the detection of new anomaly classes with minimal labeled data. Our innovative DRL- VAE and Active Learning combination significantly improves existing methods, as shown by our evaluations on real-world datasets, enhancing anomaly detection techniques and advancing time series analysis.