# DRTA: Dynamic Reward Scaling for Reinforcement Learning in Time Series Anomaly Detection

arXiv: https://arxiv.org/abs/2508.18474 · HTML: https://mlsanity.com/paper/2508.18474/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 37 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 10 of 10)

- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection](https://mlsanity.com/paper/1607.00148/) (2016)
- [Time-Series Anomaly Detection Service at Microsoft](https://mlsanity.com/paper/1906.03821/) (2019)
- [Deep Anomaly Detection with Deviation Networks](https://mlsanity.com/paper/1911.08623/) (2019)
- [Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learning](https://mlsanity.com/paper/2408.03029/) (2024)
- [Adaptive Preference Scaling for Reinforcement Learning with Human Feedback](https://mlsanity.com/paper/2406.02764/) (2024)
- [RLAD: Time Series Anomaly Detection through Reinforcement Learning and Active Learning](https://mlsanity.com/paper/2104.00543/) (2021)
- [Object-sensitive Deep Reinforcement Learning](https://mlsanity.com/paper/1809.06064/) (2018)
- [Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning](https://mlsanity.com/paper/2504.02999/) (2025)
- [Emotion Detection in Twitter Messages Using Combination of Long Short-Term Memory and Convolutional Deep Neural Networks](https://mlsanity.com/paper/2503.20163/) (2025)

## Abstract (arXiv)

Anomaly detection in time series data is important for applications in finance, healthcare, sensor networks, and industrial monitoring. Traditional methods usually struggle with limited labeled data, high false-positive rates, and difficulty generalizing to novel anomaly types. To overcome these challenges, we propose a reinforcement learning-based framework that integrates dynamic reward shaping, Variational Autoencoder (VAE), and active learning, called DRTA. Our method uses an adaptive reward mechanism that balances exploration and exploitation by dynamically scaling the effect of VAE-based reconstruction error and classification rewards. This approach enables the agent to detect anomalies effectively in low-label systems while maintaining high precision and recall. Our experimental results on the Yahoo A1 and Yahoo A2 benchmark datasets demonstrate that the proposed method consistently outperforms state-of-the-art unsupervised and semi-supervised approaches. These findings show that our framework is a scalable and efficient solution for real-world anomaly detection tasks.