mlsanityData to 14 Oct 2026

DRTA: Dynamic Reward Scaling for Reinforcement Learning in Time Series Anomaly Detection

arXiv 2508.18474 ↗First posted 25 Aug 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Anomaly detection in time series data is important for applications in finance, healthcare, sensor networks, and industrial monitoring. Traditional methods usually struggle with limited labeled data, high false-positive rates, and difficulty generalizing to novel anomaly types. To overcome these challenges, we propose a reinforcement learning-based framework that integrates dynamic reward shaping, Variational Autoencoder (VAE), and active learning, called DRTA. Our method uses an adaptive reward mechanism that balances exploration and exploitation by dynamically scaling the effect of VAE-based reconstruction error and classification rewards. This approach enables the agent to detect anomalies effectively in low-label systems while maintaining high precision and recall. Our experimental results on the Yahoo A1 and Yahoo A2 benchmark datasets demonstrate that the proposed method consistently outperforms state-of-the-art unsupervised and semi-supervised approaches. These findings show that our framework is a scalable and efficient solution for real-world anomaly detection tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 10 of its 37 references that are in the map. First posted on arXiv 25 Aug 2025.

Builds on

10 of 37 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning3 Apr
  2. 2024Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learning6 Aug

Other references · 8

  1. 2017Deep Reinforcement Learning: An Overviewcited by 333
  2. 2016LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detectioncited by 215
  3. 2019Time-Series Anomaly Detection Service at Microsoftcited by 165
  4. 2019Deep Anomaly Detection with Deviation Networkscited by 153
  5. 2024Adaptive Preference Scaling for Reinforcement Learning with Human Feedbackcited by 20
  6. 2021RLAD: Time Series Anomaly Detection through Reinforcement Learning and Active Learningcited by 13
  7. 2018Object-sensitive Deep Reinforcement Learningcited by 10
  8. 2025Emotion Detection in Twitter Messages Using Combination of Long Short-Term Memory and Convolutional Deep Neural Networkscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Dynamic Reward Scaling for Multivariate Time Series Anomaly Detection: A VAE-Enhanced Reinforcement Learning Approach
  2. 2026LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detection
  3. 2025Calibrated Unsupervised Anomaly Detection in Multivariate Time-series using Reinforcement Learning
  4. 2020Sequential Anomaly Detection using Inverse Reinforcement Learning
  5. 2024An Unsupervised Anomaly Detection in Electricity Consumption Using Reinforcement Learning and Time Series Forest Based Framework
  6. 2025Anomalous Decision Discovery using Inverse Reinforcement Learning
  7. 2024OIL-AD: An Anomaly Detection Framework for Sequential Decision Sequences
  8. 2024Angel or Devil: Discriminating Hard Samples and Anomaly Contaminations for Unsupervised Time Series Anomaly Detection