mlsanityData to 14 Oct 2026

Dynamic Reward Scaling for Multivariate Time Series Anomaly Detection: A VAE-Enhanced Reinforcement Learning Approach

arXiv 2511.12351 ↗First posted 15 Nov 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Detecting anomalies in multivariate time series is essential for monitoring complex industrial systems, where high dimensionality, limited labeled data, and subtle dependencies between sensors cause significant challenges. This paper presents a deep reinforcement learning framework that combines a Variational Autoencoder (VAE), an LSTM-based Deep Q-Network (DQN), dynamic reward shaping, and an active learning module to address these issues in a unified learning framework. The main contribution is the implementation of Dynamic Reward Scaling for Multivariate Time Series Anomaly Detection (DRSMT), which demonstrates how each component enhances the detection process. The VAE captures compact latent representations and reduces noise. The DQN enables adaptive, sequential anomaly classification, and the dynamic reward shaping balances exploration and exploitation during training by adjusting the importance of reconstruction and classification signals. In addition, active learning identifies the most uncertain samples for labeling, reducing the need for extensive manual supervision. Experiments on two multivariate benchmarks, namely Server Machine Dataset (SMD) and Water Distribution Testbed (WADI), show that the proposed method outperforms existing baselines in F1-score and AU-PR. These results highlight the effectiveness of combining generative modeling, reinforcement learning, and selective supervision for accurate and scalable anomaly detection in real-world multivariate systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 37 references that are in the map. First posted on arXiv 15 Nov 2025.

Builds on

24 of 37 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning3 Apr
  2. 2025Calibrated Unsupervised Anomaly Detection in Multivariate Time-series using Reinforcement Learning5 Feb

Other references · 22

  1. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  2. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 471
  3. 2018Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholdingcited by 410
  4. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 394
  5. 2022TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Datacited by 214
  6. 2017A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencodercited by 209
  7. 2020Multivariate Time-series Anomaly Detection via Graph Attention Networkcited by 167
  8. 2020Deep Learning for Anomaly Detection: A Reviewcited by 159
  9. 2022Deep Learning for Time Series Anomaly Detection: A Surveycited by 131
  10. 2023DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detectioncited by 103
  11. 2023AnomalyBERT: Self-Supervised Transformer for Time Series Anomaly Detection using Data Degradation Schemecited by 25
  12. 2023Unsupervised anomaly detection algorithms on real-world data: how many do we need?cited by 20
10 more
  1. 2024Adaptive Preference Scaling for Reinforcement Learning with Human Feedbackcited by 20
  2. 2025Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignmentcited by 20
  3. 2023CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detectioncited by 15
  4. 2021RLAD: Time Series Anomaly Detection through Reinforcement Learning and Active Learningcited by 13
  5. 2021Online false discovery rate control for anomaly detection in time seriescited by 13
  6. 2024Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Valuescited by 10
  7. 2019Online Multivariate Anomaly Detection and Localization for High-dimensional Settingscited by 6
  8. 2024Variational Autoencoder for Anomaly Detection: A Comparative Studycited by 6
  9. 2025Anomaly Detection for Industrial Applications, Its Challenges, Solutions, and Future Directions: A Reviewcited by 6
  10. 2025Cyber-Physical Systems Security: A Comprehensive Review of Anomaly Detection Techniquescited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025DRTA: Dynamic Reward Scaling for Reinforcement Learning in Time Series Anomaly Detection
  2. 2026LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detection
  3. 2024Towards efficient deep autoencoders for multivariate time series anomaly detection
  4. 2024An Unsupervised Anomaly Detection in Electricity Consumption Using Reinforcement Learning and Time Series Forest Based Framework
  5. 2024Angel or Devil: Discriminating Hard Samples and Anomaly Contaminations for Unsupervised Time Series Anomaly Detection
  6. 2024Semi-supervised learning via DQN for log anomaly detection
  7. 2026CARE: A Cascaded Framework for Efficient and Reliable Time Series Anomaly Detection
  8. 2026Have an LLM Write Your Anomaly Detector: Autonomous Discovery of Compact, Interpretable Detectors for Time Series