# LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detection

arXiv: https://arxiv.org/abs/2601.02511 · HTML: https://mlsanity.com/paper/2601.02511/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 5 Jan 2026.

## Builds on (most cited first, 13 of 13)

- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [TS2Vec: Towards Universal Representation of Time Series](https://mlsanity.com/paper/2106.10466/) (2021)
- [TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data](https://mlsanity.com/paper/2201.07284/) (2022)
- [Deep Learning for Time Series Anomaly Detection: A Survey](https://mlsanity.com/paper/2211.05244/) (2022)
- [DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection](https://mlsanity.com/paper/2306.10347/) (2023)
- [Improving Human Sequential Decision-Making with Reinforcement Learning](https://mlsanity.com/paper/2108.08454/) (2021)
- [Potential-Based Reward Shaping For Intrinsic Motivation](https://mlsanity.com/paper/2402.07411/) (2024)
- [Extracting Heuristics from Large Language Models for Reward Shaping in Reinforcement Learning](https://mlsanity.com/paper/2405.15194/) (2024)
- [Sparse Deep Learning for Time Series Data: Theory and Applications](https://mlsanity.com/paper/2310.03243/) (2023)
- [Anomaly Detection in Time Series Data Using Reinforcement Learning, Variational Autoencoder, and Active Learning](https://mlsanity.com/paper/2504.02999/) (2025)
- [Semi-supervised Anomaly Detection via Adaptive Reinforcement Learning-Enabled Method with Causal Inference for Sensor Signals](https://mlsanity.com/paper/2405.06925/) (2024)
- [Anomalous State Sequence Modeling to Enhance Safety in Reinforcement Learning](https://mlsanity.com/paper/2407.19860/) (2024)

## Built on by (most cited first, 1 of 1)

- [Estimating Pedestrian Volumes from GIS-Derived Built-Environment Features: A Machine Learning Framework](https://mlsanity.com/paper/2609.12173/) (2026)

## Abstract (arXiv)

Detecting anomalies in time series data is crucial for finance, healthcare, sensor networks, and industrial monitoring applications. However, time series anomaly detection often suffers from sparse labels, complex temporal patterns, and costly expert annotation. We propose a unified framework that integrates Large Language Model (LLM)-based potential functions for reward shaping with Reinforcement Learning (RL), Variational Autoencoder (VAE)-enhanced dynamic reward scaling, and active learning with label propagation. An LSTM-based RL agent leverages LLM-derived semantic rewards to guide exploration, while VAE reconstruction errors add unsupervised anomaly signals. Active learning selects the most uncertain samples, and label propagation efficiently expands labeled data. Evaluations on Yahoo-A1 and SMD benchmarks demonstrate that our method achieves state-of-the-art detection accuracy under limited labeling budgets and operates effectively in data-constrained settings. This study highlights the promise of combining LLMs with RL and advanced unsupervised techniques for robust, scalable anomaly detection in real-world applications.