# Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection

arXiv: https://arxiv.org/abs/2401.15123 · HTML: https://mlsanity.com/paper/2401.15123/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 36 of its 47 references that are in the map.
- First posted on arXiv 26 Jan 2024.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 30 of 36)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [A Unifying Review of Deep and Shallow Anomaly Detection](https://mlsanity.com/paper/2009.11732/) (2020)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [Time-Series Representation Learning via Temporal and Contextual Contrasting](https://mlsanity.com/paper/2106.14112/) (2021)
- [Graph Neural Network-Based Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2106.06947/) (2021)
- [Multiresolution Knowledge Distillation for Anomaly Detection](https://mlsanity.com/paper/2011.11108/) (2020)
- [Monash Time Series Forecasting Archive](https://mlsanity.com/paper/2105.06643/) (2021)
- [TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data](https://mlsanity.com/paper/2201.07284/) (2022)
- [A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder](https://mlsanity.com/paper/1711.00614/) (2017)
- [TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting](https://mlsanity.com/paper/2310.04948/) (2023)
- [TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series](https://mlsanity.com/paper/2308.08241/) (2023)
- [TimeGPT-1](https://mlsanity.com/paper/2310.03589/) (2023)
- [Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress](https://mlsanity.com/paper/2009.13807/) (2020)
- [An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networks](https://mlsanity.com/paper/2007.15951/) (2020)
- [Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook](https://mlsanity.com/paper/2310.10196/) (2023)
- [DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection](https://mlsanity.com/paper/2306.10347/) (2023)
- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [Towards a Rigorous Evaluation of Time-series Anomaly Detection](https://mlsanity.com/paper/2109.05257/) (2021)
- [Local Evaluation of Time Series Anomaly Detection Algorithms](https://mlsanity.com/paper/2206.13167/) (2022)
- [Rethinking Assumptions in Deep Anomaly Detection](https://mlsanity.com/paper/2006.00339/) (2020)
- [Neural Contextual Anomaly Detection for Time Series](https://mlsanity.com/paper/2107.07702/) (2021)
- [Graph-Augmented Normalizing Flows for Anomaly Detection of Multiple Time Series](https://mlsanity.com/paper/2202.07857/) (2022)
- [ImDiffusion: Imputed Diffusion Models for Multivariate Time Series Anomaly Detection](https://mlsanity.com/paper/2307.00754/) (2023)
- [MEMTO: Memory-guided Transformer for Multivariate Time Series Anomaly Detection](https://mlsanity.com/paper/2312.02530/) (2023)

## Built on by (most cited first, 26 of 26)

- [Can LLMs Understand Time Series Anomalies?](https://mlsanity.com/paper/2410.05440/) (2024)
- [Empowering Time Series Analysis with Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2405.02358/) (2024)
- [Can LLMs Serve As Time Series Anomaly Detectors?](https://mlsanity.com/paper/2408.03475/) (2024)
- [Benchmarking Visual Language Models on Standardized Visualization Literacy Tests](https://mlsanity.com/paper/2503.16632/) (2025)
- [Efficient Multivariate Time Series Forecasting via Calibrated Language Models with Privileged Knowledge Distillation](https://mlsanity.com/paper/2505.02138/) (2025)
- [Deep Learning Advancements in Anomaly Detection: A Comprehensive Survey](https://mlsanity.com/paper/2503.13195/) (2025)
- [Can Multimodal LLMs Perform Time Series Anomaly Detection?](https://mlsanity.com/paper/2502.17812/) (2025)
- [Towards Cross-Modality Modeling for Time Series Analytics: A Survey in the LLM Era](https://mlsanity.com/paper/2505.02583/) (2025)
- [DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Series](https://mlsanity.com/paper/2404.11269/) (2024)
- [AnomaMind: Agentic Time Series Anomaly Detection with Tool-Augmented Reasoning](https://mlsanity.com/paper/2602.13807/) (2026)
- [CALM: A Framework for Continuous, Adaptive, and LLM-Mediated Anomaly Detection in Time-Series Streams](https://mlsanity.com/paper/2508.21273/) (2025)
- [Foundation Models for Anomaly Detection: Vision and Challenges](https://mlsanity.com/paper/2502.06911/) (2025)
- [MADLLM: Multivariate Anomaly Detection via Pre-trained LLMs](https://mlsanity.com/paper/2504.09504/) (2025)
- [Large Language models for Time Series Analysis: Techniques, Applications, and Challenges](https://mlsanity.com/paper/2506.11040/) (2025)
- [LLM-Assisted Logic Rule Learning: Scaling Human Expertise for Time Series Anomaly Detection](https://mlsanity.com/paper/2601.19255/) (2026)
- [Synergizing Large Language Models and Task-specific Models for Time Series Anomaly Detection](https://mlsanity.com/paper/2501.05675/) (2025)
- [KKA: Improving Vision Anomaly Detection through Anomaly-related Knowledge from Large Language Models](https://mlsanity.com/paper/2502.14880/) (2025)
- [Knowledge Graph-Guided Multi-Agent Distillation for Reliable Industrial Question Answering with Datasets](https://mlsanity.com/paper/2510.06240/) (2025)
- [ASTER: Latent Pseudo-Anomaly Generation for Unsupervised Time-Series Anomaly Detection](https://mlsanity.com/paper/2604.13924/) (2026)
- [Deep Multi-Manifold Transformation Based Multivariate Time Series Fault Detection](https://mlsanity.com/paper/2405.16258/) (2024)
- [Mini-ResEmoteNet: Leveraging Knowledge Distillation for Human-Centered Design](https://mlsanity.com/paper/2501.18538/) (2025)
- [From Time Series Analysis to Question Answering: A Survey in the LLM Era](https://mlsanity.com/paper/2506.11512/) (2025)
- [SPADE: A Large Language Model Framework for Soil Moisture Pattern Recognition and Anomaly Detection in Precision Agriculture](https://mlsanity.com/paper/2509.18123/) (2025)
- [Powerful Teachers Matter: Text-Guided Multi-view Knowledge Distillation with Visual Prior Enhancement](https://mlsanity.com/paper/2603.24208/) (2026)
- [VAN-AD: Visual Masked Autoencoder with Normalizing Flow For Time Series Anomaly Detection](https://mlsanity.com/paper/2603.26842/) (2026)
- [PRISM: Powerful Time Series to Image (TS2I) Representations for Multivariate Anomaly Detection](https://mlsanity.com/paper/2608.03926/) (2026)

## Abstract (arXiv)

Self-supervised methods have gained prominence in time series anomaly detection due to the scarcity of available annotations. Nevertheless, they typically demand extensive training data to acquire a generalizable representation map, which conflicts with scenarios of a few available samples, thereby limiting their performance. To overcome the limitation, we propose AnomalyLLM, a knowledge distillation-based time series anomaly detection approach where the student network is trained to mimic the features of the large language model (LLM)-based teacher network that is pretrained on large-scale datasets. During the testing phase, anomalies are detected when the discrepancy between the features of the teacher and student networks is large. To circumvent the student network from learning the teacher network's feature of anomalous samples, we devise two key strategies. 1) Prototypical signals are incorporated into the student network to consolidate the normal feature extraction. 2) We use synthetic anomalies to enlarge the representation gap between the two networks. AnomalyLLM demonstrates state-of-the-art performance on 15 datasets, improving accuracy by at least 14.5% in the UCR dataset.