mlsanityData to 14 Oct 2026

Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection

arXiv 2401.15123 ↗First posted 26 Jan 2024cs.LG cs.AIcited by 26

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

Self-supervised methods have gained prominence in time series anomaly detection due to the scarcity of available annotations. Nevertheless, they typically demand extensive training data to acquire a generalizable representation map, which conflicts with scenarios of a few available samples, thereby limiting their performance. To overcome the limitation, we propose AnomalyLLM, a knowledge distillation-based time series anomaly detection approach where the student network is trained to mimic the features of the large language model (LLM)-based teacher network that is pretrained on large-scale datasets. During the testing phase, anomalies are detected when the discrepancy between the features of the teacher and student networks is large. To circumvent the student network from learning the teacher network's feature of anomalous samples, we devise two key strategies. 1) Prototypical signals are incorporated into the student network to consolidate the normal feature extraction. 2) We use synthetic anomalies to enlarge the representation gap between the two networks. AnomalyLLM demonstrates state-of-the-art performance on 15 datasets, improving accuracy by at least 14.5% in the UCR dataset.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 36 of its 47 references that are in the map. First posted on arXiv 26 Jan 2024.

Builds on

36 of 47 references.

References · 36

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  3. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  4. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  5. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  6. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  7. 2020A Unifying Review of Deep and Shallow Anomaly Detectioncited by 425
  8. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 394
  9. 2019Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddingscited by 386
  10. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 345
  11. 2021Graph Neural Network-Based Anomaly Detection in Multivariate Time Seriescited by 301
  12. 2020Multiresolution Knowledge Distillation for Anomaly Detectioncited by 285
24 more
  1. 2021Monash Time Series Forecasting Archivecited by 267
  2. 2022TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Datacited by 214
  3. 2017A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencodercited by 209
  4. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 174
  5. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 169
  6. 2023TimeGPT-1cited by 158
  7. 2020Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progresscited by 151
  8. 2020An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networkscited by 128
  9. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 110
  10. 2023DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detectioncited by 103
  11. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 99
  12. 2021Towards a Rigorous Evaluation of Time-series Anomaly Detectioncited by 97
  13. 2022Local Evaluation of Time Series Anomaly Detection Algorithmscited by 79
  14. 2020Rethinking Assumptions in Deep Anomaly Detectioncited by 45
  15. 2021Neural Contextual Anomaly Detection for Time Seriescited by 43
  16. 2022Graph-Augmented Normalizing Flows for Anomaly Detection of Multiple Time Seriescited by 42
  17. 2023ImDiffusion: Imputed Diffusion Models for Multivariate Time Series Anomaly Detectioncited by 40
  18. 2023MEMTO: Memory-guided Transformer for Multivariate Time Series Anomaly Detectioncited by 26
  19. 2022Unsupervised Model Selection for Time-series Anomaly Detectioncited by 25
  20. 2023AnomalyBERT: Self-Supervised Transformer for Time Series Anomaly Detection using Data Degradation Schemecited by 25
  21. 2022Detecting Multivariate Time Series Anomalies with Zero Known Labelcited by 21
  22. 2023Label-Free Multivariate Time Series Anomaly Detectioncited by 14
  23. 2023Unraveling the "Anomaly" in Time Series Anomaly Detection: A Self-supervised Tri-domain Solutioncited by 12
  24. 2022Deep Contrastive One-Class Time Series Anomaly Detectioncited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 15; 2026: 6.

26 papers.

Sort:

By area:

  1. 2024Can LLMs Understand Time Series Anomalies?cited by 41
  2. 2024Empowering Time Series Analysis with Foundation Models: A Comprehensive Surveycited by 23
  3. 2024Can LLMs Serve As Time Series Anomaly Detectors?cited by 22
  4. 2025Benchmarking Visual Language Models on Standardized Visualization Literacy Testscited by 20
  5. 2025Efficient Multivariate Time Series Forecasting via Calibrated Language Models with Privileged Knowledge Distillationcited by 20
  6. 2025Deep Learning Advancements in Anomaly Detection: A Comprehensive Surveycited by 17
  7. 2025Can Multimodal LLMs Perform Time Series Anomaly Detection?cited by 12
  8. 2025Towards Cross-Modality Modeling for Time Series Analytics: A Survey in the LLM Eracited by 12
  9. 2024DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Seriescited by 7
  10. 2026AnomaMind: Agentic Time Series Anomaly Detection with Tool-Augmented Reasoningcited by 5
  11. 2025CALM: A Framework for Continuous, Adaptive, and LLM-Mediated Anomaly Detection in Time-Series Streamscited by 4
  12. 2025Foundation Models for Anomaly Detection: Vision and Challengescited by 3
14 more
  1. 2025MADLLM: Multivariate Anomaly Detection via Pre-trained LLMscited by 3
  2. 2025Large Language models for Time Series Analysis: Techniques, Applications, and Challengescited by 3
  3. 2026LLM-Assisted Logic Rule Learning: Scaling Human Expertise for Time Series Anomaly Detectioncited by 2
  4. 2025Synergizing Large Language Models and Task-specific Models for Time Series Anomaly Detectioncited by 1
  5. 2025KKA: Improving Vision Anomaly Detection through Anomaly-related Knowledge from Large Language Modelscited by 1
  6. 2025Knowledge Graph-Guided Multi-Agent Distillation for Reliable Industrial Question Answering with Datasetscited by 1
  7. 2026ASTER: Latent Pseudo-Anomaly Generation for Unsupervised Time-Series Anomaly Detectioncited by 1
  8. 2024Deep Multi-Manifold Transformation Based Multivariate Time Series Fault Detectioncited by 0
  9. 2025Mini-ResEmoteNet: Leveraging Knowledge Distillation for Human-Centered Designcited by 0
  10. 2025From Time Series Analysis to Question Answering: A Survey in the LLM Eracited by 0
  11. 2025SPADE: A Large Language Model Framework for Soil Moisture Pattern Recognition and Anomaly Detection in Precision Agriculturecited by 0
  12. 2026Powerful Teachers Matter: Text-Guided Multi-view Knowledge Distillation with Visual Prior Enhancementcited by 0
  13. 2026VAN-AD: Visual Masked Autoencoder with Normalizing Flow For Time Series Anomaly Detectioncited by 0
  14. 2026PRISM: Powerful Time Series to Image (TS2I) Representations for Multivariate Anomaly Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Have an LLM Write Your Anomaly Detector: Autonomous Discovery of Compact, Interpretable Detectors for Time Series
  2. 2025AXIS: Explainable Time Series Anomaly Detection with Large Language Models
  3. 2025RATFM: Retrieval-augmented Time Series Foundation Model for Anomaly Detection
  4. 2025Refining Time Series Anomaly Detectors using Large Language Models
  5. 2024Large Language Models can Deliver Accurate and Interpretable Time Series Anomaly Detection
  6. 2026LLM-Generated Feature Pools for Time Series Anomaly Detection
  7. 2026Detecting Time Series Anomalies Like an Expert: A Multi-Agent LLM Framework with Specialized Analyzers
  8. 2026Tiny but Trusted: Efficient Vision-Language Reasoning for Time-Series Anomaly Detection