mlsanityData to 7 Oct 2026

AD-LLM: Benchmarking Large Language Models for Anomaly Detection

arXiv 2412.11142 ↗First posted 15 Dec 2024cs.CL cs.AIcited by 25

Not on the map. Why?

Abstract (arXiv)

Anomaly detection (AD) is an important machine learning task with many real-world uses, including fraud detection, medical diagnosis, and industrial monitoring. Within natural language processing (NLP), AD helps detect issues like spam, misinformation, and unusual user activity. Although large language models (LLMs) have had a strong impact on tasks such as text generation and summarization, their potential in AD has not been studied enough. This paper introduces AD-LLM, the first benchmark that evaluates how LLMs can help with NLP anomaly detection. We examine three key tasks: (i) zero-shot detection, using LLMs' pre-trained knowledge to perform AD without tasks-specific training; (ii) data augmentation, generating synthetic data and category descriptions to improve AD models; and (iii) model selection, using LLMs to suggest unsupervised AD models. Through experiments with different datasets, we find that LLMs can work well in zero-shot AD, that carefully designed augmentation methods are useful, and that explaining model selection for specific datasets remains challenging. Based on these results, we outline six future research directions on LLMs for AD.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 35 of its 64 references that are in the map. First posted on arXiv 15 Dec 2024.

Builds on

35 of 64 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  4. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  5. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  6. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
  7. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 240
  8. 2019PyOD: A Python Toolbox for Scalable Outlier Detectioncited by 221
  9. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 157
  10. 2022ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functionscited by 153
  11. 2020Deep Learning for Medical Anomaly Detection -- A Surveycited by 127
  12. 2023Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventionscited by 121
23 more
  1. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 97
  2. 2024Real-Time Anomaly Detection and Reactive Planning with Large Language Modelscited by 67
  3. 2018Generative Adversarial Active Learning for Unsupervised Outlier Detectioncited by 65
  4. 2023Towards Generic Anomaly Detection and Understanding: Large-scale Visual-linguistic Model (GPT-4V) Takes the Leadcited by 40
  5. 2024Insights into LLM Long-Context Failures: When Transformers Know but Don't Tellcited by 36
  6. 2024Do LLMs Understand Visual Anomalies? Uncovering LLM's Capabilities in Zero-shot Anomaly Detectioncited by 35
  7. 2023Interpretable Online Log Analysis Using Large Language Models with Prompt Strategiescited by 34
  8. 2022PyGOD: A Python Library for Graph Outlier Detectioncited by 31
  9. 2022On the Importance of Building High-quality Training Datasets for Neural Code Searchcited by 30
  10. 2024Political-LLM: Large Language Models in Political Sciencecited by 27
  11. 2021DATE: Detecting Anomalies in Text via Self-Supervision of Transformerscited by 25
  12. 2024Large Language Models for Anomaly and Out-of-Distribution Detection: A Surveycited by 24
  13. 2024RDRec: Rationale Distillation for LLM-based Recommendationcited by 21
  14. 2024NLP-ADBench: NLP Anomaly Detection Benchmarkcited by 17
  15. 2024PyOD 2: A Python Library for Outlier Detection with LLM-powered Model Selectioncited by 17
  16. 2024MetaOOD: Automatic Selection of OOD Detection Modelscited by 16
  17. 2024Data-Efficient Learning via Clustering-Based Sensitivity Sampling: Foundation Models and Beyondcited by 15
  18. 2024AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deploymentcited by 12
  19. 2022Data Augmentation is a Hyperparameter: Cherry-picked Self-Supervision for Unsupervised Anomaly Detection is Creating the Illusion of Successcited by 11
  20. 2023ADGym: Design Choices for Deep Anomaly Detectioncited by 11
  21. 2020Diverse Data Selection under Fairness Constraintscited by 7
  22. 2023DSV: An Alignment Validation Loss for Self-supervised Outlier Model Selectioncited by 6
  23. 2024Language Models Meet Anomaly Detection for Better Interpretability and Generalizabilitycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 15; 2026: 10.

25 papers.

Sort:

By area:

  1. 2025Adaptive Fault Tolerance Mechanisms of Large Language Models in Cloud Computing Environmentscited by 12
  2. 2025Can Multimodal LLMs Perform Time Series Anomaly Detection?cited by 11
  3. 2026FORTIS: Benchmarking Over-Privilege in Agent Skillscited by 11
  4. 2025StealthRank: LLM Ranking Manipulation via Stealthy Prompt Optimizationcited by 10
  5. 2025Scalability Optimization in Cloud-Based AI Inference Services: Strategies for Real-Time Load Balancing and Automated Scalingcited by 10
  6. 2025ClimateLLM: Efficient Weather Forecasting via Frequency-Aware Large Language Modelscited by 9
  7. 2026The Autonomy Tax: Defense Training Breaks LLM Agentscited by 8
  8. 2025TAD-Bench: A Comprehensive Benchmark for Embedding-Based Text Anomaly Detectioncited by 6
  9. 2025Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Basecited by 6
  10. 2025MUSE: MCTS-Driven Red Teaming Framework for Enhanced Multi-Turn Dialogue Safety in Large Language Modelscited by 4
  11. 2025Multimodal RAG-driven Anomaly Detection and Classification in Laser Powder Bed Fusion using Large Language Modelscited by 3
  12. 2026CoAct: Co-Active LLM Preference Learning with Human-AI Synergycited by 3
13 more
  1. 2025Detecting Voice Phishing with Precision: Fine-Tuning Small Language Modelscited by 2
  2. 2026Towards Robust LLM Post-Training: Automatic Failure Management for Reinforcement Fine-Tuningcited by 2
  3. 2025Text-ADBench: Text Anomaly Detection Benchmark Based on LLM Embeddingscited by 1
  4. 2026Cat-DPO: Category-Adaptive Safety Alignmentcited by 1
  5. 2025Bridging Unsupervised and Semi-Supervised Anomaly Detection: A Theoretically-Grounded and Practical Framework with Synthetic Anomaliescited by 0
  6. 2025TAGFN: A Text-Attributed Graph Dataset for Fake News Detection in the Age of LLMscited by 0
  7. 2025AuditCopilot: Leveraging LLMs for Fraud Detection in Double-Entry Bookkeepingcited by 0
  8. 2025Spatio-Temporal Graphs Beyond Grids: Benchmark for Maritime Anomaly Detectioncited by 0
  9. 2026ChatAD: Reasoning-Enhanced Time-Series Anomaly Detection with Multi-Turn Instruction Evolutioncited by 0
  10. 2026Detection of adversarial intent in Human-AI teams using LLMscited by 0
  11. 2026Agent Safety Is Action Alignmentcited by 0
  12. 2026GTA: Graph Theory Agent and Benchmark for Algorithmic Graph Reasoning with LLMscited by 0
  13. 2026LLM4Trust: Exploring the Capabilities of Large Language Models for Trust Evaluationcited by 0