# NLP-ADBench: NLP Anomaly Detection Benchmark

arXiv: https://arxiv.org/abs/2412.04784 · HTML: https://mlsanity.com/paper/2412.04784/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 35 references that are in the map.
- First posted on arXiv 6 Dec 2024.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Unifying Review of Deep and Shallow Anomaly Detection](https://mlsanity.com/paper/2009.11732/) (2020)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [Understanding disentangling in β-VAE](https://mlsanity.com/paper/1804.03599/) (2018)
- [PyOD: A Python Toolbox for Scalable Outlier Detection](https://mlsanity.com/paper/1901.01588/) (2019)
- [ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions](https://mlsanity.com/paper/2201.00382/) (2022)
- [BOND: Benchmarking Unsupervised Outlier Node Detection on Static Attributed Graphs](https://mlsanity.com/paper/2206.10071/) (2022)
- [Generative Adversarial Active Learning for Unsupervised Outlier Detection](https://mlsanity.com/paper/1809.10816/) (2018)
- [N24News: A New Dataset for Multimodal News Classification](https://mlsanity.com/paper/2108.13327/) (2021)
- [DATE: Detecting Anomalies in Text via Self-Supervision of Transformers](https://mlsanity.com/paper/2104.05591/) (2021)
- [Few-shot Anomaly Detection in Text with Deviation Learning](https://mlsanity.com/paper/2308.11780/) (2023)

## Built on by (most cited first, 17 of 17)

- [AD-LLM: Benchmarking Large Language Models for Anomaly Detection](https://mlsanity.com/paper/2412.11142/) (2024)
- [Adaptive Fault Tolerance Mechanisms of Large Language Models in Cloud Computing Environments](https://mlsanity.com/paper/2503.12228/) (2025)
- [Scalability Optimization in Cloud-Based AI Inference Services: Strategies for Real-Time Load Balancing and Automated Scaling](https://mlsanity.com/paper/2504.15296/) (2025)
- [ClimateLLM: Efficient Weather Forecasting via Frequency-Aware Large Language Models](https://mlsanity.com/paper/2502.11059/) (2025)
- [TAD-Bench: A Comprehensive Benchmark for Embedding-Based Text Anomaly Detection](https://mlsanity.com/paper/2501.11960/) (2025)
- [Discovering Knowledge Deficiencies of Language Models on Massive Knowledge Base](https://mlsanity.com/paper/2503.23361/) (2025)
- [Text Anomaly Detection with Simplified Isolation Kernel](https://mlsanity.com/paper/2510.13197/) (2025)
- [Beyond a Single Perspective: Text Anomaly Detection with Multi-View Language Representations](https://mlsanity.com/paper/2601.17786/) (2026)
- [Anomaly Detection and Early Warning Mechanism for Intelligent Monitoring Systems in Multi-Cloud Environments Based on LLM](https://mlsanity.com/paper/2506.07407/) (2025)
- [Text-ADBench: Text Anomaly Detection Benchmark Based on LLM Embeddings](https://mlsanity.com/paper/2507.12295/) (2025)
- [TAGFN: A Text-Attributed Graph Dataset for Fake News Detection in the Age of LLMs](https://mlsanity.com/paper/2511.21624/) (2025)
- [Useful Features, Backward Scores: OOD in Language-Model Trajectories](https://mlsanity.com/paper/2605.00269/) (2026)
- [Rethinking Weak Supervision in Anomaly Detection: A Comprehensive Benchmark](https://mlsanity.com/paper/2605.26068/) (2026)
- [Masked Diffusion Modeling for Anomaly Detection](https://mlsanity.com/paper/2605.30046/) (2026)
- [AI in Cyberpsychology: A systematic literature review of Cybersecurity enhancement by using AI for analyzing psychology of Victims, Attackers, and Defenders](https://mlsanity.com/paper/2607.13123/) (2026)
- [SIM: Subspace Interaction-based Method for Token-Level Text Anomaly Detection](https://mlsanity.com/paper/2609.08200/) (2026)
- [DiFA: Dual Evidence Fusion and Aggregation for Token-Level Text Anomaly Detection](https://mlsanity.com/paper/2609.22136/) (2026)

## Abstract (arXiv)

Anomaly detection (AD) is an important machine learning task with applications in fraud detection, content moderation, and user behavior analysis. However, AD is relatively understudied in a natural language processing (NLP) context, limiting its effectiveness in detecting harmful content, phishing attempts, and spam reviews. We introduce NLP-ADBench, the most comprehensive NLP anomaly detection (NLP-AD) benchmark to date, which includes eight curated datasets and 19 state-of-the-art algorithms. These span 3 end-to-end methods and 16 two-step approaches that adapt classical, non-AD methods to language embeddings from BERT and OpenAI. Our empirical results show that no single model dominates across all datasets, indicating a need for automated model selection. Moreover, two-step methods with transformer-based embeddings consistently outperform specialized end-to-end approaches, with OpenAI embeddings outperforming those of BERT. We release NLP-ADBench at https://github.com/USC-FORTIS/NLP-ADBench, providing a unified framework for NLP-AD and supporting future investigations.