# Few-shot Anomaly Detection in Text with Deviation Learning

arXiv: https://arxiv.org/abs/2308.11780 · HTML: https://mlsanity.com/paper/2308.11780/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 22 Aug 2023.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [Deep Semi-Supervised Anomaly Detection](https://mlsanity.com/paper/1906.02694/) (2019)
- [Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning](https://mlsanity.com/paper/2101.10030/) (2021)
- [Deep Anomaly Detection with Deviation Networks](https://mlsanity.com/paper/1911.08623/) (2019)
- [Types of Out-of-Distribution Texts and How to Detect Them](https://mlsanity.com/paper/2109.06827/) (2021)
- [Toward Supervised Anomaly Detection](https://mlsanity.com/paper/1401.6424/) (2014)
- [Explainable Deep Few-shot Anomaly Detection with Deviation Networks](https://mlsanity.com/paper/2108.00462/) (2021)
- [Towards Few-Shot Fact-Checking via Perplexity](https://mlsanity.com/paper/2103.09535/) (2021)
- [Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialog](https://mlsanity.com/paper/1912.12800/) (2019)
- [DATE: Detecting Anomalies in Text via Self-Supervision of Transformers](https://mlsanity.com/paper/2104.05591/) (2021)
- [Classifying Idiomatic and Literal Expressions Using Topic Models and Intensity of Emotions](https://mlsanity.com/paper/1802.09961/) (2018)
- [Multi-Level Knowledge Distillation for Out-of-Distribution Detection in Text](https://mlsanity.com/paper/2211.11300/) (2022)
- [Self-Supervised Losses for One-Class Textual Anomaly Detection](https://mlsanity.com/paper/2204.05695/) (2022)

## Built on by (most cited first, 11 of 11)

- [NLP-ADBench: NLP Anomaly Detection Benchmark](https://mlsanity.com/paper/2412.04784/) (2024)
- [Text Anomaly Detection with Simplified Isolation Kernel](https://mlsanity.com/paper/2510.13197/) (2025)
- [Adaptive Deviation Learning for Visual Anomaly Detection with Data Contamination](https://mlsanity.com/paper/2411.09558/) (2024)
- [Beyond a Single Perspective: Text Anomaly Detection with Multi-View Language Representations](https://mlsanity.com/paper/2601.17786/) (2026)
- [A Robust Autoencoder Ensemble-Based Approach for Anomaly Detection in Text](https://mlsanity.com/paper/2405.13031/) (2024)
- [Text-ADBench: Text Anomaly Detection Benchmark Based on LLM Embeddings](https://mlsanity.com/paper/2507.12295/) (2025)
- [Good Enough to Learn: LLM-based Anomaly Detection in ECU Logs without Reliable Labels](https://mlsanity.com/paper/2507.01077/) (2025)
- [Multi-Cue Anomaly Detection and Localization under Data Contamination](https://mlsanity.com/paper/2601.22913/) (2026)
- [Masked Diffusion Modeling for Anomaly Detection](https://mlsanity.com/paper/2605.30046/) (2026)
- [SIM: Subspace Interaction-based Method for Token-Level Text Anomaly Detection](https://mlsanity.com/paper/2609.08200/) (2026)
- [DiFA: Dual Evidence Fusion and Aggregation for Token-Level Text Anomaly Detection](https://mlsanity.com/paper/2609.22136/) (2026)

## Abstract (arXiv)

Most current methods for detecting anomalies in text concentrate on constructing models solely relying on unlabeled data. These models operate on the presumption that no labeled anomalous examples are available, which prevents them from utilizing prior knowledge of anomalies that are typically present in small numbers in many real-world applications. Furthermore, these models prioritize learning feature embeddings rather than optimizing anomaly scores directly, which could lead to suboptimal anomaly scoring and inefficient use of data during the learning process. In this paper, we introduce FATE, a deep few-shot learning-based framework that leverages limited anomaly examples and learns anomaly scores explicitly in an end-to-end method using deviation learning. In this approach, the anomaly scores of normal examples are adjusted to closely resemble reference scores obtained from a prior distribution. Conversely, anomaly samples are forced to have anomalous scores that considerably deviate from the reference score in the upper tail of the prior. Additionally, our model is optimized to learn the distinct behavior of anomalies by utilizing a multi-head self-attention layer and multiple instance learning approaches. Comprehensive experiments on several benchmark datasets demonstrate that our proposed approach attains a new level of state-of-the-art performance.