Oddballness: universal anomaly detection with language models
Not on the map. Why?
Abstract (arXiv)
We present a new method to detect anomalies in texts (in general: in sequences of any data), using language models, in a totally unsupervised manner. The method considers probabilities (likelihoods) generated by a language model, but instead of focusing on low-likelihood tokens, it considers a new metric introduced in this paper: oddballness. Oddballness measures how ``strange'' a given token is according to the language model. We demonstrate in grammatical error detection tasks (a specific case of text anomaly detection) that oddballness is better than just considering low-likelihood events, if a totally unsupervised setup is assumed.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 7 of its 20 references that are in the map. First posted on arXiv 4 Sep 2024.
Builds on
7 of 20 references.
References · 7
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2021LogBERT: Log Anomaly Detection via BERTcited by 114
- 2022Grammatical Error Correction: A Survey of the State of the Artcited by 43
- 2023LogGPT: Log Anomaly Detection via GPTcited by 26
- 2016Compositional Sequence Labeling Models for Error Detection in Learner Writingcited by 16
- 2019Context is Key: Grammatical Error Detection with Contextual Word Representationscited by 14
- 2019Multi-Head Multi-Layer Attention to Deep Language Representations for Grammatical Error Detectioncited by 7
Built on by
Citing papers per year: 2025: 2.
2 papers.