# Detecting Spelling and Grammatical Anomalies in Russian Poetry Texts

arXiv: https://arxiv.org/abs/2505.04507 · HTML: https://mlsanity.com/paper/2505.04507/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 7 May 2025.

## Builds on (most cited first, 21 of 21)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [A Simple Recipe for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2106.03830/) (2021)
- [JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction](https://mlsanity.com/paper/1702.04066/) (2017)
- [An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correction](https://mlsanity.com/paper/1909.00502/) (2019)
- [Lower Perplexity is Not Always Human-Like](https://mlsanity.com/paper/2106.01229/) (2021)
- [Testing AI on language comprehension tasks reveals insensitivity to underlying meaning](https://mlsanity.com/paper/2302.12313/) (2023)
- [Maximum-Likelihood Augmented Discrete Generative Adversarial Networks](https://mlsanity.com/paper/1702.07983/) (2017)
- [GRUEN for Evaluating Linguistic Quality of Generated Text](https://mlsanity.com/paper/2010.02498/) (2020)
- [How is BERT surprised? Layerwise detection of linguistic anomalies](https://mlsanity.com/paper/2105.07452/) (2021)
- [RuBLiMP: Russian Benchmark of Linguistic Minimal Pairs](https://mlsanity.com/paper/2406.19232/) (2024)
- [Acceptability Judgements via Examining the Topology of Attention Maps](https://mlsanity.com/paper/2205.09630/) (2022)
- [PyOD 2: A Python Library for Outlier Detection with LLM-powered Model Selection](https://mlsanity.com/paper/2412.12154/) (2024)
- [Acrostic Poem Generation](https://mlsanity.com/paper/2010.02239/) (2020)
- [Say What? Collaborative Pop Lyric Generation Using Multitask Transfer Learning](https://mlsanity.com/paper/2111.07592/) (2021)
- [PoetryDiffusion: Towards Joint Semantic and Metrical Manipulation in Poetry Generation](https://mlsanity.com/paper/2306.08456/) (2023)
- [A Methodology for Generative Spelling Correction via Natural Spelling Errors Emulation across Multiple Domains and Languages](https://mlsanity.com/paper/2308.09435/) (2023)
- [FlaCGEC: A Chinese Grammatical Error Correction Dataset with Fine-grained Linguistic Annotation](https://mlsanity.com/paper/2311.04906/) (2023)
- [Lingxi: A Diversity-aware Chinese Modern Poetry Generation System](https://mlsanity.com/paper/2108.12108/) (2021)
- [Oddballness: universal anomaly detection with language models](https://mlsanity.com/paper/2409.03046/) (2024)
- [Pre-trained Language Models Return Distinguishable Probability Distributions to Unfaithfully Hallucinated Texts](https://mlsanity.com/paper/2409.16658/) (2024)

## Abstract (arXiv)

The quality of natural language texts in fine-tuning datasets plays a critical role in the performance of generative models, particularly in computational creativity tasks such as poem or song lyric generation. Fluency defects in generated poems significantly reduce their value. However, training texts are often sourced from internet-based platforms without stringent quality control, posing a challenge for data engineers to manage defect levels effectively. To address this issue, we propose the use of automated linguistic anomaly detection to identify and filter out low-quality texts from training datasets for creative models. In this paper, we present a comprehensive comparison of unsupervised and supervised text anomaly detection approaches, utilizing both synthetic and human-labeled datasets. We also introduce the RUPOR dataset, a collection of Russian-language human-labeled poems designed for cross-sentence grammatical error detection, and provide the full evaluation code. Our work aims to empower the community with tools and insights to improve the quality of training datasets for generative models in creative domains.