# Evaluating Named Entity Recognition Models for Russian Cultural News Texts: From BERT to LLM

arXiv: https://arxiv.org/abs/2506.02589 · HTML: https://mlsanity.com/paper/2506.02589/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 10 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Builds on (most cited first, 4 of 4)

- [Lessons from the Trenches on Reproducible Evaluation of Language Models](https://mlsanity.com/paper/2405.14782/) (2024)
- [Named Entity Recognition and Classification on Historical Documents: A Survey](https://mlsanity.com/paper/2109.11406/) (2021)
- [MetaIE: Distilling a Meta Model from LLM for All Kinds of Information Extraction Tasks](https://mlsanity.com/paper/2404.00457/) (2024)
- [Yes but.. Can ChatGPT Identify Entities in Historical Documents?](https://mlsanity.com/paper/2303.17322/) (2023)

## Built on by (most cited first, 1 of 1)

- [Large Language Models for Geolocation Extraction in Humanitarian Crisis Response](https://mlsanity.com/paper/2602.08872/) (2026)

## Abstract (arXiv)

This paper addresses the challenge of Named Entity Recognition (NER) for person names within the specialized domain of Russian news texts concerning cultural events. The study utilizes the unique SPbLitGuide dataset, a collection of event announcements from Saint Petersburg spanning 1999 to 2019. A comparative evaluation of diverse NER models is presented, encompassing established transformer-based architectures such as DeepPavlov, RoBERTa, and SpaCy, alongside recent Large Language Models (LLMs) including GPT-3.5, GPT-4, and GPT-4o. Key findings highlight the superior performance of GPT-4o when provided with specific prompting for JSON output, achieving an F1 score of 0.93. Furthermore, GPT-4 demonstrated the highest precision at 0.99. The research contributes to a deeper understanding of current NER model capabilities and limitations when applied to morphologically rich languages like Russian within the cultural heritage domain, offering insights for researchers and practitioners. Follow-up evaluation with GPT-4.1 (April 2025) achieves F1=0.94 for both simple and structured prompts, demonstrating rapid progress across model families and simplified deployment requirements.