# Universal NER v2: Towards a Massively Multilingual Named Entity Recognition Benchmark

arXiv: https://arxiv.org/abs/2604.12744 · HTML: https://mlsanity.com/paper/2604.12744/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 14 Apr 2026.

## Builds on (most cited first, 16 of 16)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Large Language Models for Data Annotation and Synthesis: A Survey](https://mlsanity.com/paper/2402.13446/) (2024)
- [Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning](https://mlsanity.com/paper/2402.06619/) (2024)
- [Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language](https://mlsanity.com/paper/2004.12440/) (2020)
- [NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data](https://mlsanity.com/paper/2402.15343/) (2024)
- [Frustratingly Easy Label Projection for Cross-lingual Transfer](https://mlsanity.com/paper/2211.15613/) (2022)
- [Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs](https://mlsanity.com/paper/2502.14830/) (2025)
- [Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark](https://mlsanity.com/paper/2311.09122/) (2023)
- [NorNE: Annotating Named Entities for Norwegian](https://mlsanity.com/paper/1911.12146/) (2019)
- [Retrofitting Large Language Models with Dynamic Tokenization](https://mlsanity.com/paper/2411.18553/) (2024)
- [Translation and Fusion Improves Zero-shot Cross-lingual Information Extraction](https://mlsanity.com/paper/2305.13582/) (2023)
- [Multilingual Pretraining for Pixel Language Models](https://mlsanity.com/paper/2505.21265/) (2025)
- [Improving Large Language Models for Clinical Named Entity Recognition via Prompt Engineering](https://mlsanity.com/paper/2303.16416/) (2023)
- [The Privileged Students: On the Value of Initialization in Multilingual Knowledge Distillation](https://mlsanity.com/paper/2406.16524/) (2024)
- [skLEP: A Slovak General Language Understanding Benchmark](https://mlsanity.com/paper/2506.21508/) (2025)
- [Meta-Pretraining for Zero-Shot Cross-Lingual Named Entity Recognition in Low-Resource Philippine Languages](https://mlsanity.com/paper/2509.02160/) (2025)

## Abstract (arXiv)

While multilingual language models promise to bring the benefits of LLMs to speakers of many languages, gold-standard evaluation benchmarks in most languages to interrogate these assumptions remain scarce. The Universal NER project, now entering its fourth year, is dedicated to building gold-standard multilingual Named Entity Recognition (NER) benchmark datasets. Inspired by existing massively multilingual efforts for other core NLP tasks (e.g., Universal Dependencies), the project uses a general tagset and thorough annotation guidelines to collect standardized, cross-lingual annotations of named entity spans. The first installment (UNER v1) was released in 2024, and the project has continued and expanded since then, with various organizers, annotators, and collaborators in an active community.