# OpenNER 1.0: Standardized Open-Access Named Entity Recognition Datasets in 50+ Languages

arXiv: https://arxiv.org/abs/2412.09587 · HTML: https://mlsanity.com/paper/2412.09587/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 82 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [GPT-NER: Named Entity Recognition via Large Language Models](https://mlsanity.com/paper/2304.10428/) (2023)
- [Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier](https://mlsanity.com/paper/2412.04261/) (2024)
- [Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages](https://mlsanity.com/paper/2305.12182/) (2023)
- [Empirical Study of Zero-Shot NER with ChatGPT](https://mlsanity.com/paper/2310.10035/) (2023)
- [Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT](https://mlsanity.com/paper/2205.09651/) (2022)
- [Neural Modeling for Named Entities and Morphology (NEMO^2)](https://mlsanity.com/paper/2007.15620/) (2020)
- [Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark](https://mlsanity.com/paper/2311.09122/) (2023)
- [L3Cube-MahaNER: A Marathi Named Entity Recognition Dataset and BERT models](https://mlsanity.com/paper/2204.06029/) (2022)
- [Parsing Tweets into Universal Dependencies](https://mlsanity.com/paper/1804.08228/) (2018)
- [NorNE: Annotating Named Entities for Norwegian](https://mlsanity.com/paper/1911.12146/) (2019)
- [SemEval-2023 Task 2: Fine-grained Multilingual Named Entity Recognition (MultiCoNER 2)](https://mlsanity.com/paper/2305.06586/) (2023)
- [KazNERD: Kazakh Named Entity Recognition Dataset](https://mlsanity.com/paper/2111.13419/) (2021)
- [Introducing RONEC -- the Romanian Named Entity Corpus](https://mlsanity.com/paper/1909.01247/) (2019)
- [HiNER: A Large Hindi Named Entity Recognition Dataset](https://mlsanity.com/paper/2204.13743/) (2022)
- [SeqScore: Addressing Barriers to Reproducible Named Entity Recognition Evaluation](https://mlsanity.com/paper/2107.14154/) (2021)
- [Sebastian, Basti, Wastl?! Recognizing Named Entities in Bavarian Dialectal Data](https://mlsanity.com/paper/2403.12749/) (2024)
- [Cross-lingual Named Entity Corpus for Slavic Languages](https://mlsanity.com/paper/2404.00482/) (2024)
- [Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages](https://mlsanity.com/paper/2405.04829/) (2024)

## Built on by (most cited first, 2 of 2)

- [Beyond statistical significance: Quantifying uncertainty and statistical variability in multilingual and multitask NLP evaluation](https://mlsanity.com/paper/2509.22612/) (2025)
- [YoNER: A New Yor`ub'a Multi-domain Named Entity Recognition Dataset](https://mlsanity.com/paper/2604.05624/) (2026)

## Abstract (arXiv)

We present OpenNER 1.0, a standardized collection of openly-available named entity recognition (NER) datasets. OpenNER contains 36 NER corpora that span 52 languages, human-annotated in varying named entity ontologies. We correct annotation format issues, standardize the original datasets into a uniform representation with consistent entity type names across corpora, and provide the collection in a structure that enables research in multilingual and multi-ontology NER. We provide baseline results using three pretrained multilingual language models and two large language models to compare the performance of recent models and facilitate future research in NER. We find that no single model is best in all languages and that significant work remains to obtain high performance from LLMs on the NER task. OpenNER is released at https://github.com/bltlab/open-ner.