# MultiCoNER: A Large-scale Multilingual dataset for Complex Named Entity Recognition

arXiv: https://arxiv.org/abs/2208.14536 · HTML: https://mlsanity.com/paper/2208.14536/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 17 references that are in the map.
- First posted on arXiv 30 Aug 2022.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search](https://mlsanity.com/paper/2006.05324/) (2020)
- [Generalisation in Named Entity Recognition: A Quantitative Analysis](https://mlsanity.com/paper/1701.02877/) (2017)
- [A Multi-task Approach for Named Entity Recognition in Social Media Data](https://mlsanity.com/paper/1906.04135/) (2019)
- [ner and pos when nothing is capitalized](https://mlsanity.com/paper/1903.11222/) (2019)

## Built on by (most cited first, 19 of 19)

- [A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias](https://mlsanity.com/paper/2404.00929/) (2024)
- [The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges](https://mlsanity.com/paper/2212.09660/) (2022)
- [BOUQuET: dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation](https://mlsanity.com/paper/2502.04314/) (2025)
- [SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understanding](https://mlsanity.com/paper/2308.10529/) (2023)
- [SemEval-2023 Task 2: Fine-grained Multilingual Named Entity Recognition (MultiCoNER 2)](https://mlsanity.com/paper/2305.06586/) (2023)
- [Improving Named Entity Recognition in Telephone Conversations via Effective Active Learning with Human in the Loop](https://mlsanity.com/paper/2211.01354/) (2022)
- [WIDER & CLOSER: Mixture of Short-channel Distillers for Zero-shot Cross-lingual Named Entity Recognition](https://mlsanity.com/paper/2212.03506/) (2022)
- [Linear programming word problems formulation using EnsembleCRF NER labeler and T5 text generator with data augmentations](https://mlsanity.com/paper/2212.14657/) (2022)
- [Statistical Dataset Evaluation: Reliability, Difficulty, and Validity](https://mlsanity.com/paper/2212.09272/) (2022)
- [An Evaluation Study of Hybrid Methods for Multilingual PII Detection](https://mlsanity.com/paper/2510.07551/) (2025)
- [IRC-Bench: Recognizing Entities from Contextual Cues in First-Person Reminiscences](https://mlsanity.com/paper/2605.06142/) (2026)
- [Exploring the Potential of Machine Translation for Generating Named Entity Datasets: A Case Study between Persian and English](https://mlsanity.com/paper/2302.09611/) (2023)
- [IXA/Cogcomp at SemEval-2023 Task 2: Context-enriched Multilingual Named Entity Recognition using Knowledge Bases](https://mlsanity.com/paper/2304.10637/) (2023)
- [PAI at SemEval-2023 Task 2: A Universal System for Named Entity Recognition with External Entity Information](https://mlsanity.com/paper/2305.06099/) (2023)
- [2M-NER: Contrastive Learning for Multilingual and Multimodal NER with Language and Modal Fusion](https://mlsanity.com/paper/2404.17122/) (2024)
- [A Multi-way Parallel Named Entity Annotated Corpus for English, Tamil and Sinhala](https://mlsanity.com/paper/2412.02056/) (2024)
- [MANER: Mask Augmented Named Entity Recognition for Extreme Low-Resource Languages](https://mlsanity.com/paper/2212.09723/) (2022)
- [Novel Benchmark for NER in the Wastewater and Stormwater Domain](https://mlsanity.com/paper/2506.01938/) (2025)
- [Surrogate Substitution Preserves PHI Detectability: A Multi-Detector Equivalence Study](https://mlsanity.com/paper/2608.03172/) (2026)

## Abstract (arXiv)

We present MultiCoNER, a large multilingual dataset for Named Entity Recognition that covers 3 domains (Wiki sentences, questions, and search queries) across 11 languages, as well as multilingual and code-mixing subsets. This dataset is designed to represent contemporary challenges in NER, including low-context scenarios (short and uncased text), syntactically complex entities like movie titles, and long-tail entity distributions. The 26M token dataset is compiled from public resources using techniques such as heuristic-based sentence sampling, template extraction and slotting, and machine translation. We applied two NER models on our dataset: a baseline XLM-RoBERTa model, and a state-of-the-art GEMNET model that leverages gazetteers. The baseline achieves moderate performance (macro-F1=54%), highlighting the difficulty of our data. GEMNET, which uses gazetteers, improvement significantly (average improvement of macro-F1=+30%). MultiCoNER poses challenges even for large pre-trained language models, and we believe that it can help further research in building robust NER systems. MultiCoNER is publicly available at https://registry.opendata.aws/multiconer/ and we hope that this resource will help advance research in various aspects of NER.