# The Role of Global and Local Context in Named Entity Recognition

arXiv: https://arxiv.org/abs/2305.03132 · HTML: https://mlsanity.com/paper/2305.03132/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 4 May 2023.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)
- [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020)
- [Sparse Sinkhorn Attention](https://mlsanity.com/paper/2002.11296/) (2020)
- [Synthesizer: Rethinking Self-Attention in Transformer Models](https://mlsanity.com/paper/2005.00743/) (2020)
- [Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning](https://mlsanity.com/paper/2105.03654/) (2021)
- [Exploring Cross-sentence Contexts for Named Entity Recognition with BERT](https://mlsanity.com/paper/2006.01563/) (2020)
- [Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing](https://mlsanity.com/paper/1906.07108/) (2019)
- [Domain-Specific NER via Retrieving Correlated Samples](https://mlsanity.com/paper/2208.12995/) (2022)
- [Named Entity Recognition -- Is there a glass ceiling?](https://mlsanity.com/paper/1910.02403/) (2019)

## Built on by (most cited first, 2 of 2)

- [Learning to Rank Context for Named Entity Recognition Using a Synthetic Dataset](https://mlsanity.com/paper/2310.10118/) (2023)
- [Annotation Guidelines for Corpus Novelties: Part 1 -- Named Entity Recognition](https://mlsanity.com/paper/2410.02281/) (2024)

## Abstract (arXiv)

Pre-trained transformer-based models have recently shown great performance when applied to Named Entity Recognition (NER). As the complexity of their self-attention mechanism prevents them from processing long documents at once, these models are usually applied in a sequential fashion. Such an approach unfortunately only incorporates local context and prevents leveraging global document context in long documents such as novels, which might hinder performance. In this article, we explore the impact of global document context, and its relationships with local context. We find that correctly retrieving global document context has a greater impact on performance than only leveraging local context, prompting for further research on how to better retrieve that context.