# Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning

arXiv: https://arxiv.org/abs/2105.03654 · HTML: https://mlsanity.com/paper/2105.03654/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 59 references that are in the map.
- First posted on arXiv 8 May 2021.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://mlsanity.com/paper/1603.01354/) (2016)
- [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020)
- [Dice Loss for Data-imbalanced NLP Tasks](https://mlsanity.com/paper/1911.02855/) (2019)
- [A Survey on Multi-view Learning](https://mlsanity.com/paper/1304.5634/) (2013)
- [Named Entity Recognition as Dependency Parsing](https://mlsanity.com/paper/2005.07150/) (2020)
- [Automated Concatenation of Embeddings for Structured Prediction](https://mlsanity.com/paper/2010.05006/) (2020)
- [Retrieve and Refine: Improved Sequence Generation Models For Dialogue](https://mlsanity.com/paper/1808.04776/) (2018)
- [Semi-Supervised Sequence Modeling with Cross-View Training](https://mlsanity.com/paper/1809.08370/) (2018)
- [A Retrieve-and-Edit Framework for Predicting Structured Outputs](https://mlsanity.com/paper/1812.01194/) (2018)
- [Guiding Neural Machine Translation with Retrieved Translation Pieces](https://mlsanity.com/paper/1804.02559/) (2018)
- [CrossWeigh: Training Named Entity Tagger from Imperfect Annotations](https://mlsanity.com/paper/1909.01441/) (2019)
- [Retrieval-Based Neural Code Generation](https://mlsanity.com/paper/1808.10025/) (2018)
- [Cloze-driven Pretraining of Self-attention Networks](https://mlsanity.com/paper/1903.07785/) (2019)
- [Exploring Cross-sentence Contexts for Named Entity Recognition with BERT](https://mlsanity.com/paper/2006.01563/) (2020)
- [Named Entity Recognition for Social Media Texts with Semantic Augmentation](https://mlsanity.com/paper/2010.15458/) (2020)
- [Coupling Retrieval and Meta-Learning for Context-Dependent Semantic Parsing](https://mlsanity.com/paper/1906.07108/) (2019)
- [Structure-Level Knowledge Distillation For Multilingual Sequence Labeling](https://mlsanity.com/paper/2004.03846/) (2020)
- [BioFLAIR: Pretrained Pooled Contextualized Embeddings for Biomedical Sequence Labeling Tasks](https://mlsanity.com/paper/1908.05760/) (2019)
- [Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor](https://mlsanity.com/paper/2010.05010/) (2020)
- [More Embeddings, Better Sequence Labelers?](https://mlsanity.com/paper/2009.08330/) (2020)

## Built on by (most cited first, 30 of 76)

- [GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction](https://mlsanity.com/paper/2310.03668/) (2023)
- [Retrieval-Augmented Generation for Natural Language Processing: A Survey](https://mlsanity.com/paper/2407.13193/) (2024)
- [Empirical Study of Zero-Shot NER with ChatGPT](https://mlsanity.com/paper/2310.10035/) (2023)
- [EcomGPT: Instruction-tuning Large Language Models with Chain-of-Task Tasks for E-commerce](https://mlsanity.com/paper/2308.06966/) (2023)
- [Universal Information Extraction as Unified Semantic Matching](https://mlsanity.com/paper/2301.03282/) (2023)
- [LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model](https://mlsanity.com/paper/2304.06248/) (2023)
- [Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER](https://mlsanity.com/paper/2110.08454/) (2021)
- [LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs -- No Silver Bullet for LC or RAG Routing](https://mlsanity.com/paper/2502.09977/) (2025)
- [Named Entity and Relation Extraction with Multi-Modal Retrieval](https://mlsanity.com/paper/2212.01612/) (2022)
- [ITA: Image-Text Alignments for Multi-Modal Named Entity Recognition](https://mlsanity.com/paper/2112.06482/) (2021)
- [Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learning](https://mlsanity.com/paper/2208.14565/) (2022)
- [Distantly Supervised Named Entity Recognition via Confidence-Based Multi-Class Positive and Unlabeled Learning](https://mlsanity.com/paper/2204.09589/) (2022)
- [Parallel Instance Query Network for Named Entity Recognition](https://mlsanity.com/paper/2203.10545/) (2022)
- [KPI-BERT: A Joint Named Entity Recognition and Relation Extraction Model for Financial Reports](https://mlsanity.com/paper/2208.02140/) (2022)
- [DAMO-NLP at SemEval-2022 Task 11: A Knowledge-based System for Multilingual Named Entity Recognition](https://mlsanity.com/paper/2203.00545/) (2022)
- [FiNER-ORD: Financial Named Entity Recognition Open Research Dataset](https://mlsanity.com/paper/2302.11157/) (2023)
- [Adapt in Contexts: Retrieval-Augmented Domain Adaptation via In-Context Learning](https://mlsanity.com/paper/2311.11551/) (2023)
- [medIKAL: Integrating Knowledge Graphs as Assistants of LLMs for Enhanced Clinical Diagnosis on EMRs](https://mlsanity.com/paper/2406.14326/) (2024)
- [KnowGL: Knowledge Generation and Linking from Text](https://mlsanity.com/paper/2210.13952/) (2022)
- [Context-NER : Contextual Phrase Generation at Scale](https://mlsanity.com/paper/2109.08079/) (2021)
- [Domain-Specific NER via Retrieving Correlated Samples](https://mlsanity.com/paper/2208.12995/) (2022)
- [Prompting ChatGPT in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledge](https://mlsanity.com/paper/2305.12212/) (2023)
- [ProductAgent: Benchmarking Conversational Product Search Agent with Asking Clarification Questions](https://mlsanity.com/paper/2407.00942/) (2024)
- [KPI-EDGAR: A Novel Dataset and Accompanying Metric for Relation Extraction from Financial Documents](https://mlsanity.com/paper/2210.09163/) (2022)
- [DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-augmented System for Multilingual Named Entity Recognition](https://mlsanity.com/paper/2305.03688/) (2023)
- [SeqScore: Addressing Barriers to Reproducible Named Entity Recognition Evaluation](https://mlsanity.com/paper/2107.14154/) (2021)
- [PeerDA: Data Augmentation via Modeling Peer Relation for Span Identification Tasks](https://mlsanity.com/paper/2210.08855/) (2022)
- [Representation biases in sentence transformers](https://mlsanity.com/paper/2301.13039/) (2023)
- [RexUIE: A Recursive Method with Explicit Schema Instructor for Universal Information Extraction](https://mlsanity.com/paper/2304.14770/) (2023)
- [Chain-of-Thought Prompt Distillation for Multimodal Named Entity Recognition and Multimodal Relation Extraction](https://mlsanity.com/paper/2306.14122/) (2023)

## Abstract (arXiv)

Recent advances in Named Entity Recognition (NER) show that document-level contexts can significantly improve model performance. In many application scenarios, however, such contexts are not available. In this paper, we propose to find external contexts of a sentence by retrieving and selecting a set of semantically relevant texts through a search engine, with the original sentence as the query. We find empirically that the contextual representations computed on the retrieval-based input view, constructed through the concatenation of a sentence and its external contexts, can achieve significantly improved performance compared to the original input view based only on the sentence. Furthermore, we can improve the model performance of both input views by Cooperative Learning, a training method that encourages the two input views to produce similar contextual representations or output label distributions. Experiments show that our approach can achieve new state-of-the-art performance on 8 NER data sets across 5 domains.