# Learning Robust Named Entity Recognizers From Noisy Data With Retrieval Augmentation

arXiv: https://arxiv.org/abs/2407.18562 · HTML: https://mlsanity.com/paper/2407.18562/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 26 Jul 2024.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [Whitening Sentence Representations for Better Semantics and Faster Retrieval](https://mlsanity.com/paper/2103.15316/) (2021)
- [Improving the Robustness of Deep Neural Networks via Stability Training](https://mlsanity.com/paper/1604.04326/) (2016)
- [Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning](https://mlsanity.com/paper/2105.03654/) (2021)
- [Robust Encodings: A Framework for Combating Adversarial Typos](https://mlsanity.com/paper/2005.01229/) (2020)
- [Towards Robust Neural Machine Translation](https://mlsanity.com/paper/1805.06130/) (2018)
- [Named Entity and Relation Extraction with Multi-Modal Retrieval](https://mlsanity.com/paper/2212.01612/) (2022)
- [Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Study](https://mlsanity.com/paper/1807.01270/) (2018)
- [Misspelling Oblivious Word Embeddings](https://mlsanity.com/paper/1905.09755/) (2019)
- [NAT: Noise-Aware Training for Robust Neural Sequence Labeling](https://mlsanity.com/paper/2005.07162/) (2020)
- [Empirical Error Modeling Improves Robustness of Noisy Neural Sequence Labeling](https://mlsanity.com/paper/2105.11872/) (2021)
- [From the Paft to the Fiiture: a Fully Automatic NMT and Word Embeddings Method for OCR Post-Correction](https://mlsanity.com/paper/1910.05535/) (2019)

## Built on by (most cited first, 1 of 1)

- [Structure-Guided Entity Resolution: Fine-Tuning LLMs for Robust Name Matching in Complex Linguistic Contexts](https://mlsanity.com/paper/2605.23597/) (2026)

## Abstract (arXiv)

Named entity recognition (NER) models often struggle with noisy inputs, such as those with spelling mistakes or errors generated by Optical Character Recognition processes, and learning a robust NER model is challenging. Existing robust NER models utilize both noisy text and its corresponding gold text for training, which is infeasible in many real-world applications in which gold text is not available. In this paper, we consider a more realistic setting in which only noisy text and its NER labels are available. We propose to retrieve relevant text of the noisy text from a knowledge corpus and use it to enhance the representation of the original noisy input. We design three retrieval methods: sparse retrieval based on lexicon similarity, dense retrieval based on semantic similarity, and self-retrieval based on task-specific text. After retrieving relevant text, we concatenate the retrieved text with the original noisy text and encode them with a transformer network, utilizing self-attention to enhance the contextual token representations of the noisy text using the retrieved text. We further employ a multi-view training framework that improves robust NER without retrieving text during inference. Experiments show that our retrieval-augmented model achieves significant improvements in various noisy NER settings.