# Learning from Noisy Labels for Entity-Centric Information Extraction

arXiv: https://arxiv.org/abs/2104.08656 · HTML: https://mlsanity.com/paper/2104.08656/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 50 references that are in the map.
- First posted on arXiv 17 Apr 2021.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [An Empirical Study of Example Forgetting during Deep Neural Network Learning](https://mlsanity.com/paper/1812.05159/) (2018)
- [MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels](https://mlsanity.com/paper/1712.05055/) (2017)
- [Symmetric Cross Entropy for Robust Learning with Noisy Labels](https://mlsanity.com/paper/1908.06112/) (2019)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Decoupling "when to update" from "how to update"](https://mlsanity.com/paper/1706.02613/) (2017)
- [Simple BERT Models for Relation Extraction and Semantic Role Labeling](https://mlsanity.com/paper/1904.05255/) (2019)
- [Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples](https://mlsanity.com/paper/1704.07433/) (2017)
- [Graph Convolution over Pruned Dependency Trees Improves Relation Extraction](https://mlsanity.com/paper/1809.10185/) (2018)
- [TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task](https://mlsanity.com/paper/2004.14855/) (2020)
- [CrossWeigh: Training Named Entity Tagger from Imperfect Annotations](https://mlsanity.com/paper/1909.01441/) (2019)
- [Learning with Bounded Instance- and Label-dependent Label Noise](https://mlsanity.com/paper/1709.03768/) (2017)
- [Robust Distant Supervision Relation Extraction via Deep Reinforcement Learning](https://mlsanity.com/paper/1805.09927/) (2018)
- [Named Entity Recognition with Partially Annotated Training Data](https://mlsanity.com/paper/1909.09270/) (2019)
- [Finding Influential Instances for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2009.09841/) (2020)
- [Learning with Noisy Labels for Sentence-level Sentiment Classification](https://mlsanity.com/paper/1909.00124/) (2019)
- [Robust Training with Ensemble Consensus](https://mlsanity.com/paper/1910.09792/) (2019)
- [Are Noisy Sentences Useless for Distant Supervised Relation Extraction?](https://mlsanity.com/paper/1911.09788/) (2019)
- [Solving Hard Coreference Problems](https://mlsanity.com/paper/1907.05524/) (2019)

## Built on by (most cited first, 30 of 34)

- [ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT](https://mlsanity.com/paper/2302.10205/) (2023)
- [How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks](https://mlsanity.com/paper/2303.00293/) (2023)
- [From Shortcuts to Triggers: Backdoor Defense with Denoised PoE](https://mlsanity.com/paper/2305.14910/) (2023)
- [Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidance](https://mlsanity.com/paper/2311.01108/) (2023)
- [Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling](https://mlsanity.com/paper/2401.14556/) (2024)
- [Unified Semantic Typing with Meaningful Label Inference](https://mlsanity.com/paper/2205.01826/) (2022)
- [Detecting Label Errors by using Pre-Trained Language Models](https://mlsanity.com/paper/2205.12702/) (2022)
- [CleanCoNLL: A Nearly Noise-Free Named Entity Recognition Dataset](https://mlsanity.com/paper/2310.16225/) (2023)
- [DyGen: Learning from Noisy Labels via Dynamics-Enhanced Generative Modeling](https://mlsanity.com/paper/2305.19395/) (2023)
- [NERetrieve: Dataset for Next Generation Named Entity Recognition and Retrieval](https://mlsanity.com/paper/2310.14282/) (2023)
- [CrowdChecked: Detecting Previously Fact-Checked Claims in Social Media](https://mlsanity.com/paper/2210.04447/) (2022)
- [ACLM: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex NER](https://mlsanity.com/paper/2306.00928/) (2023)
- [ChatEL: Entity Linking with Chatbots](https://mlsanity.com/paper/2402.14858/) (2024)
- [Informed Named Entity Recognition Decoding for Generative Language Models](https://mlsanity.com/paper/2308.07791/) (2023)
- [CleaR: Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learning](https://mlsanity.com/paper/2411.00873/) (2024)
- [BioAug: Conditional Generation based Data Augmentation for Low-Resource Biomedical NER](https://mlsanity.com/paper/2305.10647/) (2023)
- [Enhancing Contrastive Learning with Noise-Guided Attack: Towards Continual Relation Extraction in the Wild](https://mlsanity.com/paper/2305.07085/) (2023)
- [On the use of Silver Standard Data for Zero-shot Classification Tasks in Information Extraction](https://mlsanity.com/paper/2402.18061/) (2024)
- [Span-Oriented Information Extraction -- A Unifying Perspective on Information Extraction](https://mlsanity.com/paper/2403.15453/) (2024)
- [Multilinguals at SemEval-2022 Task 11: Complex NER in Semantically Ambiguous Settings for Low Resource Languages](https://mlsanity.com/paper/2207.06882/) (2022)
- [Technical Report: Impact of Position Bias on Language Models in Token Classification](https://mlsanity.com/paper/2304.13567/) (2023)
- [Bridging Generative and Discriminative Noisy-Label Learning via Direction-Agnostic EM Formulation](https://mlsanity.com/paper/2308.01184/) (2023)
- [Software Entity Recognition with Noise-Robust Learning](https://mlsanity.com/paper/2308.10564/) (2023)
- [NoiseBench: Benchmarking the Impact of Real Label Noise on Named Entity Recognition](https://mlsanity.com/paper/2405.07609/) (2024)
- [SubRegWeigh: Effective and Efficient Annotation Weighing with Subword Regularization](https://mlsanity.com/paper/2409.06216/) (2024)
- [An Empirical Study on Relation Extraction in the Biomedical Domain](https://mlsanity.com/paper/2112.05910/) (2021)
- [TIER-A: Denoising Learning Framework for Information Extraction](https://mlsanity.com/paper/2211.11527/) (2022)
- [Learning with Silver Standard Data for Zero-shot Relation Extraction](https://mlsanity.com/paper/2211.13883/) (2022)
- [On-the-fly Denoising for Data Augmentation in Natural Language Understanding](https://mlsanity.com/paper/2212.10558/) (2022)
- [Information Extraction from Documents: Question Answering vs Token Classification in real-world setups](https://mlsanity.com/paper/2304.10994/) (2023)

## Abstract (arXiv)

Recent information extraction approaches have relied on training deep neural models. However, such models can easily overfit noisy labels and suffer from performance degradation. While it is very costly to filter noisy labels in large learning resources, recent studies show that such labels take more training steps to be memorized and are more frequently forgotten than clean labels, therefore are identifiable in training. Motivated by such properties, we propose a simple co-regularization framework for entity-centric information extraction, which consists of several neural models with identical structures but different parameter initialization. These models are jointly optimized with the task-specific losses and are regularized to generate similar predictions based on an agreement loss, which prevents overfitting on noisy labels. Extensive experiments on two widely used but noisy benchmarks for information extraction, TACRED and CoNLL03, demonstrate the effectiveness of our framework. We release our code to the community for future research.