# Uncertainty-Aware Bootstrap Learning for Joint Extraction on Distantly-Supervised Data

arXiv: https://arxiv.org/abs/2305.03827 · HTML: https://mlsanity.com/paper/2305.03827/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 5 May 2023.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Competence-based Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/1903.09848/) (2019)
- [End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures](https://mlsanity.com/paper/1601.00770/) (2016)
- [Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme](https://mlsanity.com/paper/1706.05075/) (2017)
- [Quantifying Uncertainties in Natural Language Processing Tasks](https://mlsanity.com/paper/1811.07253/) (2018)
- [Joint Extraction of Entities and Relations Based on a Novel Decomposition Strategy](https://mlsanity.com/paper/1909.04273/) (2019)
- [Self-Ensemble Adversarial Training for Improved Robustness](https://mlsanity.com/paper/2203.09678/) (2022)
- [Improving Back-Translation with Uncertainty-based Confidence Estimation](https://mlsanity.com/paper/1909.00157/) (2019)
- [Confidence Modeling for Neural Semantic Parsing](https://mlsanity.com/paper/1805.04604/) (2018)
- [Gradient Imitation Reinforcement Learning for Low Resource Relation Extraction](https://mlsanity.com/paper/2109.06415/) (2021)
- [Deep Semantic Role Labeling with Self-Attention](https://mlsanity.com/paper/1712.01586/) (2017)
- [SHARE: a System for Hierarchical Assistive Recipe Editing](https://mlsanity.com/paper/2105.08185/) (2021)
- [Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach](https://mlsanity.com/paper/1707.00166/) (2017)
- [Empower Distantly Supervised Relation Extraction with Collaborative Adversarial Training](https://mlsanity.com/paper/2106.10835/) (2021)
- [Knowing False Negatives: An Adversarial Training Method for Distantly Supervised Relation Extraction](https://mlsanity.com/paper/2109.02099/) (2021)

## Built on by (most cited first, 7 of 7)

- [An Open Source Data Contamination Report for Large Language Models](https://mlsanity.com/paper/2310.17589/) (2023)
- [RT-LM: Uncertainty-Aware Resource Management for Real-Time Inference of Language Models](https://mlsanity.com/paper/2309.06619/) (2023)
- [GLAD: Content-aware Dynamic Graphs For Log Anomaly Detection](https://mlsanity.com/paper/2309.05953/) (2023)
- [Uncertainty Awareness of Large Language Models Under Code Distribution Shifts: A Benchmark Study](https://mlsanity.com/paper/2402.05939/) (2024)
- [LeMix: Unified Scheduling for LLM Training and Inference on Multi-GPU Systems](https://mlsanity.com/paper/2507.21276/) (2025)
- [Distantly-Supervised Joint Extraction with Noise-Robust Learning](https://mlsanity.com/paper/2310.04994/) (2023)
- [MACE: A Hybrid LLM Serving System with Colocated SLO-aware Continuous Retraining Alignment](https://mlsanity.com/paper/2510.03283/) (2025)

## Abstract (arXiv)

Jointly extracting entity pairs and their relations is challenging when working on distantly-supervised data with ambiguous or noisy labels. To mitigate such impact, we propose uncertainty-aware bootstrap learning, which is motivated by the intuition that the higher uncertainty of an instance, the more likely the model confidence is inconsistent with the ground truths. Specifically, we first explore instance-level data uncertainty to create an initial high-confident examples. Such subset serves as filtering noisy instances and facilitating the model to converge fast at the early stage. During bootstrap learning, we propose self-ensembling as a regularizer to alleviate inter-model uncertainty produced by noisy labels. We further define probability variance of joint tagging probabilities to estimate inner-model parametric uncertainty, which is used to select and build up new reliable training instances for the next iteration. Experimental results on two large datasets reveal that our approach outperforms existing strong baselines and related methods.