# BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision

arXiv: https://arxiv.org/abs/2006.15509 · HTML: https://mlsanity.com/paper/2006.15509/

## Key facts

- Cited by 91 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2020.

## Built on by (most cited first, 30 of 91)

- [WRENCH: A Comprehensive Benchmark for Weak Supervision](https://mlsanity.com/paper/2109.11377/) (2021)
- [Template-free Prompt Tuning for Few-shot NER](https://mlsanity.com/paper/2109.13532/) (2021)
- [A Survey on Programmatic Weak Supervision](https://mlsanity.com/paper/2202.05433/) (2022)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training](https://mlsanity.com/paper/2109.05003/) (2021)
- [Noisy-Labeled NER with Confidence Estimation](https://mlsanity.com/paper/2104.04318/) (2021)
- [ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval](https://mlsanity.com/paper/2305.10703/) (2023)
- [De-biasing Distantly Supervised Named Entity Recognition via Causal Intervention](https://mlsanity.com/paper/2106.09233/) (2021)
- [Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learning](https://mlsanity.com/paper/2208.14565/) (2022)
- [A general-purpose material property data extraction pipeline from large polymer corpora using Natural Language Processing](https://mlsanity.com/paper/2209.13136/) (2022)
- [Distantly Supervised Named Entity Recognition via Confidence-Based Multi-Class Positive and Unlabeled Learning](https://mlsanity.com/paper/2204.09589/) (2022)
- [Coarse-to-fine Knowledge Graph Domain Adaptation based on Distantly-supervised Iterative Training](https://mlsanity.com/paper/2211.02849/) (2022)
- [Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models](https://mlsanity.com/paper/2311.00287/) (2023)
- [KPI-BERT: A Joint Named Entity Recognition and Relation Extraction Model for Financial Reports](https://mlsanity.com/paper/2208.02140/) (2022)
- [Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data](https://mlsanity.com/paper/2106.08977/) (2021)
- [Searching for Optimal Subword Tokenization in Cross-domain NER](https://mlsanity.com/paper/2206.03352/) (2022)
- [A Simple but Effective Approach to Improve Structured Language Model Output for Information Extraction](https://mlsanity.com/paper/2402.13364/) (2024)
- [BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Recognition](https://mlsanity.com/paper/2105.12848/) (2021)
- [Reinforced Iterative Knowledge Distillation for Cross-Lingual Named Entity Recognition](https://mlsanity.com/paper/2106.00241/) (2021)
- [UCPhrase: Unsupervised Context-aware Quality Phrase Tagging](https://mlsanity.com/paper/2105.14078/) (2021)
- [PRBoost: Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learning](https://mlsanity.com/paper/2203.09735/) (2022)
- [Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble](https://mlsanity.com/paper/2212.06522/) (2022)
- [Neighborhood-Regularized Self-Training for Learning with Few Labels](https://mlsanity.com/paper/2301.03726/) (2023)
- [QUEACO: Borrowing Treasures from Weakly-labeled Behavior Data for Query Attribute Value Extraction](https://mlsanity.com/paper/2108.08468/) (2021)
- [A Review on Knowledge Graphs for Healthcare: Resources, Applications, and Promises](https://mlsanity.com/paper/2306.04802/) (2023)
- [Improving Distantly-Supervised Named Entity Recognition with Self-Collaborative Denoising Learning](https://mlsanity.com/paper/2110.04429/) (2021)
- [Mix of Experts Language Model for Named Entity Recognition](https://mlsanity.com/paper/2404.19192/) (2024)
- [Information Extraction in Low-Resource Scenarios: Survey and Perspective](https://mlsanity.com/paper/2202.08063/) (2022)
- [PTGB: Pre-Train Graph Neural Networks for Brain Network Analysis](https://mlsanity.com/paper/2305.14376/) (2023)
- [DyGen: Learning from Noisy Labels via Dynamics-Enhanced Generative Modeling](https://mlsanity.com/paper/2305.19395/) (2023)

## Abstract (arXiv)

We study the open-domain named entity recognition (NER) problem under distant supervision. The distant supervision, though does not require large amounts of manual annotations, yields highly incomplete and noisy distant labels via external knowledge bases. To address this challenge, we propose a new computational framework -- BOND, which leverages the power of pre-trained language models (e.g., BERT and RoBERTa) to improve the prediction performance of NER models. Specifically, we propose a two-stage training algorithm: In the first stage, we adapt the pre-trained language model to the NER tasks using the distant labels, which can significantly improve the recall and precision; In the second stage, we drop the distant labels, and propose a self-training approach to further improve the model performance. Thorough experiments on 5 benchmark datasets demonstrate the superiority of BOND over existing distantly supervised NER methods. The code and distantly labeled data have been released in https://github.com/cliang1453/BOND.