# Uncertainty-Aware Label Refinement for Sequence Labeling

arXiv: https://arxiv.org/abs/2012.10608 · HTML: https://mlsanity.com/paper/2012.10608/

## Key facts

- Cited by 11 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 19 Dec 2020.

## Built on by (most cited first, 11 of 11)

- [Template-Based Named Entity Recognition Using BART](https://mlsanity.com/paper/2106.01760/) (2021)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting](https://mlsanity.com/paper/2109.00720/) (2021)
- [Optimizing Bi-Encoder for Named Entity Recognition via Contrastive Learning](https://mlsanity.com/paper/2208.14565/) (2022)
- [A Survey on Deep Learning Event Extraction: Approaches and Applications](https://mlsanity.com/paper/2107.02126/) (2021)
- [Training Multimedia Event Extraction With Generated Images and Captions](https://mlsanity.com/paper/2306.08966/) (2023)
- [Investigating Non-local Features for Neural Constituency Parsing](https://mlsanity.com/paper/2109.12814/) (2021)
- [Improving Graph-based Sentence Ordering with Iteratively Predicted Pairwise Orderings](https://mlsanity.com/paper/2110.06446/) (2021)
- [TURNER: The Uncertainty-based Retrieval Framework for Chinese NER](https://mlsanity.com/paper/2202.09022/) (2022)
- [YATO: Yet Another deep learning based Text analysis Open toolkit](https://mlsanity.com/paper/2209.13877/) (2022)
- [SetCSE: Set Operations using Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2404.17606/) (2024)

## Abstract (arXiv)

Conditional random fields (CRF) for label decoding has become ubiquitous in sequence labeling tasks. However, the local label dependencies and inefficient Viterbi decoding have always been a problem to be solved. In this work, we introduce a novel two-stage label decoding framework to model long-term label dependencies, while being much more computationally efficient. A base model first predicts draft labels, and then a novel two-stream self-attention model makes refinements on these draft predictions based on long-range label dependencies, which can achieve parallel decoding for a faster prediction. In addition, in order to mitigate the side effects of incorrect draft labels, Bayesian neural networks are used to indicate the labels with a high probability of being wrong, which can greatly assist in preventing error propagation. The experimental results on three sequence labeling benchmarks demonstrated that the proposed method not only outperformed the CRF-based methods but also greatly accelerated the inference process.