# Sparse Conditional Hidden Markov Model for Weakly Supervised Named Entity Recognition

arXiv: https://arxiv.org/abs/2205.14228 · HTML: https://mlsanity.com/paper/2205.14228/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 27 May 2022.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Training Complex Models with Multi-Task Weak Supervision](https://mlsanity.com/paper/1810.02840/) (2018)
- [WRENCH: A Comprehensive Benchmark for Weak Supervision](https://mlsanity.com/paper/2109.11377/) (2021)
- [DrNAS: Dirichlet Neural Architecture Search](https://mlsanity.com/paper/2006.10355/) (2020)
- [Learning Named Entity Tagger using Domain-Specific Dictionary](https://mlsanity.com/paper/1809.03599/) (2018)
- [Learning Neural Templates for Text Generation](https://mlsanity.com/paper/1808.10122/) (2018)
- [Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling](https://mlsanity.com/paper/2012.06046/) (2020)
- [Named Entity Recognition without Labelled Data: A Weak Supervision Approach](https://mlsanity.com/paper/2004.14723/) (2020)
- [Denoising Multi-Source Weak Supervision for Neural Text Classification](https://mlsanity.com/paper/2010.04582/) (2020)
- [Pathwise Derivatives Beyond the Reparameterization Trick](https://mlsanity.com/paper/1806.01851/) (2018)
- [Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling](https://mlsanity.com/paper/1910.04289/) (2019)
- [Unsupervised Neural Hidden Markov Models](https://mlsanity.com/paper/1609.09007/) (2016)
- [skweak: Weak Supervision Made Easy for NLP](https://mlsanity.com/paper/2104.09683/) (2021)
- [BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Recognition](https://mlsanity.com/paper/2105.12848/) (2021)

## Built on by (most cited first, 4 of 4)

- [ChatPD: An LLM-driven Paper-Dataset Networking System](https://mlsanity.com/paper/2505.22349/) (2025)
- [Neural-Hidden-CRF: A Robust Weakly-Supervised Sequence Labeler](https://mlsanity.com/paper/2309.05086/) (2023)
- [DTBench: A Synthetic Benchmark for Document-to-Table Extraction](https://mlsanity.com/paper/2602.13812/) (2026)
- [Beyond Tables: Doc2DB-Bench for Relationally Faithful Document-to-Database Construction](https://mlsanity.com/paper/2608.08459/) (2026)

## Abstract (arXiv)

Weakly supervised named entity recognition methods train label models to aggregate the token annotations of multiple noisy labeling functions (LFs) without seeing any manually annotated labels. To work well, the label model needs to contextually identify and emphasize well-performed LFs while down-weighting the under-performers. However, evaluating the LFs is challenging due to the lack of ground truths. To address this issue, we propose the sparse conditional hidden Markov model (Sparse-CHMM). Instead of predicting the entire emission matrix as other HMM-based methods, Sparse-CHMM focuses on estimating its diagonal elements, which are considered as the reliability scores of the LFs. The sparse scores are then expanded to the full-fledged emission matrix with pre-defined expansion functions. We also augment the emission with weighted XOR scores, which track the probabilities of an LF observing incorrect entities. Sparse-CHMM is optimized through unsupervised learning with a three-stage training pipeline that reduces the training difficulty and prevents the model from falling into local optima. Compared with the baselines in the Wrench benchmark, Sparse-CHMM achieves a 3.01 average F1 score improvement on five comprehensive datasets. Experiments show that each component of Sparse-CHMM is effective, and the estimated LF reliabilities strongly correlate with true LF F1 scores.