# An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labeling

arXiv: https://arxiv.org/abs/2402.13534 · HTML: https://mlsanity.com/paper/2402.13534/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Improving Back-Translation with Uncertainty-based Confidence Estimation](https://mlsanity.com/paper/1909.00157/) (2019)
- [Lexicon Enhanced Chinese Sequence Labeling Using BERT Adapter](https://mlsanity.com/paper/2105.07148/) (2021)
- [Self-Paced Learning for Neural Machine Translation](https://mlsanity.com/paper/2010.04505/) (2020)
- [Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF](https://mlsanity.com/paper/1704.01314/) (2017)
- [Data Selection Curriculum for Neural Machine Translation](https://mlsanity.com/paper/2203.13867/) (2022)
- [Generative Entity Typing with Curriculum Learning](https://mlsanity.com/paper/2210.02914/) (2022)
- [Switch-LSTMs for Multi-Criteria Chinese Word Segmentation](https://mlsanity.com/paper/1812.08033/) (2018)
- [That Slepen Al the Nyght with Open Ye! Cross-era Sequence Segmentation with Switch-memory](https://mlsanity.com/paper/2209.02967/) (2022)
- [A Feature-Enriched Neural Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging](https://mlsanity.com/paper/1611.05384/) (2016)
- [Joint Chinese Word Segmentation and Part-of-speech Tagging via Two-stage Span Labeling](https://mlsanity.com/paper/2112.09488/) (2021)
- [DAG-based Long Short-Term Memory for Neural Word Segmentation](https://mlsanity.com/paper/1707.00248/) (2017)

## Built on by (most cited first, 1 of 1)

- [Disentangling Curriculum Learning in NLP: Towards a Unifying Taxonomy](https://mlsanity.com/paper/2607.18984/) (2026)

## Abstract (arXiv)

Sequence labeling models often benefit from incorporating external knowledge. However, this practice introduces data heterogeneity and complicates the model with additional modules, leading to increased expenses for training a high-performing model. To address this challenge, we propose a two-stage curriculum learning (TCL) framework specifically designed for sequence labeling tasks. The TCL framework enhances training by gradually introducing data instances from easy to hard, aiming to improve both performance and training speed. Furthermore, we explore different metrics for assessing the difficulty levels of sequence labeling tasks. Through extensive experimentation on six Chinese word segmentation (CWS) and Part-of-speech tagging (POS) datasets, we demonstrate the effectiveness of our model in enhancing the performance of sequence labeling models. Additionally, our analysis indicates that TCL accelerates training and alleviates the slow training problem associated with complex models.