# Semi-supervised Multitask Learning for Sequence Labeling

arXiv: https://arxiv.org/abs/1704.07156 · HTML: https://mlsanity.com/paper/1704.07156/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2017.

## Built on by (most cited first, 20 of 20)

- [A Survey on Deep Semi-supervised Learning](https://mlsanity.com/paper/2103.00550/) (2021)
- [BERT: A Review of Applications in Natural Language Processing and Understanding](https://mlsanity.com/paper/2103.11943/) (2021)
- [Multi-Task Learning in Natural Language Processing: An Overview](https://mlsanity.com/paper/2109.09138/) (2021)
- [A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods](https://mlsanity.com/paper/2204.03508/) (2022)
- [Structural Information Preserving for Graph-to-Text Generation](https://mlsanity.com/paper/2102.06749/) (2021)
- [Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction](https://mlsanity.com/paper/2105.04443/) (2021)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [Recent Trends in Named Entity Recognition (NER)](https://mlsanity.com/paper/2101.11420/) (2021)
- [Ensemble Learning for Large Language Models in Text and Code Generation: A Survey](https://mlsanity.com/paper/2503.13505/) (2025)
- [Pay More Attention to Relation Exploration for Knowledge Base Question Answering](https://mlsanity.com/paper/2305.02118/) (2023)
- [Spatio-Temporal Momentum: Jointly Learning Time-Series and Cross-Sectional Strategies](https://mlsanity.com/paper/2302.10175/) (2023)
- [Assessing Phrase Break of ESL Speech with Pre-trained Language Models and Large Language Models](https://mlsanity.com/paper/2306.04980/) (2023)
- [PCA-based Multi Task Learning: a Random Matrix Approach](https://mlsanity.com/paper/2111.00924/) (2021)
- [Exploring the Capacity of a Large-scale Masked Language Model to Recognize Grammatical Errors](https://mlsanity.com/paper/2108.12216/) (2021)
- [In Situ Answer Sentence Selection at Web-scale](https://mlsanity.com/paper/2201.05984/) (2022)
- [Learning Multitask Gaussian Bayesian Networks](https://mlsanity.com/paper/2205.05343/) (2022)
- [Assessing Phrase Break of ESL speech with Pre-trained Language Models](https://mlsanity.com/paper/2210.16029/) (2022)
- [Dynamic Named Entity Recognition](https://mlsanity.com/paper/2302.10314/) (2023)
- [CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community](https://mlsanity.com/paper/2407.06485/) (2024)
- [Enhancing Grammatical Error Detection using BERT with Cleaned Lang-8 Dataset](https://mlsanity.com/paper/2411.15523/) (2024)

## Abstract (arXiv)

We propose a sequence labeling framework with a secondary training objective, learning to predict surrounding words for every word in the dataset. This language modeling objective incentivises the system to learn general-purpose patterns of semantic and syntactic composition, which are also useful for improving accuracy on different sequence labeling tasks. The architecture was evaluated on a range of datasets, covering the tasks of error detection in learner texts, named entity recognition, chunking and POS-tagging. The novel language modeling objective provided consistent performance improvements on every benchmark, without requiring any additional annotated or unannotated data.