# Multi-Task Learning for Sequence Tagging: An Empirical Study

arXiv: https://arxiv.org/abs/1808.04151 · HTML: https://mlsanity.com/paper/1808.04151/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2018.

## Built on by (most cited first, 15 of 15)

- [Structured Prediction as Translation between Augmented Natural Languages](https://mlsanity.com/paper/2101.05779/) (2021)
- [Multi-Task Learning in Natural Language Processing: An Overview](https://mlsanity.com/paper/2109.09138/) (2021)
- [Specializing Multilingual Language Models: An Empirical Study](https://mlsanity.com/paper/2106.09063/) (2021)
- [Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning](https://mlsanity.com/paper/2204.11117/) (2022)
- [When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learning](https://mlsanity.com/paper/2205.08124/) (2022)
- [Cross-Domain Multi-Task Learning for Sequential Sentence Classification in Research Papers](https://mlsanity.com/paper/2102.06008/) (2021)
- [Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection](https://mlsanity.com/paper/2104.09261/) (2021)
- [When Multi-Task Learning Meets Partial Supervision: A Computer Vision Review](https://mlsanity.com/paper/2307.14382/) (2023)
- [Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generation](https://mlsanity.com/paper/2305.07393/) (2023)
- [DeepAnalyze: Learning to Localize Crashes at Scale](https://mlsanity.com/paper/2109.14326/) (2021)
- [Multi-Task Learning for Front-End Text Processing in TTS](https://mlsanity.com/paper/2401.06321/) (2024)
- [MetaXT: Meta Cross-Task Transfer between Disparate Label Spaces](https://mlsanity.com/paper/2109.04240/) (2021)
- [Don't Waste Data: Transfer Learning to Leverage All Data for Machine-Learnt Climate Model Emulation](https://mlsanity.com/paper/2210.04001/) (2022)
- [Multi-head Sequence Tagging Model for Grammatical Error Correction](https://mlsanity.com/paper/2410.16473/) (2024)
- [A Hybrid Modeling Framework for Crop Prediction Tasks via Dynamic Parameter Calibration and Multi-Task Learning](https://mlsanity.com/paper/2603.15411/) (2026)

## Abstract (arXiv)

We study three general multi-task learning (MTL) approaches on 11 sequence tagging tasks. Our extensive empirical results show that in about 50% of the cases, jointly learning all 11 tasks improves upon either independent or pairwise learning of the tasks. We also show that pairwise MTL can inform us what tasks can benefit others or what tasks can be benefited if they are learned jointly. In particular, we identify tasks that can always benefit others as well as tasks that can always be harmed by others. Interestingly, one of our MTL approaches yields embeddings of the tasks that reveal the natural clustering of semantic and syntactic tasks. Our inquiries have opened the doors to further utilization of MTL in NLP.