Multi-Task Learning for Sequence Tagging: An Empirical Study
Not on the map. Why?
Abstract (arXiv)
We study three general multi-task learning (MTL) approaches on 11 sequence tagging tasks. Our extensive empirical results show that in about 50% of the cases, jointly learning all 11 tasks improves upon either independent or pairwise learning of the tasks. We also show that pairwise MTL can inform us what tasks can benefit others or what tasks can be benefited if they are learned jointly. In particular, we identify tasks that can always benefit others as well as tasks that can always be harmed by others. Interestingly, one of our MTL approaches yields embeddings of the tasks that reveal the natural clustering of semantic and syntactic tasks. Our inquiries have opened the doors to further utilization of MTL in NLP.
Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Aug 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 7; 2022: 3; 2023: 2; 2024: 2; 2025: 0; 2026: 1.
15 papers.
Sort:
- 2021Structured Prediction as Translation between Augmented Natural Languagescited by 183
- 2021Multi-Task Learning in Natural Language Processing: An Overviewcited by 61
- 2021Specializing Multilingual Language Models: An Empirical Studycited by 21
- 2022Exploring the Role of Task Transferability in Large-Scale Multi-Task Learningcited by 17
- 2022When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learningcited by 17
- 2021Cross-Domain Multi-Task Learning for Sequential Sentence Classification in Research Paperscited by 11
- 2021Latent-Optimized Adversarial Neural Transfer for Sarcasm Detectioncited by 6
- 2023When Multi-Task Learning Meets Partial Supervision: A Computer Vision Reviewcited by 6
- 2023Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generationcited by 4
- 2021DeepAnalyze: Learning to Localize Crashes at Scalecited by 2
- 2024Multi-Task Learning for Front-End Text Processing in TTScited by 1
- 2021MetaXT: Meta Cross-Task Transfer between Disparate Label Spacescited by 0
3 more
- 2022Don't Waste Data: Transfer Learning to Leverage All Data for Machine-Learnt Climate Model Emulationcited by 0
- 2024Multi-head Sequence Tagging Model for Grammatical Error Correctioncited by 0
- 2026A Hybrid Modeling Framework for Crop Prediction Tasks via Dynamic Parameter Calibration and Multi-Task Learningcited by 0
- 2026A Hybrid Modeling Framework for Crop Prediction Tasks via Dynamic Parameter Calibration and Multi-Task Learning16 Mar 2026
- 2024Multi-head Sequence Tagging Model for Grammatical Error Correction21 Oct 2024
- 2024Multi-Task Learning for Front-End Text Processing in TTS12 Jan 2024
- 2023When Multi-Task Learning Meets Partial Supervision: A Computer Vision Review25 Jul 2023
- 2023Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generation12 May 2023
- 2022Don't Waste Data: Transfer Learning to Leverage All Data for Machine-Learnt Climate Model Emulation8 Oct 2022
- 2022When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learning17 May 2022
- 2022Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning23 Apr 2022
- 2021DeepAnalyze: Learning to Localize Crashes at Scale29 Sep 2021
- 2021Multi-Task Learning in Natural Language Processing: An Overview19 Sep 2021
- 2021MetaXT: Meta Cross-Task Transfer between Disparate Label Spaces9 Sep 2021
- 2021Specializing Multilingual Language Models: An Empirical Study16 Jun 2021
- 2021Structured Prediction as Translation between Augmented Natural Languagescited by 183
- 2021Multi-Task Learning in Natural Language Processing: An Overviewcited by 61
- 2021Specializing Multilingual Language Models: An Empirical Studycited by 21
- 2022Exploring the Role of Task Transferability in Large-Scale Multi-Task Learningcited by 17
- 2022When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learningcited by 17
- 2021Cross-Domain Multi-Task Learning for Sequential Sentence Classification in Research Paperscited by 11
- 2021Latent-Optimized Adversarial Neural Transfer for Sarcasm Detectioncited by 6
- 2023When Multi-Task Learning Meets Partial Supervision: A Computer Vision Reviewcited by 6
- 2023Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generationcited by 4
- 2021DeepAnalyze: Learning to Localize Crashes at Scalecited by 2
- 2024Multi-Task Learning for Front-End Text Processing in TTScited by 1
- 2021MetaXT: Meta Cross-Task Transfer between Disparate Label Spacescited by 0
12 most cited of 15 All 15 (JSON)
- 2026A Hybrid Modeling Framework for Crop Prediction Tasks via Dynamic Parameter Calibration and Multi-Task Learning16 Mar 2026
- 2024Multi-head Sequence Tagging Model for Grammatical Error Correction21 Oct 2024
- 2024Multi-Task Learning for Front-End Text Processing in TTS12 Jan 2024
- 2023When Multi-Task Learning Meets Partial Supervision: A Computer Vision Review25 Jul 2023
- 2023Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generation12 May 2023
- 2022Don't Waste Data: Transfer Learning to Leverage All Data for Machine-Learnt Climate Model Emulation8 Oct 2022
- 2022When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learning17 May 2022
- 2022Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning23 Apr 2022
- 2021DeepAnalyze: Learning to Localize Crashes at Scale29 Sep 2021
- 2021Multi-Task Learning in Natural Language Processing: An Overview19 Sep 2021
- 2021MetaXT: Meta Cross-Task Transfer between Disparate Label Spaces9 Sep 2021
- 2021Specializing Multilingual Language Models: An Empirical Study16 Jun 2021
12 newest of 15 All 15 (JSON)