# When is multitask learning effective? Semantic sequence prediction under varying data conditions

arXiv: https://arxiv.org/abs/1612.02251 · HTML: https://mlsanity.com/paper/1612.02251/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2016.

## Built on by (most cited first, 21 of 21)

- [SLAPS: Self-Supervision Improves Structure Learning for Graph Neural Networks](https://mlsanity.com/paper/2102.05034/) (2021)
- [Multi-Task Learning in Natural Language Processing: An Overview](https://mlsanity.com/paper/2109.09138/) (2021)
- [PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval](https://mlsanity.com/paper/2108.06027/) (2021)
- [π-Tuning: Transferring Multimodal Foundation Models with Optimal Multi-task Interpolation](https://mlsanity.com/paper/2304.14381/) (2023)
- [GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learning](https://mlsanity.com/paper/2207.09858/) (2022)
- [De-identification of Privacy-related Entities in Job Postings](https://mlsanity.com/paper/2105.11223/) (2021)
- [Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning](https://mlsanity.com/paper/2310.10318/) (2023)
- [Multi-Task Meta Learning: learn how to adapt to unseen tasks](https://mlsanity.com/paper/2210.06989/) (2022)
- [Active2 Learning: Actively reducing redundancies in Active Learning methods for Sequence Tagging and Machine Translation](https://mlsanity.com/paper/2103.06490/) (2021)
- [Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection](https://mlsanity.com/paper/2104.09261/) (2021)
- [A Flexible Multi-Task Model for BERT Serving](https://mlsanity.com/paper/2107.05377/) (2021)
- [Generative multitask learning mitigates target-causing confounding](https://mlsanity.com/paper/2202.04136/) (2022)
- [Identifying Task Groupings for Multi-Task Learning Using Pointwise V-Usable Information](https://mlsanity.com/paper/2410.12774/) (2024)
- [One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets](https://mlsanity.com/paper/2106.04476/) (2021)
- [SLAW: Scaled Loss Approximate Weighting for Efficient Multi-Task Learning](https://mlsanity.com/paper/2109.08218/) (2021)
- [Polarity and Subjectivity Detection with Multitask Learning and BERT Embedding](https://mlsanity.com/paper/2201.05363/) (2022)
- [CONTRASTE: Supervised Contrastive Pre-training With Aspect-based Prompts For Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2310.15577/) (2023)
- [Improving Dialectal Slot and Intent Detection with Auxiliary Tasks: A Multi-Dialectal Bavarian Case Study](https://mlsanity.com/paper/2501.03863/) (2025)
- [On Steering Multi-Annotations per Sample for Multi-Task Learning](https://mlsanity.com/paper/2203.02946/) (2022)
- [Label Agnostic Pre-training for Zero-shot Text Classification](https://mlsanity.com/paper/2305.16521/) (2023)
- [On the Limits of Multi-modal Meta-Learning with Auxiliary Task Modulation Using Conditional Batch Normalization](https://mlsanity.com/paper/2405.18751/) (2024)

## Abstract (arXiv)

Multitask learning has been applied successfully to a range of tasks, mostly morphosyntactic. However, little is known on when MTL works and whether there are data characteristics that help to determine its success. In this paper we evaluate a range of semantic sequence labeling tasks in a MTL setup. We examine different auxiliary tasks, amongst which a novel setup, and correlate their impact to data-dependent conditions. Our results show that MTL is not always effective, significant improvements are obtained only for 1 out of 5 tasks. When successful, auxiliary tasks with compact and more uniform label distributions are preferable.