# Meta Multi-Task Learning for Sequence Modeling

arXiv: https://arxiv.org/abs/1802.08969 · HTML: https://mlsanity.com/paper/1802.08969/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2018.

## Built on by (most cited first, 18 of 18)

- [Multi-Task Learning in Natural Language Processing: An Overview](https://mlsanity.com/paper/2109.09138/) (2021)
- [Towards Spatio-Temporal Aware Traffic Time Series Forecasting--Full Version](https://mlsanity.com/paper/2203.15737/) (2022)
- [ToAlign: Task-oriented Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2106.10812/) (2021)
- [Heterogeneity-Informed Meta-Parameter Learning for Spatiotemporal Time Series Forecasting](https://mlsanity.com/paper/2405.10800/) (2024)
- [Meta-Learning for Effective Multi-task and Multilingual Modelling](https://mlsanity.com/paper/2101.10368/) (2021)
- [Learning Graph Meta Embeddings for Cold-Start Ads in Click-Through Rate Prediction](https://mlsanity.com/paper/2105.08909/) (2021)
- [Diverse Distributions of Self-Supervised Tasks for Meta-Learning in NLP](https://mlsanity.com/paper/2111.01322/) (2021)
- [Multi-Task Time Series Forecasting With Shared Attention](https://mlsanity.com/paper/2101.09645/) (2021)
- [Enhance Long Text Understanding via Distilled Gist Detector from Abstractive Summarization](https://mlsanity.com/paper/2110.04741/) (2021)
- [Sharing to learn and learning to share; Fitting together Meta-Learning, Multi-Task Learning, and Transfer Learning: A meta review](https://mlsanity.com/paper/2111.12146/) (2021)
- [MetaTPTrans: A Meta Learning Approach for Multilingual Code Representation Learning](https://mlsanity.com/paper/2206.06460/) (2022)
- [Improving Long Text Understanding with Knowledge Distilled from Summarization Model](https://mlsanity.com/paper/2405.04955/) (2024)
- [A Speaker Turn-Aware Multi-Task Adversarial Network for Joint User Satisfaction Estimation and Sentiment Analysis](https://mlsanity.com/paper/2410.09556/) (2024)
- [DIGMN: Dynamic Intent Guided Meta Network for Differentiated User Engagement Forecasting in Online Professional Social Platforms](https://mlsanity.com/paper/2210.12402/) (2022)
- [Transition-Aware Multi-Activity Knowledge Tracing](https://mlsanity.com/paper/2301.12916/) (2023)
- [Unsupervised Sentiment Analysis by Transferring Multi-source Knowledge](https://mlsanity.com/paper/2105.11902/) (2021)
- [MetaXT: Meta Cross-Task Transfer between Disparate Label Spaces](https://mlsanity.com/paper/2109.04240/) (2021)
- [Tackling Multiple Tasks with One Single Learning Framework](https://mlsanity.com/paper/2206.06322/) (2022)

## Abstract (arXiv)

Semantic composition functions have been playing a pivotal role in neural representation learning of text sequences. In spite of their success, most existing models suffer from the underfitting problem: they use the same shared compositional function on all the positions in the sequence, thereby lacking expressive power due to incapacity to capture the richness of compositionality. Besides, the composition functions of different tasks are independent and learned from scratch. In this paper, we propose a new sharing scheme of composition function across multiple tasks. Specifically, we use a shared meta-network to capture the meta-knowledge of semantic composition and generate the parameters of the task-specific semantic composition models. We conduct extensive experiments on two types of tasks, text classification and sequence tagging, which demonstrate the benefits of our approach. Besides, we show that the shared meta-knowledge learned by our proposed model can be regarded as off-the-shelf knowledge and easily transferred to new tasks.