# Tackling Multiple Tasks with One Single Learning Framework

arXiv: https://arxiv.org/abs/2206.06322 · HTML: https://mlsanity.com/paper/2206.06322/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 29 May 2022.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Multi-Task Deep Neural Networks for Natural Language Understanding](https://mlsanity.com/paper/1901.11504/) (2019)
- [A Riemannian Network for SPD Matrix Learning](https://mlsanity.com/paper/1608.04233/) (2016)
- [Adversarial Multi-task Learning for Text Classification](https://mlsanity.com/paper/1704.05742/) (2017)
- [Learning Activation Functions to Improve Deep Neural Networks](https://mlsanity.com/paper/1412.6830/) (2014)
- [A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks](https://mlsanity.com/paper/1811.06031/) (2018)
- [An Interactive Multi-Task Learning Network for End-to-End Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/1906.06906/) (2019)
- [Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks](https://mlsanity.com/paper/1908.10423/) (2019)
- [Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction](https://mlsanity.com/paper/1905.08511/) (2019)
- [Fashion Captioning: Towards Generating Accurate Descriptions with Semantic Rewards](https://mlsanity.com/paper/2008.02693/) (2020)
- [ReFormer: The Relational Transformer for Image Captioning](https://mlsanity.com/paper/2107.14178/) (2021)
- [Multi-task Learning for Aggregated Data using Gaussian Processes](https://mlsanity.com/paper/1906.09412/) (2019)
- [Meta Multi-Task Learning for Sequence Modeling](https://mlsanity.com/paper/1802.08969/) (2018)
- [Journalistic Guidelines Aware News Image Captioning](https://mlsanity.com/paper/2109.02865/) (2021)
- [Learning to Multitask](https://mlsanity.com/paper/1805.07541/) (2018)
- [Multi-Task Learning with Multi-View Attention for Answer Selection and Knowledge Base Question Answering](https://mlsanity.com/paper/1812.02354/) (2018)
- [Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data](https://mlsanity.com/paper/1906.08931/) (2019)
- [Multi-Task Networks With Universe, Group, and Task Feature Learning](https://mlsanity.com/paper/1907.01791/) (2019)
- [Different Absorption from the Same Sharing: Sifted Multi-task Learning for Fake News Detection](https://mlsanity.com/paper/1909.01720/) (2019)
- [Crossing-Domain Generative Adversarial Networks for Unsupervised Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/2008.11882/) (2020)
- [Latent Part-of-Speech Sequences for Neural Machine Translation](https://mlsanity.com/paper/1908.11782/) (2019)
- [Learning Tuple Compatibility for Conditional OutfitRecommendation](https://mlsanity.com/paper/2008.08189/) (2020)
- [Recognizing License Plates in Real-Time](https://mlsanity.com/paper/1906.04376/) (2019)
- [GIRNet: Interleaved Multi-Task Recurrent State Sequence Models](https://mlsanity.com/paper/1811.11456/) (2018)
- [Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings](https://mlsanity.com/paper/1809.08928/) (2018)

## Abstract (arXiv)

Deep Multi-Task Learning (DMTL) has been widely studied in the machine learning community and applied to a broad range of real-world applications. Searching for the optimal knowledge sharing in DMTL is more challenging for sequential learning problems, as the task relationship will change in the temporal dimension. In this paper, we propose a flexible and efficient framework called HierarchicalTemporal Activation Network (HTAN) to simultaneously explore the optimal sharing of the neural network hierarchy (hierarchical axis) and the time-variant task relationship (temporal axis). HTAN learns a set of time-variant activation functions to encode the task relation. A functional regularization implemented by a modulated SPDNet and adversarial learning is further proposed to enhance the DMTL performance. Comprehensive experiments on several challenging applications demonstrate that our HTAN-SPD framework outperforms SOTA methods significantly in sequential DMTL.