# Adaptive Activation Network and Functional Regularization for Efficient and Flexible Deep Multi-Task Learning

arXiv: https://arxiv.org/abs/1911.08065 · HTML: https://mlsanity.com/paper/1911.08065/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2019.

## Built on by (most cited first, 7 of 7)

- [AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning](https://mlsanity.com/paper/2211.15055/) (2022)
- [ReFormer: The Relational Transformer for Image Captioning](https://mlsanity.com/paper/2107.14178/) (2021)
- [Learning Multi-Pattern Normalities in the Frequency Domain for Efficient Time Series Anomaly Detection](https://mlsanity.com/paper/2311.16191/) (2023)
- [Controllable Image Captioning](https://mlsanity.com/paper/2204.13324/) (2022)
- [Adaptive Activation Network For Low Resource Multilingual Speech Recognition](https://mlsanity.com/paper/2205.14326/) (2022)
- [Visual Transformer for Object Detection](https://mlsanity.com/paper/2206.06323/) (2022)
- [A Masked Bounding-Box Selection Based ResNet Predictor for Text Rotation Prediction](https://mlsanity.com/paper/2209.09198/) (2022)

## Abstract (arXiv)

Multi-task learning (MTL) is a common paradigm that seeks to improve the generalization performance of task learning by training related tasks simultaneously. However, it is still a challenging problem to search the flexible and accurate architecture that can be shared among multiple tasks. In this paper, we propose a novel deep learning model called Task Adaptive Activation Network (TAAN) that can automatically learn the optimal network architecture for MTL. The main principle of TAAN is to derive flexible activation functions for different tasks from the data with other parameters of the network fully shared. We further propose two functional regularization methods that improve the MTL performance of TAAN. The improved performance of both TAAN and the regularization methods is demonstrated by comprehensive experiments.