# Multi-Task Variational Information Bottleneck

arXiv: https://arxiv.org/abs/2007.00339 · HTML: https://mlsanity.com/paper/2007.00339/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jul 2020.

## Built on by (most cited first, 7 of 7)

- [Variational Multi-Task Learning with Gumbel-Softmax Priors](https://mlsanity.com/paper/2111.05323/) (2021)
- [Episodic Multi-Task Learning with Heterogeneous Neural Processes](https://mlsanity.com/paper/2310.18713/) (2023)
- [Multi-Task Neural Processes](https://mlsanity.com/paper/2111.05820/) (2021)
- [Compressed Hierarchical Representations for Multi-Task Learning and Task Clustering](https://mlsanity.com/paper/2205.15882/) (2022)
- [An Information-theoretic Multi-task Representation Learning Framework for Natural Language Understanding](https://mlsanity.com/paper/2503.04667/) (2025)
- [Geometric and Information Compression of Representations in Deep Learning](https://mlsanity.com/paper/2606.21593/) (2026)
- [Capacity and Redundancy Trade-offs in Multi-Task Learning](https://mlsanity.com/paper/2607.16554/) (2026)

## Abstract (arXiv)

Multi-task learning (MTL) is an important subject in machine learning and artificial intelligence. Its applications to computer vision, signal processing, and speech recognition are ubiquitous. Although this subject has attracted considerable attention recently, the performance and robustness of the existing models to different tasks have not been well balanced. This article proposes an MTL model based on the architecture of the variational information bottleneck (VIB), which can provide a more effective latent representation of the input features for the downstream tasks. Extensive observations on three public data sets under adversarial attacks show that the proposed model is competitive to the state-of-the-art algorithms concerning the prediction accuracy. Experimental results suggest that combining the VIB and the task-dependent uncertainties is a very effective way to abstract valid information from the input features for accomplishing multiple tasks.