# PCA-based Multi Task Learning: a Random Matrix Approach

arXiv: https://arxiv.org/abs/2111.00924 · HTML: https://mlsanity.com/paper/2111.00924/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 65 references that are in the map.
- First posted on arXiv 1 Nov 2021.

## Builds on (most cited first, 18 of 18)

- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [A Survey on Deep Transfer Learning](https://mlsanity.com/paper/1808.01974/) (2018)
- [Unsupervised Domain Adaptation with Residual Transfer Networks](https://mlsanity.com/paper/1602.04433/) (2016)
- [Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning](https://mlsanity.com/paper/2002.05651/) (2020)
- [Random Matrix Theory Proves that Deep Learning Representations of GAN-data Behave as Gaussian Mixtures](https://mlsanity.com/paper/2001.08370/) (2020)
- [Efficient Learning of Domain-invariant Image Representations](https://mlsanity.com/paper/1301.3224/) (2013)
- [Semi-supervised Multitask Learning for Sequence Labeling](https://mlsanity.com/paper/1704.07156/) (2017)
- [Asymptotic Bayes risk for Gaussian mixture in a semi-supervised setting](https://mlsanity.com/paper/1907.03792/) (2019)
- [Unsupervised and Supervised Principal Component Analysis: Tutorial](https://mlsanity.com/paper/1906.03148/) (2019)
- [Large Dimensional Analysis and Improvement of Multi Task Learning](https://mlsanity.com/paper/2009.01591/) (2020)
- [Two-way kernel matrix puncturing: towards resource-efficient PCA and spectral clustering](https://mlsanity.com/paper/2102.12293/) (2021)
- [Safe Screening for Multi-Task Feature Learning with Multiple Data Matrices](https://mlsanity.com/paper/1505.04073/) (2015)
- [Supervised Principal Component Regression for Functional Responses with High Dimensional Predictors](https://mlsanity.com/paper/2103.11567/) (2021)

## Built on by (most cited first, 4 of 4)

- [Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting](https://mlsanity.com/paper/2406.10327/) (2024)
- [Random Matrix Analysis to Balance between Supervised and Unsupervised Learning under the Low Density Separation Assumption](https://mlsanity.com/paper/2310.13434/) (2023)
- [Asymptotic Bayes risk of semi-supervised multitask learning on Gaussian mixture](https://mlsanity.com/paper/2303.02048/) (2023)
- [Incorporating priors in learning: a random matrix study under a teacher-student framework](https://mlsanity.com/paper/2509.22124/) (2025)

## Abstract (arXiv)

The article proposes and theoretically analyses a computationally efficient multi-task learning (MTL) extension of popular principal component analysis (PCA)-based supervised learning schemes citebarshan2011supervised,bair2006prediction. The analysis reveals that (i) by default learning may dramatically fail by suffering from negative transfer, but that (ii) simple counter-measures on data labels avert negative transfer and necessarily result in improved performances. Supporting experiments on synthetic and real data benchmarks show that the proposed method achieves comparable performance with state-of-the-art MTL methods but at a significantly reduced computational cost.