# Privacy-Preserving Self-Taught Federated Learning for Heterogeneous Data

arXiv: https://arxiv.org/abs/2102.05883 · HTML: https://mlsanity.com/paper/2102.05883/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 11 Feb 2021.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [Federated Adversarial Domain Adaptation](https://mlsanity.com/paper/1911.02054/) (2019)

## Built on by (most cited first, 5 of 5)

- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [A Hybrid Self-Supervised Learning Framework for Vertical Federated Learning](https://mlsanity.com/paper/2208.08934/) (2022)
- [Federated Learning in Non-IID Settings Aided by Differentially Private Synthetic Data](https://mlsanity.com/paper/2206.00686/) (2022)
- [Continual Horizontal Federated Learning for Heterogeneous Data](https://mlsanity.com/paper/2203.02108/) (2022)
- [TabVFL: Improving Latent Representation in Vertical Federated Learning](https://mlsanity.com/paper/2404.17990/) (2024)

## Abstract (arXiv)

Many application scenarios call for training a machine learning model among multiple participants. Federated learning (FL) was proposed to enable joint training of a deep learning model using the local data in each party without revealing the data to others. Among various types of FL methods, vertical FL is a category to handle data sources with the same ID space and different feature spaces. However, existing vertical FL methods suffer from limitations such as restrictive neural network structure, slow training speed, and often lack the ability to take advantage of data with unmatched IDs. In this work, we propose an FL method called self-taught federated learning to address the aforementioned issues, which uses unsupervised feature extraction techniques for distributed supervised deep learning tasks. In this method, only latent variables are transmitted to other parties for model training, while privacy is preserved by storing the data and parameters of activations, weights, and biases locally. Extensive experiments are performed to evaluate and demonstrate the validity and efficiency of the proposed method.