# Continual Horizontal Federated Learning for Heterogeneous Data

arXiv: https://arxiv.org/abs/2203.02108 · HTML: https://mlsanity.com/paper/2203.02108/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 46 references that are in the map.
- First posted on arXiv 4 Mar 2022.

## Builds on (most cited first, 27 of 27)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints](https://mlsanity.com/paper/1910.01991/) (2019)
- [Adaptive Personalized Federated Learning](https://mlsanity.com/paper/2003.13461/) (2020)
- [Personalized Federated Learning with First Order Model Optimization](https://mlsanity.com/paper/2012.08565/) (2020)
- [Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption](https://mlsanity.com/paper/1711.10677/) (2017)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [Federated Continual Learning with Weighted Inter-client Transfer](https://mlsanity.com/paper/2003.03196/) (2020)
- [Privacy Preserving Vertical Federated Learning for Tree-based Models](https://mlsanity.com/paper/2008.06170/) (2020)
- [Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator](https://mlsanity.com/paper/1911.09824/) (2019)
- [Federated Forest](https://mlsanity.com/paper/1905.10053/) (2019)
- [A distillation-based approach integrating continual learning and federated learning for pervasive services](https://mlsanity.com/paper/2109.04197/) (2021)
- [Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating](https://mlsanity.com/paper/2103.00958/) (2021)
- [Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data](https://mlsanity.com/paper/2008.06197/) (2020)
- [PyVertical: A Vertical Federated Learning Framework for Multi-headed SplitNN](https://mlsanity.com/paper/2104.00489/) (2021)
- [Continual Local Training for Better Initialization of Federated Models](https://mlsanity.com/paper/2005.12657/) (2020)
- [FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data](https://mlsanity.com/paper/2103.03918/) (2021)
- [FederBoost: Private Federated Learning for GBDT](https://mlsanity.com/paper/2011.02796/) (2020)
- [Entity Resolution and Federated Learning get a Federated Resolution](https://mlsanity.com/paper/1803.04035/) (2018)
- [Vertical Federated Learning without Revealing Intersection Membership](https://mlsanity.com/paper/2106.05508/) (2021)
- [Federated Reconnaissance: Efficient, Distributed, Class-Incremental Learning](https://mlsanity.com/paper/2109.00150/) (2021)
- [Privacy-Preserving Self-Taught Federated Learning for Heterogeneous Data](https://mlsanity.com/paper/2102.05883/) (2021)
- [Multi-Party Dual Learning](https://mlsanity.com/paper/2104.06677/) (2021)

## Built on by (most cited first, 8 of 8)

- [Federated Continual Learning via Knowledge Fusion: A Survey](https://mlsanity.com/paper/2312.16475/) (2023)
- [Federated Continual Learning: Concepts, Challenges, and Solutions](https://mlsanity.com/paper/2502.07059/) (2025)
- [Application of federated learning techniques for arrhythmia classification using 12-lead ECG signals](https://mlsanity.com/paper/2208.10993/) (2022)
- [Personalised Federated Learning On Heterogeneous Feature Spaces](https://mlsanity.com/paper/2301.11447/) (2023)
- [A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques](https://mlsanity.com/paper/2302.08571/) (2023)
- [GraphFedMIG: Tackling Class Imbalance in Federated Graph Learning via Mutual Information-Guided Generation](https://mlsanity.com/paper/2508.10471/) (2025)
- [Survey of Privacy Threats and Countermeasures in Federated Learning](https://mlsanity.com/paper/2402.00342/) (2024)
- [GeFL: Model-Agnostic Federated Learning with Generative Models](https://mlsanity.com/paper/2412.18460/) (2024)

## Abstract (arXiv)

Federated learning is a promising machine learning technique that enables multiple clients to collaboratively build a model without revealing the raw data to each other. Among various types of federated learning methods, horizontal federated learning (HFL) is the best-studied category and handles homogeneous feature spaces. However, in the case of heterogeneous feature spaces, HFL uses only common features and leaves client-specific features unutilized. In this paper, we propose a HFL method using neural networks named continual horizontal federated learning (CHFL), a continual learning approach to improve the performance of HFL by taking advantage of unique features of each client. CHFL splits the network into two columns corresponding to common features and unique features, respectively. It jointly trains the first column by using common features through vanilla HFL and locally trains the second column by using unique features and leveraging the knowledge of the first one via lateral connections without interfering with the federated training of it. We conduct experiments on various real world datasets and show that CHFL greatly outperforms vanilla HFL that only uses common features and local learning that uses all features that each client has.