# Multi-Tier Federated Learning for Vertically Partitioned Data

arXiv: https://arxiv.org/abs/2102.03620 · HTML: https://mlsanity.com/paper/2102.03620/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 18 references that are in the map.
- First posted on arXiv 6 Feb 2021.

## Builds on (most cited first, 14 of 14)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Local SGD Converges Fast and Communicates Little](https://mlsanity.com/paper/1805.09767/) (2018)
- [Federated Optimization: Distributed Machine Learning for On-Device Intelligence](https://mlsanity.com/paper/1610.02527/) (2016)
- [Client-Edge-Cloud Hierarchical Federated Learning](https://mlsanity.com/paper/1905.06641/) (2019)
- [Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption](https://mlsanity.com/paper/1711.10677/) (2017)
- [Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms](https://mlsanity.com/paper/1808.07576/) (2018)
- [Hierarchical Federated Learning Across Heterogeneous Cellular Networks](https://mlsanity.com/paper/1909.02362/) (2019)
- [VAFL: a Method of Vertical Asynchronous Federated Learning](https://mlsanity.com/paper/2007.06081/) (2020)
- [Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator](https://mlsanity.com/paper/1911.09824/) (2019)
- [Multi-Participant Multi-Class Vertical Federated Learning](https://mlsanity.com/paper/2001.11154/) (2020)
- [A Communication Efficient Collaborative Learning Framework for Distributed Features](https://mlsanity.com/paper/1912.11187/) (2019)
- [Multi-Level Local SGD for Heterogeneous Hierarchical Networks](https://mlsanity.com/paper/2007.13819/) (2020)

## Built on by (most cited first, 7 of 7)

- [Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data](https://mlsanity.com/paper/2206.08330/) (2022)
- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [ADI: Adversarial Dominating Inputs in Vertical Federated Learning Systems](https://mlsanity.com/paper/2201.02775/) (2022)
- [Cross-Silo Federated Learning for Multi-Tier Networks with Vertical and Horizontal Data Partitioning](https://mlsanity.com/paper/2108.08930/) (2021)
- [Communication-Efficient Hybrid Federated Learning for E-health with Horizontal and Vertical Data Partitioning](https://mlsanity.com/paper/2404.10110/) (2024)
- [A Vertical Federated Learning Framework for Horizontally Partitioned Labels](https://mlsanity.com/paper/2106.10056/) (2021)
- [Federated and Transfer Learning for Cancer Detection Based on Image Analysis](https://mlsanity.com/paper/2405.20126/) (2024)

## Abstract (arXiv)

We consider decentralized model training in tiered communication networks. Our network model consists of a set of silos, each holding a vertical partition of the data. Each silo contains a hub and a set of clients, with the silo's vertical data shard partitioned horizontally across its clients. We propose Tiered Decentralized Coordinate Descent (TDCD), a communication-efficient decentralized training algorithm for such two-tiered networks. To reduce communication overhead, the clients in each silo perform multiple local gradient steps before sharing updates with their hub. Each hub adjusts its coordinates by averaging its workers' updates, and then hubs exchange intermediate updates with one another. We present a theoretical analysis of our algorithm and show the dependence of the convergence rate on the number of vertical partitions, the number of local updates, and the number of clients in each hub. We further validate our approach empirically via simulation-based experiments using a variety of datasets and both convex and non-convex objectives.