mlsanityData to 7 Oct 2026

An Information-Theoretic Analysis of The Cost of Decentralization for Learning and Inference Under Privacy Constraints

arXiv 2110.05014 ↗First posted 11 Oct 2021cs.IT cs.CR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In vertical federated learning (FL), the features of a data sample are distributed across multiple agents. As such, inter-agent collaboration can be beneficial not only during the learning phase, as is the case for standard horizontal FL, but also during the inference phase. A fundamental theoretical question in this setting is how to quantify the cost, or performance loss, of decentralization for learning and/or inference. In this paper, we consider general supervised learning problems with any number of agents, and provide a novel information-theoretic quantification of the cost of decentralization in the presence of privacy constraints on inter-agent communication within a Bayesian framework. The cost of decentralization for learning and/or inference is shown to be quantified in terms of conditional mutual information terms involving features and label variables.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 18 references that are in the map. First posted on arXiv 11 Oct 2021.

Builds on

13 of 18 references.

References · 13

  1. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  2. 2018MINE: Mutual Information Neural Estimationcited by 680
  3. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 391
  4. 2019SecureBoost: A Lossless Federated Learning Frameworkcited by 174
  5. 2020Feature Inference Attack on Model Predictions in Vertical Federated Learningcited by 105
  6. 2016Differential Privacy as a Mutual Information Constraintcited by 41
  7. 2020Minimum Excess Risk in Bayesian Learningcited by 35
  8. 2021PyVertical: A Vertical Federated Learning Framework for Multi-headed SplitNNcited by 35
  9. 2020Information-Theoretic Bounds on the Generalization Error and Privacy Leakage in Federated Learningcited by 20
  10. 2020Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learningcited by 19
  11. 2020A Vertical Federated Learning Method for Interpretable Scorecard and Its Application in Credit Scoringcited by 16
  12. 2021Rate-Distortion Analysis of Minimum Excess Risk in Bayesian Learningcited by 10
1 more
  1. 2019Federated Learning for Coalition Operationscited by 2

Built on by

Not cited yet.