# Vertical Federated Knowledge Transfer via Representation Distillation for Healthcare Collaboration Networks

arXiv: https://arxiv.org/abs/2302.05675 · HTML: https://mlsanity.com/paper/2302.05675/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 64 references that are in the map.
- First posted on arXiv 11 Feb 2023.

## Builds on (most cited first, 24 of 24)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Federated Machine Learning: Concept and Applications](https://mlsanity.com/paper/1902.04885/) (2019)
- [Applied Federated Learning: Improving Google Keyboard Query Suggestions](https://mlsanity.com/paper/1812.02903/) (2018)
- [FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare](https://mlsanity.com/paper/1907.09173/) (2019)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [Vertical Federated Learning: Concepts, Advances and Challenges](https://mlsanity.com/paper/2211.12814/) (2022)
- [Privacy Preserving Vertical Federated Learning for Tree-based Models](https://mlsanity.com/paper/2008.06170/) (2020)
- [Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach](https://mlsanity.com/paper/2101.07714/) (2021)
- [FedHome: Cloud-Edge based Personalized Federated Learning for In-Home Health Monitoring](https://mlsanity.com/paper/2012.07450/) (2020)
- [FLOP: Federated Learning on Medical Datasets using Partial Networks](https://mlsanity.com/paper/2102.05218/) (2021)
- [BlindFL: Vertical Federated Machine Learning without Peeking into Your Data](https://mlsanity.com/paper/2206.07975/) (2022)
- [FedCVT: Semi-supervised Vertical Federated Learning with Cross-view Training](https://mlsanity.com/paper/2008.10838/) (2020)
- [An Accuracy-Lossless Perturbation Method for Defending Privacy Attacks in Federated Learning](https://mlsanity.com/paper/2002.09843/) (2020)
- [PFA: Privacy-preserving Federated Adaptation for Effective Model Personalization](https://mlsanity.com/paper/2103.01548/) (2021)
- [Practical Lossless Federated Singular Vector Decomposition over Billion-Scale Data](https://mlsanity.com/paper/2105.08925/) (2021)
- [Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates](https://mlsanity.com/paper/2207.14628/) (2022)
- [A Coupled Design of Exploiting Record Similarity for Practical Vertical Federated Learning](https://mlsanity.com/paper/2106.06312/) (2021)
- [Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Data](https://mlsanity.com/paper/2203.01752/) (2022)
- [Demystifying Swarm Learning: A New Paradigm of Blockchain-based Decentralized Federated Learning](https://mlsanity.com/paper/2201.05286/) (2022)
- [Bursts of Activity: Temporal Patterns of Help-Seeking and Support in Online Mental Health Forums](https://mlsanity.com/paper/2004.10330/) (2020)

## Built on by (most cited first, 19 of 19)

- [Federated Distillation: A Survey](https://mlsanity.com/paper/2404.08564/) (2024)
- [Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey](https://mlsanity.com/paper/2405.17495/) (2024)
- [Knowledge Distillation in Federated Learning: a Survey on Long Lasting Challenges and New Solutions](https://mlsanity.com/paper/2406.10861/) (2024)
- [Federated Transformer: Multi-Party Vertical Federated Learning on Practical Fuzzily Linked Data](https://mlsanity.com/paper/2410.17986/) (2024)
- [Vertical Federated Learning with Missing Features During Training and Inference](https://mlsanity.com/paper/2410.22564/) (2024)
- [A Survey on Contribution Evaluation in Vertical Federated Learning](https://mlsanity.com/paper/2405.02364/) (2024)
- [UniTrans: A Unified Vertical Federated Knowledge Transfer Framework for Enhancing Cross-Hospital Collaboration](https://mlsanity.com/paper/2501.11388/) (2025)
- [Federated and Transfer Learning for Cancer Detection Based on Image Analysis](https://mlsanity.com/paper/2405.20126/) (2024)
- [Towards Active Participant Centric Vertical Federated Learning: Some Representations May Be All You Need](https://mlsanity.com/paper/2410.17648/) (2024)
- [MIND: Modality-Informed Knowledge Distillation Framework for Multimodal Clinical Prediction Tasks](https://mlsanity.com/paper/2502.01158/) (2025)
- [Is the Trigger Essential? A Feature-Based Triggerless Backdoor Attack in Vertical Federated Learning](https://mlsanity.com/paper/2602.20593/) (2026)
- [Deep Latent Variable Model based Vertical Federated Learning with Flexible Alignment and Labeling Scenarios](https://mlsanity.com/paper/2505.11035/) (2025)
- [On the Inference (In-)Security of Vertical Federated Learning: Efficient Auditing against Inference Tampering Attack](https://mlsanity.com/paper/2507.02376/) (2025)
- [Proto-EVFL: Enhanced Vertical Federated Learning via Dual Prototype with Extremely Unaligned Data](https://mlsanity.com/paper/2507.22488/) (2025)
- [X-VFL: A New Vertical Federated Learning Framework with Cross Completion and Decision Subspace Alignment](https://mlsanity.com/paper/2508.05568/) (2025)
- [PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture](https://mlsanity.com/paper/2510.12494/) (2025)
- [Mixture of Predefined Experts: Maximizing Data Usage on Vertical Federated Learning](https://mlsanity.com/paper/2602.12708/) (2026)
- [Autonomous Collaborative Learning Among an Ensemble of Tsetlin Machines with Consensus-Based Inference](https://mlsanity.com/paper/2607.20124/) (2026)
- [Understanding Backdoor Vulnerabilities in Vertical Federated Learning: The Gap Between Research and Practice](https://mlsanity.com/paper/2608.12962/) (2026)

## Abstract (arXiv)

Collaboration between healthcare institutions can significantly lessen the imbalance in medical resources across various geographic areas. However, directly sharing diagnostic information between institutions is typically not permitted due to the protection of patients' highly sensitive privacy. As a novel privacy-preserving machine learning paradigm, federated learning (FL) makes it possible to maximize the data utility among multiple medical institutions. These feature-enrichment FL techniques are referred to as vertical FL (VFL). Traditional VFL can only benefit multi-parties' shared samples, which strongly restricts its application scope. In order to improve the information-sharing capability and innovation of various healthcare-related institutions, and then to establish a next-generation open medical collaboration network, we propose a unified framework for vertical federated knowledge transfer mechanism (VFedTrans) based on a novel cross-hospital representation distillation component. Specifically, our framework includes three steps. First, shared samples' federated representations are extracted by collaboratively modeling multi-parties' joint features with current efficient vertical federated representation learning methods. Second, for each hospital, we learn a local-representation-distilled module, which can transfer the knowledge from shared samples' federated representations to enrich local samples' representations. Finally, each hospital can leverage local samples' representations enriched by the distillation module to boost arbitrary downstream machine learning tasks. The experiments on real-life medical datasets verify the knowledge transfer effectiveness of our framework.