# Federated Transformer: Multi-Party Vertical Federated Learning on Practical Fuzzily Linked Data

arXiv: https://arxiv.org/abs/2410.17986 · HTML: https://mlsanity.com/paper/2410.17986/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 23 Oct 2024.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection](https://mlsanity.com/paper/1907.09693/) (2019)
- [Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising](https://mlsanity.com/paper/1805.06530/) (2018)
- [Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption](https://mlsanity.com/paper/1711.10677/) (2017)
- [Feature Inference Attack on Model Predictions in Vertical Federated Learning](https://mlsanity.com/paper/2010.10152/) (2020)
- [Spatial-Temporal Transformer for Dynamic Scene Graph Generation](https://mlsanity.com/paper/2107.12309/) (2021)
- [Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding](https://mlsanity.com/paper/2106.02795/) (2021)
- [NoPeek: Information leakage reduction to share activations in distributed deep learning](https://mlsanity.com/paper/2008.09161/) (2020)
- [ResSFL: A Resistance Transfer Framework for Defending Model Inversion Attack in Split Federated Learning](https://mlsanity.com/paper/2205.04007/) (2022)
- [BlindFL: Vertical Federated Machine Learning without Peeking into Your Data](https://mlsanity.com/paper/2206.07975/) (2022)
- [Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data](https://mlsanity.com/paper/2206.08330/) (2022)
- [FedCVT: Semi-supervised Vertical Federated Learning with Cross-view Training](https://mlsanity.com/paper/2008.10838/) (2020)
- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [Asymmetrical Vertical Federated Learning](https://mlsanity.com/paper/2004.07427/) (2020)
- [Hybrid Differentially Private Federated Learning on Vertically Partitioned Data](https://mlsanity.com/paper/2009.02763/) (2020)
- [Practical Vertical Federated Learning with Unsupervised Representation Learning](https://mlsanity.com/paper/2208.10278/) (2022)
- [Vertical Federated Knowledge Transfer via Representation Distillation for Healthcare Collaboration Networks](https://mlsanity.com/paper/2302.05675/) (2023)
- [VertiBench: Advancing Feature Distribution Diversity in Vertical Federated Learning Benchmarks](https://mlsanity.com/paper/2307.02040/) (2023)
- [Multi-VFL: A Vertical Federated Learning System for Multiple Data and Label Owners](https://mlsanity.com/paper/2106.05468/) (2021)
- [EFMVFL: An Efficient and Flexible Multi-party Vertical Federated Learning without a Third Party](https://mlsanity.com/paper/2201.06244/) (2022)

## Built on by (most cited first, 8 of 8)

- [Vertical Federated Learning in Practice: The Good, the Bad, and the Ugly](https://mlsanity.com/paper/2502.08160/) (2025)
- [Unlearning Clients, Features and Samples in Vertical Federated Learning](https://mlsanity.com/paper/2501.13683/) (2025)
- [Deep Latent Variable Model based Vertical Federated Learning with Flexible Alignment and Labeling Scenarios](https://mlsanity.com/paper/2505.11035/) (2025)
- [WikiDBGraph: A Data Management Benchmark Suite for Collaborative Learning over Database Silos](https://mlsanity.com/paper/2505.16635/) (2025)
- [SelectiveShield: Lightweight Hybrid Defense Against Gradient Leakage in Federated Learning](https://mlsanity.com/paper/2508.04265/) (2025)
- [DeFRiS: Silo-Cooperative IoT Applications Scheduling via Decentralized Federated Reinforcement Learning](https://mlsanity.com/paper/2603.14729/) (2026)
- [Secure and Privacy-Preserving Vertical Federated Learning](https://mlsanity.com/paper/2604.13474/) (2026)
- [Practical Anonymous Two-Party Gradient Boosting Decision Tree](https://mlsanity.com/paper/2605.26903/) (2026)

## Abstract (arXiv)

Federated Learning (FL) is an evolving paradigm that enables multiple parties to collaboratively train models without sharing raw data. Among its variants, Vertical Federated Learning (VFL) is particularly relevant in real-world, cross-organizational collaborations, where distinct features of a shared instance group are contributed by different parties. In these scenarios, parties are often linked using fuzzy identifiers, leading to a common practice termed as multi-party fuzzy VFL. Existing models generally address either multi-party VFL or fuzzy VFL between two parties. Extending these models to practical multi-party fuzzy VFL typically results in significant performance degradation and increased costs for maintaining privacy. To overcome these limitations, we introduce the Federated Transformer (FeT), a novel framework that supports multi-party VFL with fuzzy identifiers. FeT innovatively encodes these identifiers into data representations and employs a transformer architecture distributed across different parties, incorporating three new techniques to enhance performance. Furthermore, we have developed a multi-party privacy framework for VFL that integrates differential privacy with secure multi-party computation, effectively protecting local representations while minimizing associated utility costs. Our experiments demonstrate that the FeT surpasses the baseline models by up to 46% in terms of accuracy when scaled to 50 parties. Additionally, in two-party fuzzy VFL settings, FeT also shows improved performance and privacy over cutting-edge VFL models.