mlsanityData to 14 Oct 2026

Federated Transformer: Multi-Party Vertical Federated Learning on Practical Fuzzily Linked Data

arXiv 2410.17986 ↗First posted 23 Oct 2024cs.LG cs.AI cs.CRcited by 8

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is an evolving paradigm that enables multiple parties to collaboratively train models without sharing raw data. Among its variants, Vertical Federated Learning (VFL) is particularly relevant in real-world, cross-organizational collaborations, where distinct features of a shared instance group are contributed by different parties. In these scenarios, parties are often linked using fuzzy identifiers, leading to a common practice termed as multi-party fuzzy VFL. Existing models generally address either multi-party VFL or fuzzy VFL between two parties. Extending these models to practical multi-party fuzzy VFL typically results in significant performance degradation and increased costs for maintaining privacy. To overcome these limitations, we introduce the Federated Transformer (FeT), a novel framework that supports multi-party VFL with fuzzy identifiers. FeT innovatively encodes these identifiers into data representations and employs a transformer architecture distributed across different parties, incorporating three new techniques to enhance performance. Furthermore, we have developed a multi-party privacy framework for VFL that integrates differential privacy with secure multi-party computation, effectively protecting local representations while minimizing associated utility costs. Our experiments demonstrate that the FeT surpasses the baseline models by up to 46% in terms of accuracy when scaled to 50 parties. Additionally, in two-party fuzzy VFL settings, FeT also shows improved performance and privacy over cutting-edge VFL models.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 25 of its 47 references that are in the map. First posted on arXiv 23 Oct 2024.

Builds on

25 of 47 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Deep Learning with Differential Privacycited by 2,728
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  5. 2017Renyi Differential Privacycited by 650
  6. 2018Split learning for health: Distributed deep learning without sharing raw patient datacited by 395
  7. 2019A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncited by 374
  8. 2018Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoisingcited by 218
  9. 2017Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryptioncited by 176
  10. 2020Feature Inference Attack on Model Predictions in Vertical Federated Learningcited by 106
  11. 2021Spatial-Temporal Transformer for Dynamic Scene Graph Generationcited by 96
  12. 2021Learnable Fourier Features for Multi-Dimensional Spatial Positional Encodingcited by 93
13 more
  1. 2020NoPeek: Information leakage reduction to share activations in distributed deep learningcited by 66
  2. 2022ResSFL: A Resistance Transfer Framework for Defending Model Inversion Attack in Split Federated Learningcited by 36
  3. 2022BlindFL: Vertical Federated Machine Learning without Peeking into Your Datacited by 36
  4. 2022Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Datacited by 36
  5. 2020FedCVT: Semi-supervised Vertical Federated Learning with Cross-view Trainingcited by 32
  6. 2023A Survey on Vertical Federated Learning: From a Layered Perspectivecited by 30
  7. 2020Asymmetrical Vertical Federated Learningcited by 26
  8. 2020Hybrid Differentially Private Federated Learning on Vertically Partitioned Datacited by 22
  9. 2022Practical Vertical Federated Learning with Unsupervised Representation Learningcited by 19
  10. 2023Vertical Federated Knowledge Transfer via Representation Distillation for Healthcare Collaboration Networkscited by 19
  11. 2023VertiBench: Advancing Feature Distribution Diversity in Vertical Federated Learning Benchmarkscited by 6
  12. 2021Multi-VFL: A Vertical Federated Learning System for Multiple Data and Label Ownerscited by 5
  13. 2022EFMVFL: An Efficient and Flexible Multi-party Vertical Federated Learning without a Third Partycited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 3.

8 papers.

  1. 2025Vertical Federated Learning in Practice: The Good, the Bad, and the Uglycited by 4
  2. 2025Unlearning Clients, Features and Samples in Vertical Federated Learningcited by 3
  3. 2025Deep Latent Variable Model based Vertical Federated Learning with Flexible Alignment and Labeling Scenarioscited by 0
  4. 2025WikiDBGraph: A Data Management Benchmark Suite for Collaborative Learning over Database Siloscited by 0
  5. 2025SelectiveShield: Lightweight Hybrid Defense Against Gradient Leakage in Federated Learningcited by 0
  6. 2026DeFRiS: Silo-Cooperative IoT Applications Scheduling via Decentralized Federated Reinforcement Learningcited by 0
  7. 2026Secure and Privacy-Preserving Vertical Federated Learningcited by 0
  8. 2026Practical Anonymous Two-Party Gradient Boosting Decision Treecited by 0