# PackVFL: Efficient HE Packing for Vertical Federated Learning

arXiv: https://arxiv.org/abs/2405.00482 · HTML: https://mlsanity.com/paper/2405.00482/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 77 references that are in the map.
- First posted on arXiv 1 May 2024.

## Builds on (most cited first, 29 of 29)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Certified Robustness to Adversarial Examples with Differential Privacy](https://mlsanity.com/paper/1802.03471/) (2018)
- [Split learning for health: Distributed deep learning without sharing raw patient data](https://mlsanity.com/paper/1812.00564/) (2018)
- [Train longer, generalize better: closing the generalization gap in large batch training of neural networks](https://mlsanity.com/paper/1705.08741/) (2017)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [SecureBoost: A Lossless Federated Learning Framework](https://mlsanity.com/paper/1901.08755/) (2019)
- [A Survey on Homomorphic Encryption Schemes: Theory and Implementation](https://mlsanity.com/paper/1704.03578/) (2017)
- [TenSEAL: A Library for Encrypted Tensor Operations Using Homomorphic Encryption](https://mlsanity.com/paper/2104.03152/) (2021)
- [A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression](https://mlsanity.com/paper/1912.00513/) (2019)
- [BlindFL: Vertical Federated Machine Learning without Peeking into Your Data](https://mlsanity.com/paper/2206.07975/) (2022)
- [When Homomorphic Encryption Marries Secret Sharing: Secure Large-Scale Sparse Logistic Regression and Applications in Risk Control](https://mlsanity.com/paper/2008.08753/) (2020)
- [A Survey on Vertical Federated Learning: From a Layered Perspective](https://mlsanity.com/paper/2304.01829/) (2023)
- [Extrapolation for Large-batch Training in Deep Learning](https://mlsanity.com/paper/2006.05720/) (2020)
- [F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption (Extended Version)](https://mlsanity.com/paper/2109.05371/) (2021)
- [EVA: An Encrypted Vector Arithmetic Language and Compiler for Efficient Homomorphic Computation](https://mlsanity.com/paper/1912.11951/) (2019)
- [Additively Homomorphical Encryption based Deep Neural Network for Asymmetrically Collaborative Machine Learning](https://mlsanity.com/paper/2007.06849/) (2020)
- [FLASHE: Additively Symmetric Homomorphic Encryption for Cross-Silo Federated Learning](https://mlsanity.com/paper/2109.00675/) (2021)
- [GALA: Greedy ComputAtion for Linear Algebra in Privacy-Preserved Neural Networks](https://mlsanity.com/paper/2105.01827/) (2021)
- [Achieving Differential Privacy in Vertically Partitioned Multiparty Learning](https://mlsanity.com/paper/1911.04587/) (2019)
- [Secure Forward Aggregation for Vertical Federated Neural Networks](https://mlsanity.com/paper/2207.00165/) (2022)
- [Artificial intelligence across company borders](https://mlsanity.com/paper/2107.03912/) (2021)
- [EFMVFL: An Efficient and Flexible Multi-party Vertical Federated Learning without a Third Party](https://mlsanity.com/paper/2201.06244/) (2022)
- [XFL: A High Performace, Lightweighted Federated Learning Framework](https://mlsanity.com/paper/2302.05076/) (2023)

## Built on by (most cited first, 1 of 1)

- [Private LoRA Fine-tuning of Open-Source LLMs with Homomorphic Encryption](https://mlsanity.com/paper/2505.07329/) (2025)

## Abstract (arXiv)

As an essential tool of secure distributed machine learning, vertical federated learning (VFL) based on homomorphic encryption (HE) suffers from severe efficiency problems due to data inflation and time-consuming operations. To this core, we propose PackVFL, an efficient VFL framework based on packed HE (PackedHE), to accelerate the existing HE-based VFL algorithms. PackVFL packs multiple cleartexts into one ciphertext and supports single-instruction-multiple-data (SIMD)-style parallelism. We focus on designing a high-performant matrix multiplication (MatMult) method since it takes up most of the ciphertext computation time in HE-based VFL. Besides, devising the MatMult method is also challenging for PackedHE because a slight difference in the packing way could predominantly affect its computation and communication costs. Without domain-specific design, directly applying SOTA MatMult methods is hard to achieve optimal. Therefore, we make a three-fold design: 1) we systematically explore the current design space of MatMult and quantify the complexity of existing approaches to provide guidance; 2) we propose a hybrid MatMult method according to the unique characteristics of VFL; 3) we adaptively apply our hybrid method in representative VFL algorithms, leveraging distinctive algorithmic properties to further improve efficiency. As the batch size, feature dimension and model size of VFL scale up to large sizes, PackVFL consistently delivers enhanced performance. Empirically, PackVFL propels existing VFL algorithms to new heights, achieving up to a 51.52X end-to-end speedup. This represents a substantial 34.51X greater speedup compared to the direct application of SOTA MatMult methods.