# HE-OFT: Privacy-Preserving One-Shot Federated Fine-Tuning under Homomorphic Encryption

arXiv: https://arxiv.org/abs/2610.08255 · HTML: https://mlsanity.com/paper/2610.08255/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 134 references that are in the map.
- First posted on arXiv 6 Oct 2026.

## Builds on (most cited first, 14 of 14)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [One-Shot Federated Learning](https://mlsanity.com/paper/1902.11175/) (2019)
- [Offsite-Tuning: Transfer Learning without Full Model](https://mlsanity.com/paper/2302.04870/) (2023)
- [PUMA: Secure Inference of LLaMA-7B in Five Minutes](https://mlsanity.com/paper/2307.12533/) (2023)
- [Reconstructing Training Data From Real World Models Trained with Transfer Learning](https://mlsanity.com/paper/2407.15845/) (2024)
- [Private LoRA Fine-tuning of Open-Source LLMs with Homomorphic Encryption](https://mlsanity.com/paper/2505.07329/) (2025)
- [Efficient Decoding Methods for Language Models on Encrypted Data](https://mlsanity.com/paper/2509.08383/) (2025)
- [FIDESlib: A Fully-Fledged Open-Source FHE Library for Efficient CKKS on GPUs](https://mlsanity.com/paper/2507.04775/) (2025)
- [PrivTuner with Homomorphic Encryption and LoRA: A P3EFT Scheme for Privacy-Preserving Parameter-Efficient Fine-Tuning of AI Foundation Models](https://mlsanity.com/paper/2410.00433/) (2024)
- [A Training Framework for Optimal and Stable Training of Polynomial Neural Networks](https://mlsanity.com/paper/2505.11589/) (2025)
- [Revisiting Federated Fine-Tuning: A Single Communication Round is Enough for Foundation Models](https://mlsanity.com/paper/2412.04650/) (2024)
- [Privacy-Preserving CNN Training with Transfer Learning: Two Hidden Layers](https://mlsanity.com/paper/2504.12623/) (2025)

## Abstract (arXiv)

Many organizations adapt large pretrained models to their own tasks by fine-tuning on private data. Several of these parties often hold data for the same task and wish to fine-tune a model together without pooling that data. Federated learning (FL) enables joint fine-tuning, but reconstruction attacks on shared intermediate values (the model or its gradients) remain a privacy risk. A one-shot protocol that exchanges one encrypted contribution exposes no intermediate value. Such a protocol still gives the trained model to every participant, which is not permitted where the model is a regulated or proprietary asset. We present HE-OFT, the first cryptographically secure one-shot federated fine-tuning protocol in which no party receives the trained model. Each client fine-tunes a low-rank adapter and a classifier head on a frozen public backbone and keeps the adapter. The client uploads one encrypted head displacement, which the server combines under multiparty CKKS and never decrypts. A quorum of clients returns only the predicted label to the querier. On four text classification tasks and one vision task, HE-OFT reaches 61 to 79 per cent accuracy, against 20 to 48 per cent for a client training alone. HE-OFT keeps 85 to 96 per cent of the accuracy of a disclosed model. A test-time query takes 443.1 to 1713.1 s on one core, or 56.1 to 255.1 s with level restoration on a GPU. Restoring levels at the server cuts the traffic per query from up to 1.6 GiB to 13.5 MiB.