mlsanityData to 14 Oct 2026

Private LoRA Fine-tuning of Open-Source LLMs with Homomorphic Encryption

arXiv 2505.07329 ↗First posted 12 May 2025cs.CR cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Preserving data confidentiality during the fine-tuning of open-source Large Language Models (LLMs) is crucial for sensitive applications. This work introduces an interactive protocol adapting the Low-Rank Adaptation (LoRA) technique for private fine-tuning. Homomorphic Encryption (HE) protects the confidentiality of training data and gradients handled by remote worker nodes performing the bulk of computations involving the base model weights. The data owner orchestrates training, requiring minimal local computing power and memory, thus alleviating the need for expensive client-side GPUs. We demonstrate feasibility by fine-tuning a Llama-3.2-1B model, presenting convergence results using HE-compatible quantization and performance benchmarks for HE computations on GPU hardware. This approach enables applications such as confidential knowledge base question answering, private codebase fine-tuning for AI code assistants, AI agents for drafting emails based on a company's email archive, and adapting models to analyze sensitive legal or healthcare documents.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 10 of its 31 references that are in the map. First posted on arXiv 12 May 2025.

Builds on

10 of 31 references.

References · 10

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  6. 2018Gazelle: A Low Latency Framework for Secure Neural Network Inferencecited by 193
  7. 2019Glyph: Fast and Accurately Training Deep Neural Networks on Encrypted Datacited by 28
  8. 2023PrivateLoRA For Efficient Privacy Preserving LLMcited by 14
  9. 2024DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Modelcited by 8
  10. 2024PackVFL: Efficient HE Packing for Vertical Federated Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environmentscited by 8
  2. 2025Evolution of meta's llama models and parameter-efficient fine-tuning of large language models: a surveycited by 5
  3. 2025Key-Conditioned Orthonormal Transform Gating (K-OTG): Multi-Key Access Control with Hidden-State Scrambling for LoRA-Tuned Modelscited by 1
  4. 2026Training Machine Learning Models on Encrypted Data: A Privacy-Preserving Framework using Homomorphic Encryptioncited by 0
  5. 2026CipherGenome: Homomorphic Inference for Genomic Mixture-of-Expertscited by 0
  6. 2026DP-ES: Differentially Private Evolution Strategies for Prompt Optimizationcited by 0
  7. 2026HE-OFT: Privacy-Preserving One-Shot Federated Fine-Tuning under Homomorphic Encryptioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Prada: Black-Box LLM Adaptation with Private Data on Resource-Constrained Devices
  2. 2023Last One Standing: A Comparative Analysis of Security and Privacy of Soft Prompt Tuning, LoRA, and In-Context Learning
  3. 2026HEAT: Faster Fully Homomorphic Inference via Approximations-Weights Co-Adaptation
  4. 2025DistilLock: Safeguarding LLMs from Unauthorized Knowledge Distillation on the Edge
  5. 2026Rethinking LoRA for Privacy-Preserving Federated Learning in Large Models
  6. 2025NoEsis: Differentially Private Knowledge Transfer in Modular LLM Adaptation
  7. 2026Securing LLMs in the Wild: Privacy and Security Challenges at the Edge
  8. 2026Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptation