# ObfuscaTune: Obfuscated Offsite Fine-tuning and Inference of Proprietary LLMs on Private Datasets

arXiv: https://arxiv.org/abs/2407.02960 · HTML: https://mlsanity.com/paper/2407.02960/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 3 Jul 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Crowdsourcing Multiple Choice Science Questions](https://mlsanity.com/paper/1707.06209/) (2017)
- [Differentially Private Fine-tuning of Language Models](https://mlsanity.com/paper/2110.06500/) (2021)
- [Text Embeddings Reveal (Almost) As Much As Text](https://mlsanity.com/paper/2310.06816/) (2023)
- [Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attacks](https://mlsanity.com/paper/2203.03929/) (2022)
- [ChipNeMo: Domain-Adapted LLMs for Chip Design](https://mlsanity.com/paper/2311.00176/) (2023)
- [Stealing Part of a Production Language Model](https://mlsanity.com/paper/2403.06634/) (2024)
- [CrypTen: Secure Multi-Party Computation Meets Machine Learning](https://mlsanity.com/paper/2109.00984/) (2021)
- [ChatEDA: A Large Language Model Powered Autonomous Agent for EDA](https://mlsanity.com/paper/2308.10204/) (2023)
- [Benchmarking Large Language Models for Automated Verilog RTL Code Generation](https://mlsanity.com/paper/2212.11140/) (2022)
- [Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models](https://mlsanity.com/paper/2404.02936/) (2024)
- [Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence](https://mlsanity.com/paper/2305.03010/) (2023)
- [Offsite-Tuning: Transfer Learning without Full Model](https://mlsanity.com/paper/2302.04870/) (2023)
- [MPCFormer: fast, performant and private Transformer inference with MPC](https://mlsanity.com/paper/2211.01452/) (2022)
- [Differentially Private Decoding in Large Language Models](https://mlsanity.com/paper/2205.13621/) (2022)
- [DP-BART for Privatized Text Rewriting under Local Differential Privacy](https://mlsanity.com/paper/2302.07636/) (2023)
- [HEMET: A Homomorphic-Encryption-Friendly Privacy-Preserving Mobile Neural Network Architecture](https://mlsanity.com/paper/2106.00038/) (2021)
- [LLMs Can Understand Encrypted Prompt: Towards Privacy-Computing Friendly Transformers](https://mlsanity.com/paper/2305.18396/) (2023)

## Built on by (most cited first, 4 of 4)

- [LLM Access Shield: Domain-Specific LLM Framework for Privacy Policy Compliance](https://mlsanity.com/paper/2505.17145/) (2025)
- [Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks](https://mlsanity.com/paper/2504.17421/) (2025)
- [Prada: Black-Box LLM Adaptation with Private Data on Resource-Constrained Devices](https://mlsanity.com/paper/2503.14932/) (2025)
- [Privacy from Symmetry: Orthogonally Equivariant Transformers for LLM Inference](https://mlsanity.com/paper/2606.16461/) (2026)

## Abstract (arXiv)

This work addresses the timely yet underexplored problem of performing inference and finetuning of a proprietary LLM owned by a model provider entity on the confidential/private data of another data owner entity, in a way that ensures the confidentiality of both the model and the data. Hereby, the finetuning is conducted offsite, i.e., on the computation infrastructure of a third-party cloud provider. We tackle this problem by proposing ObfuscaTune, a novel, efficient and fully utility-preserving approach that combines a simple yet effective obfuscation technique with an efficient usage of confidential computing (only 5% of the model parameters are placed on TEE). We empirically demonstrate the effectiveness of ObfuscaTune by validating it on GPT-2 models with different sizes on four NLP benchmark datasets. Finally, we compare to a na"ive version of our approach to highlight the necessity of using random matrices with low condition numbers in our approach to reduce errors induced by the obfuscation.