# PUMA: Secure Inference of LLaMA-7B in Five Minutes

arXiv: https://arxiv.org/abs/2307.12533 · HTML: https://mlsanity.com/paper/2307.12533/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2023.

## Built on by (most cited first, 30 of 48)

- [Privacy in Large Language Models: Attacks, Defenses and Future Directions](https://mlsanity.com/paper/2310.10383/) (2023)
- [Grounding Foundation Models through Federated Transfer Learning: A General Framework](https://mlsanity.com/paper/2311.17431/) (2023)
- [Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Survey](https://mlsanity.com/paper/2406.05392/) (2024)
- [Encryption-Friendly LLM Architecture](https://mlsanity.com/paper/2410.02486/) (2024)
- [Nimbus: Secure and Efficient Two-Party Inference for Transformers](https://mlsanity.com/paper/2411.15707/) (2024)
- [No Free Lunch Theorem for Privacy-Preserving LLM Inference](https://mlsanity.com/paper/2405.20681/) (2024)
- [CipherPrune: Efficient and Scalable Private Transformer Inference](https://mlsanity.com/paper/2502.16782/) (2025)
- [East: Efficient and Accurate Secure Transformer Framework for Inference](https://mlsanity.com/paper/2308.09923/) (2023)
- [SecFormer: Fast and Accurate Privacy-Preserving Inference for Transformer Models via SMPC](https://mlsanity.com/paper/2401.00793/) (2024)
- [Ditto: Quantization-aware Secure Inference of Transformers upon MPC](https://mlsanity.com/paper/2405.05525/) (2024)
- [Power-Softmax: Towards Secure LLM Inference over Encrypted Data](https://mlsanity.com/paper/2410.09457/) (2024)
- [Lambda-Split: A Privacy-Preserving Split Computing Framework for Cloud-Powered Generative AI](https://mlsanity.com/paper/2310.14651/) (2023)
- [MPC-Minimized Secure LLM Inference](https://mlsanity.com/paper/2408.03561/) (2024)
- [CENTAUR: Bridging the Impossible Trinity of Privacy, Efficiency, and Performance in Privacy-Preserving Transformer Inference](https://mlsanity.com/paper/2412.10652/) (2024)
- [PPMI: Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databases](https://mlsanity.com/paper/2506.17336/) (2025)
- [Input Reconstruction Attack against Vertical Federated Large Language Models](https://mlsanity.com/paper/2311.07585/) (2023)
- [Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacy](https://mlsanity.com/paper/2505.05922/) (2025)
- [Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsity](https://mlsanity.com/paper/2505.07239/) (2025)
- [CipherDM: Secure Three-Party Inference for Diffusion Model Sampling](https://mlsanity.com/paper/2409.05414/) (2024)
- [TruncFormer: Private LLM Inference Using Only Truncations](https://mlsanity.com/paper/2412.01042/) (2024)
- [Private Transformer Inference in MLaaS: A Survey](https://mlsanity.com/paper/2505.10315/) (2025)
- [An Efficient Private GPT Never Autoregressively Decodes](https://mlsanity.com/paper/2505.15252/) (2025)
- [An Attack to Break Permutation-Based Private Third-Party Inference Schemes for LLMs](https://mlsanity.com/paper/2505.18332/) (2025)
- [ENSI: Efficient Non-Interactive Secure Inference for Large Language Models](https://mlsanity.com/paper/2509.09424/) (2025)
- [SecMoE: Communication-Efficient Secure MoE Inference via Select-Then-Compute](https://mlsanity.com/paper/2601.06790/) (2026)
- [On the (In-)Security of the Shuffling Defense in the Transformer Secure Inference](https://mlsanity.com/paper/2605.04901/) (2026)
- [ATLAS: Automated Approximation of Transformers for Efficient Homomorphic Inference in One Hour](https://mlsanity.com/paper/2607.23478/) (2026)
- [SelectFormer: Private and Practical Data Selection for Transformers](https://mlsanity.com/paper/2310.02373/) (2023)
- [A Framework for Cost-Effective and Self-Adaptive LLM Shaking and Recovery Mechanism](https://mlsanity.com/paper/2403.07283/) (2024)
- [SecP-Tuning: Efficient Privacy-Preserving Prompt Tuning for Large Language Models via MPC](https://mlsanity.com/paper/2506.15307/) (2025)

## Abstract (arXiv)

With ChatGPT as a representative, tons of companies have began to provide services based on large Transformers models. However, using such a service inevitably leak users' prompts to the model provider. Previous studies have studied secure inference for Transformer models using secure multiparty computation (MPC), where model parameters and clients' prompts are kept secret. Despite this, these frameworks are still limited in terms of model performance, efficiency, and deployment. To address these limitations, we propose framework PUMA to enable fast and secure Transformer model inference. Our framework designs high quality approximations for expensive functions such as GeLU and softmax, and significantly reduce the cost of secure inference while preserving the model performance. Additionally, we design secure Embedding and LayerNorm procedures that faithfully implement the desired functionality without undermining the Transformer architecture. PUMA is about 2× faster than the state-of-the-art framework MPCFORMER(ICLR 2023) and has similar accuracy as plaintext models without fine-tuning (which the previous works failed to achieve). PUMA can even evaluate LLaMA-7B in around 5 minutes to generate 1 token. To our best knowledge, this is the first time that a model with such a parameter size is able to be evaluated under MPC. PUMA has been open-sourced in the Github repository of SecretFlow-SPU.