mlsanityData to 14 Oct 2026

PUMA: Secure Inference of LLaMA-7B in Five Minutes

arXiv 2307.12533 ↗First posted 24 Jul 2023cs.CRcited by 48

Not on the map. Why?

Abstract (arXiv)

With ChatGPT as a representative, tons of companies have began to provide services based on large Transformers models. However, using such a service inevitably leak users' prompts to the model provider. Previous studies have studied secure inference for Transformer models using secure multiparty computation (MPC), where model parameters and clients' prompts are kept secret. Despite this, these frameworks are still limited in terms of model performance, efficiency, and deployment. To address these limitations, we propose framework PUMA to enable fast and secure Transformer model inference. Our framework designs high quality approximations for expensive functions such as GeLU and softmax, and significantly reduce the cost of secure inference while preserving the model performance. Additionally, we design secure Embedding and LayerNorm procedures that faithfully implement the desired functionality without undermining the Transformer architecture. PUMA is about 2× faster than the state-of-the-art framework MPCFORMER(ICLR 2023) and has similar accuracy as plaintext models without fine-tuning (which the previous works failed to achieve). PUMA can even evaluate LLaMA-7B in around 5 minutes to generate 1 token. To our best knowledge, this is the first time that a model with such a parameter size is able to be evaluated under MPC. PUMA has been open-sourced in the Github repository of SecretFlow-SPU.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Jul 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 14; 2025: 16; 2026: 12.

48 papers.

Sort:

By area:

  1. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 51
  2. 2023Grounding Foundation Models through Federated Transfer Learning: A General Frameworkcited by 24
  3. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  4. 2024Encryption-Friendly LLM Architecturecited by 17
  5. 2024Nimbus: Secure and Efficient Two-Party Inference for Transformerscited by 16
  6. 2024No Free Lunch Theorem for Privacy-Preserving LLM Inferencecited by 13
  7. 2025CipherPrune: Efficient and Scalable Private Transformer Inferencecited by 10
  8. 2023East: Efficient and Accurate Secure Transformer Framework for Inferencecited by 9
  9. 2024SecFormer: Fast and Accurate Privacy-Preserving Inference for Transformer Models via SMPCcited by 9
  10. 2024Ditto: Quantization-aware Secure Inference of Transformers upon MPCcited by 8
  11. 2024Power-Softmax: Towards Secure LLM Inference over Encrypted Datacited by 7
  12. 2023Lambda-Split: A Privacy-Preserving Split Computing Framework for Cloud-Powered Generative AIcited by 6
36 more
  1. 2024MPC-Minimized Secure LLM Inferencecited by 6
  2. 2024CENTAUR: Bridging the Impossible Trinity of Privacy, Efficiency, and Performance in Privacy-Preserving Transformer Inferencecited by 6
  3. 2025PPMI: Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databasescited by 6
  4. 2023Input Reconstruction Attack against Vertical Federated Large Language Modelscited by 5
  5. 2025Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacycited by 5
  6. 2025Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsitycited by 4
  7. 2024CipherDM: Secure Three-Party Inference for Diffusion Model Samplingcited by 3
  8. 2024TruncFormer: Private LLM Inference Using Only Truncationscited by 2
  9. 2025Private Transformer Inference in MLaaS: A Surveycited by 2
  10. 2025An Efficient Private GPT Never Autoregressively Decodescited by 2
  11. 2025An Attack to Break Permutation-Based Private Third-Party Inference Schemes for LLMscited by 2
  12. 2025ENSI: Efficient Non-Interactive Secure Inference for Large Language Modelscited by 2
  13. 2026SecMoE: Communication-Efficient Secure MoE Inference via Select-Then-Computecited by 2
  14. 2026On the (In-)Security of the Shuffling Defense in the Transformer Secure Inferencecited by 2
  15. 2026ATLAS: Automated Approximation of Transformers for Efficient Homomorphic Inference in One Hourcited by 2
  16. 2023SelectFormer: Private and Practical Data Selection for Transformerscited by 1
  17. 2024A Framework for Cost-Effective and Self-Adaptive LLM Shaking and Recovery Mechanismcited by 1
  18. 2025SecP-Tuning: Efficient Privacy-Preserving Prompt Tuning for Large Language Models via MPCcited by 1
  19. 2025Towards Provable (In)Secure Model Weight Release Schemescited by 1
  20. 2025Cascade: Token-Sharded Private LLM Inferencecited by 1
  21. 2025CrypTorch: PyTorch-based Auto-tuning Compiler for Machine Learning with Multi-party Computationcited by 1
  22. 2026OSNIP: Breaking the Privacy-Utility-Efficiency Trilemma in LLM Inference via Obfuscated Semantic Null Spacecited by 1
  23. 2026An Efficient and Privacy-Preserving Architecture for Cross-Institutional Collaborative RAGcited by 1
  24. 2026Privacy Failure in Split-LLM Training, The Returned Gradient Nullifies the Decoyscited by 1
  25. 2024Spin: An Efficient Secure Computation Framework with GPU Accelerationcited by 0
  26. 2024TensorTEE: Unifying Heterogeneous TEE Granularity for Efficient Secure Collaborative Tensor Computingcited by 0
  27. 2025Privacy-Preserving Inference for Quantized BERT Modelscited by 0
  28. 2025PRIVMARK: Private Large Language Models Watermarking with MPCcited by 0
  29. 2025ObCLIP: Oblivious CLoud-Device Hybrid Image Generation with Privacy Preservationcited by 0
  30. 2025PrivLLMSwarm: Privacy-Preserving LLM-Driven UAV Swarms for Secure IoT Surveillancecited by 0
  31. 2026Privacy-Preserving Mechanisms Enable Cheap Verifiable Inference of LLMscited by 0
  32. 2026HEAT: Faster Fully Homomorphic Inference via Approximations-Weights Co-Adaptationcited by 0
  33. 2026CipherGenome: Homomorphic Inference for Genomic Mixture-of-Expertscited by 0
  34. 2026Static Bootstrap Placement for Encrypted Language Model Decodingcited by 0
  35. 2026HE-OFT: Privacy-Preserving One-Shot Federated Fine-Tuning under Homomorphic Encryptioncited by 0
  36. 2026A Graph-Based Analysis of Comparability, Scale, and Attribution in Private Transformer Inferencecited by 0