mlsanityData to 7 Oct 2026

MPC-Minimized Secure LLM Inference

arXiv 2408.03561 ↗First posted 7 Aug 2024cs.CR cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Many inference services based on large language models (LLMs) pose a privacy concern, either revealing user prompts to the service or the proprietary weights to the user. Secure inference offers a solution to this problem through secure multi-party computation (MPC), however, it is still impractical for modern LLM workload due to the large overhead imposed by MPC. To address this overhead, we propose Marill, a framework that adapts LLM fine-tuning to minimize MPC usage during secure inference. Marill introduces high-level architectural changes during fine-tuning that significantly reduce the number of expensive operations needed within MPC during inference, by removing some and relocating others outside MPC without compromising security. As a result, Marill-generated models are more efficient across all secure inference protocols and our approach complements MPC-friendly approximations for such operations. Compared to standard fine-tuning, Marill results in 3.6-11.3x better runtime and 2.4-6.9x better communication during secure inference across various MPC settings, while typically preserving over 90% performance across downstream tasks.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 36 of its 76 references that are in the map. First posted on arXiv 7 Aug 2024.

Builds on

36 of 76 references.

References · 36

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2023Mistral 7Bcited by 3,065
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  9. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  10. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  11. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  12. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
24 more
  1. 2019Are Sixteen Heads Really Better than One?cited by 803
  2. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  3. 2018Gazelle: A Low Latency Framework for Secure Neural Network Inferencecited by 193
  4. 2021CrypTen: Secure Multi-Party Computation Meets Machine Learningcited by 131
  5. 2023Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labellingcited by 87
  6. 2020CrypTFlow2: Practical 2-Party Secure Inferencecited by 82
  7. 2023Fairness in Serving Large Language Modelscited by 77
  8. 2020FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learningcited by 74
  9. 2018Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applicationscited by 67
  10. 2022MPCFormer: fast, performant and private Transformer inference with MPCcited by 67
  11. 2019CrypTFlow: Secure TensorFlow Inferencecited by 61
  12. 2022THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryptioncited by 55
  13. 2021DeepReDuce: ReLU Reduction for Fast Private Inferencecited by 50
  14. 2021CryptGPU: Fast Privacy-Preserving Machine Learning on the GPUcited by 46
  15. 2018Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inferencecited by 45
  16. 2023PUMA: Secure Inference of LLaMA-7B in Five Minutescited by 44
  17. 2020CryptoNAS: Private Inference on a ReLU Budgetcited by 40
  18. 2023ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedbackcited by 34
  19. 2022Selective Network Linearization for Efficient Private Inferencecited by 31
  20. 2023Learning to Linearize Deep Neural Networks for Secure and Efficient Private Inferencecited by 20
  21. 2022MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attentioncited by 19
  22. 2021Sphynx: ReLU-Efficient Network Design for Private Inferencecited by 15
  23. 2023RRNet: Towards ReLU-Reduced Neural Network for Two-party Computation Based Private Inferencecited by 12
  24. 2021GALA: Greedy ComputAtion for Linear Algebra in Privacy-Preserved Neural Networkscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  2. 2026AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systemscited by 20
  3. 2026SoK: Privacy-aware LLM in Healthcare: Threat Model, Privacy Techniques, Challenges and Recommendationscited by 1
  4. 2025CryptoTensors: A Light-Weight Large Language Model File Format for Highly-Secure Model Distributioncited by 0
  5. 2026When Agents Handle Secrets: A Survey of Confidential Computing for Agentic AIcited by 0