mlsanityData to 14 Oct 2026

An Efficient Private GPT Never Autoregressively Decodes

arXiv 2505.15252 ↗First posted 21 May 2025cs.CR cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

The wide deployment of the generative pre-trained transformer (GPT) has raised privacy concerns for both clients and servers. While cryptographic primitives can be employed for secure GPT inference to protect the privacy of both parties, they introduce considerable performance overhead.To accelerate secure inference, this study proposes a public decoding and secure verification approach that utilizes public GPT models, motivated by the observation that securely decoding one and multiple tokens takes a similar latency. The client uses the public model to generate a set of tokens, which are then securely verified by the private model for acceptance. The efficiency of our approach depends on the acceptance ratio of tokens proposed by the public model, which we improve from two aspects: (1) a private sampling protocol optimized for cryptographic primitives and (2) model alignment using knowledge distillation. Our approach improves the efficiency of secure decoding while maintaining the same level of privacy and generation quality as standard secure decoding. Experiments demonstrate a 2.1× ~ 6.0× speedup compared to standard decoding across three pairs of public-private models and different network conditions.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 31 of its 56 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

31 of 56 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  7. 2023A Survey of Large Language Modelscited by 2,576
  8. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  9. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  10. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  11. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  12. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
19 more
  1. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
  2. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  3. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 365
  4. 2023Online Speculative Decodingcited by 113
  5. 2021Scale Efficiently: Insights from Pre-training and Fine-tuning Transformerscited by 106
  6. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 96
  7. 2020CrypTFlow2: Practical 2-Party Secure Inferencecited by 82
  8. 2022MPCFormer: fast, performant and private Transformer inference with MPCcited by 69
  9. 2022THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryptioncited by 57
  10. 2023ChatGPT: A Meta-Analysis after 2.5 Monthscited by 51
  11. 2023PUMA: Secure Inference of LLaMA-7B in Five Minutescited by 48
  12. 2023Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammarscited by 30
  13. 2021SIRNN: A Math Library for Secure RNN Inferencecited by 28
  14. 2022MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attentioncited by 20
  15. 2023Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryptioncited by 19
  16. 2024Nimbus: Secure and Efficient Two-Party Inference for Transformerscited by 16
  17. 2023MERGE: Fast Private Text Generationcited by 9
  18. 2024PrivCirNet: Efficient Private Inference via Block Circulant Transformationcited by 7
  19. 2021Probing Linguistic Information For Logical Inference In Pre-trained Language Modelscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Static Bootstrap Placement for Encrypted Language Model Decodingcited by 0
  2. 2026A Graph-Based Analysis of Comparability, Scale, and Attribution in Private Transformer Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CipherPrune: Efficient and Scalable Private Transformer Inference
  2. 2026Privacy-Aware Split Inference with Speculative Decoding for Large Language Models over Wide-Area Networks
  3. 2026SecMoE: Communication-Efficient Secure MoE Inference via Select-Then-Compute
  4. 2026HEAT: Faster Fully Homomorphic Inference via Approximations-Weights Co-Adaptation
  5. 2024Ditto: Quantization-aware Secure Inference of Transformers upon MPC
  6. 2026Making Every Verified Token Count: Adaptive Verification for MoE Speculative Decoding
  7. 2026SecureRouter: Encrypted Routing for Efficient Secure Inference
  8. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models