# CipherPrune: Efficient and Scalable Private Transformer Inference

arXiv: https://arxiv.org/abs/2502.16782 · HTML: https://mlsanity.com/paper/2502.16782/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 52 references that are in the map.
- First posted on arXiv 24 Feb 2025.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SnapKV: LLM Knows What You are Looking for Before Generation](https://mlsanity.com/paper/2404.14469/) (2024)
- [I-BERT: Integer-only BERT Quantization](https://mlsanity.com/paper/2101.01321/) (2021)
- [SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning](https://mlsanity.com/paper/2012.09852/) (2020)
- [Learned Token Pruning for Transformers](https://mlsanity.com/paper/2107.00910/) (2021)
- [CrypTFlow2: Practical 2-Party Secure Inference](https://mlsanity.com/paper/2010.06457/) (2020)
- [Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search](https://mlsanity.com/paper/2010.07003/) (2020)
- [LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference](https://mlsanity.com/paper/2407.14057/) (2024)
- [MPCFormer: fast, performant and private Transformer inference with MPC](https://mlsanity.com/paper/2211.01452/) (2022)
- [THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption](https://mlsanity.com/paper/2206.00216/) (2022)
- [PUMA: Secure Inference of LLaMA-7B in Five Minutes](https://mlsanity.com/paper/2307.12533/) (2023)
- [SHE: A Fast and Accurate Deep Neural Network for Encrypted Data](https://mlsanity.com/paper/1906.00148/) (2019)
- [TrojViT: Trojan Insertion in Vision Transformers](https://mlsanity.com/paper/2208.13049/) (2022)
- [TrojLLM: A Black-box Trojan Prompt Attack on Large Language Models](https://mlsanity.com/paper/2306.06815/) (2023)
- [Glyph: Fast and Accurately Training Deep Neural Networks on Encrypted Data](https://mlsanity.com/paper/1911.07101/) (2019)
- [SIRNN: A Math Library for Secure RNN Inference](https://mlsanity.com/paper/2105.04236/) (2021)
- [MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attention](https://mlsanity.com/paper/2211.13955/) (2022)
- [Primer: Fast Private Transformer Inference on Encrypted Data](https://mlsanity.com/paper/2303.13679/) (2023)
- [Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryption](https://mlsanity.com/paper/2311.08610/) (2023)
- [Exploiting Data Sparsity in Secure Cross-Platform Social Recommendation](https://mlsanity.com/paper/2202.07253/) (2022)
- [freePruner: A Training-free Approach for Large Multimodal Model Acceleration](https://mlsanity.com/paper/2411.15446/) (2024)
- [PrivCirNet: Efficient Private Inference via Block Circulant Transformation](https://mlsanity.com/paper/2405.14569/) (2024)
- [BoostCom: Towards Efficient Universal Fully Homomorphic Encryption by Boosting the Word-wise Comparisons](https://mlsanity.com/paper/2407.07308/) (2024)
- [Trinity: A General Purpose FHE Accelerator](https://mlsanity.com/paper/2410.13405/) (2024)
- [DataSeal: Ensuring the Verifiability of Private Computation on Encrypted Data](https://mlsanity.com/paper/2410.15215/) (2024)
- [MATCHA: A Fast and Energy-Efficient Accelerator for Fully Homomorphic Encryption over the Torus](https://mlsanity.com/paper/2202.08814/) (2022)
- [CryptoTrain: Fast Secure Training on Encrypted Dataset](https://mlsanity.com/paper/2409.16675/) (2024)
- [OFHE: An Electro-Optical Accelerator for Discretized TFHE](https://mlsanity.com/paper/2405.11607/) (2024)

## Built on by (most cited first, 10 of 10)

- [Trusted AI Agents in the Cloud](https://mlsanity.com/paper/2512.05951/) (2025)
- [TFHE-Coder: Evaluating LLM-agentic Fully Homomorphic Encryption Code Generation](https://mlsanity.com/paper/2503.12217/) (2025)
- [Securing Transformer-based AI Execution via Unified TEEs and Crypto-protected Accelerators](https://mlsanity.com/paper/2507.03278/) (2025)
- [DESIGN: Encrypted GNN Inference via Server-Side Input Graph Pruning](https://mlsanity.com/paper/2507.05649/) (2025)
- [Black Box Absorption: LLMs Undermining Innovative Ideas](https://mlsanity.com/paper/2510.20612/) (2025)
- [DictPFL: Efficient and Private Federated Learning on Encrypted Gradients](https://mlsanity.com/paper/2510.21086/) (2025)
- [Steerability of Instrumental-Convergence Tendencies in LLMs](https://mlsanity.com/paper/2601.01584/) (2026)
- [RobPI: Robust Private Inference against Malicious Client](https://mlsanity.com/paper/2602.19918/) (2026)
- [DegreeSpar: Structured Degree Sparsity for Efficient Secure Transformer Inference](https://mlsanity.com/paper/2609.32204/) (2026)
- [A Graph-Based Analysis of Comparability, Scale, and Attribution in Private Transformer Inference](https://mlsanity.com/paper/2610.19607/) (2026)

## Abstract (arXiv)

Private Transformer inference using cryptographic protocols offers promising solutions for privacy-preserving machine learning; however, it still faces significant runtime overhead (efficiency issues) and challenges in handling long-token inputs (scalability issues). We observe that the Transformer's operational complexity scales quadratically with the number of input tokens, making it essential to reduce the input token length. Notably, each token varies in importance, and many inputs contain redundant tokens. Additionally, prior private inference methods that rely on high-degree polynomial approximations for non-linear activations are computationally expensive. Therefore, reducing the polynomial degree for less important tokens can significantly accelerate private inference. Building on these observations, we propose textitCipherPrune, an efficient and scalable private inference framework that includes a secure encrypted token pruning protocol, a polynomial reduction protocol, and corresponding Transformer network optimizations. At the protocol level, encrypted token pruning adaptively removes unimportant tokens from encrypted inputs in a progressive, layer-wise manner. Additionally, encrypted polynomial reduction assigns lower-degree polynomials to less important tokens after pruning, enhancing efficiency without decryption. At the network level, we introduce protocol-aware network optimization via a gradient-based search to maximize pruning thresholds and polynomial reduction conditions while maintaining the desired accuracy. Our experiments demonstrate that CipherPrune reduces the execution overhead of private Transformer inference by approximately 6.1× for 128-token inputs and 10.6× for 512-token inputs, compared to previous methods, with only a marginal drop in accuracy. The code is publicly available at https://github.com/UCF-Lou-Lab-PET/cipher-prune-inference.