# OptiPrime: Optimizing Private Inference through Protocol-Hardware Co-design

arXiv: https://arxiv.org/abs/2609.16898 · HTML: https://mlsanity.com/paper/2609.16898/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 82 references that are in the map.
- First posted on arXiv 15 Sep 2026.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [CrypTFlow2: Practical 2-Party Secure Inference](https://mlsanity.com/paper/2010.06457/) (2020)
- [Privacy in Deep Learning: A Survey](https://mlsanity.com/paper/2004.12254/) (2020)
- [CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU](https://mlsanity.com/paper/2104.10949/) (2021)
- [SIRNN: A Math Library for Secure RNN Inference](https://mlsanity.com/paper/2105.04236/) (2021)
- [F1: A Fast and Programmable Accelerator for Fully Homomorphic Encryption (Extended Version)](https://mlsanity.com/paper/2109.05371/) (2021)
- [HeLayers: A Tile Tensors Framework for Large Neural Networks on Encrypted Data](https://mlsanity.com/paper/2011.01805/) (2020)
- [Nimbus: Secure and Efficient Two-Party Inference for Transformers](https://mlsanity.com/paper/2411.15707/) (2024)
- [AESPA: Accuracy Preserving Low-degree Polynomial Activation for Fast Private Inference](https://mlsanity.com/paper/2201.06699/) (2022)
- [BTS: An Accelerator for Bootstrappable Fully Homomorphic Encryption](https://mlsanity.com/paper/2112.15479/) (2021)
- [ARK: Fully Homomorphic Encryption Accelerator with Runtime Data Generation and Inter-Operation Key Reuse](https://mlsanity.com/paper/2205.00922/) (2022)
- [HEAX: An Architecture for Computing on Encrypted Data](https://mlsanity.com/paper/1909.09731/) (2019)
- [Deep Neural Networks for Encrypted Inference with TFHE](https://mlsanity.com/paper/2302.10906/) (2023)
- [CryptoGCN: Fast and Scalable Homomorphically Encrypted Graph Convolutional Network Inference](https://mlsanity.com/paper/2209.11904/) (2022)
- [TensorFHE: Achieving Practical Computation on Encrypted Data Using GPGPU](https://mlsanity.com/paper/2212.14191/) (2022)
- [Characterizing and Optimizing End-to-End Systems for Private Inference](https://mlsanity.com/paper/2207.07177/) (2022)
- [PrivCirNet: Efficient Private Inference via Block Circulant Transformation](https://mlsanity.com/paper/2405.14569/) (2024)
- [Breaking the Layer Barrier: Remodeling Private Transformer Inference with Hybrid CKKS and MPC](https://mlsanity.com/paper/2508.19525/) (2025)
- [GME: GPU-based Microarchitectural Extensions to Accelerate Homomorphic Encryption](https://mlsanity.com/paper/2309.11001/) (2023)
- [FAB: An FPGA-based Accelerator for Bootstrappable Fully Homomorphic Encryption](https://mlsanity.com/paper/2207.11872/) (2022)
- [Falcon: Accelerating Homomorphically Encrypted Convolutions for Efficient Private Mobile Network Inference](https://mlsanity.com/paper/2308.13189/) (2023)
- [Trinity: A General Purpose FHE Accelerator](https://mlsanity.com/paper/2410.13405/) (2024)
- [Osiris: A Systolic Approach to Accelerating Fully Homomorphic Encryption](https://mlsanity.com/paper/2408.09593/) (2024)
- [APINT: A Full-Stack Framework for Acceleration of Privacy-Preserving Inference of Transformers based on Garbled Circuits](https://mlsanity.com/paper/2502.16877/) (2025)

## Abstract (arXiv)

Private deep neural network (DNN) inference based on hybrid homomorphic encryption (HE) and multi-party computation (MPC) can protect user data with a formal guarantee, but at the cost of significant latency overhead due to HE. Customized HE accelerators have been proposed and have achieved orders-of-magnitude speedup for individual HE operations. However, when directly applying a commercial HE accelerator to state-of-the-art HE-MPC frameworks, we observe only limited end-to-end performance gain. This is because HE-MPC frameworks often require wireless transmission of input and output ciphertexts for each HE operation, leading to a severe network communication bottleneck. To overcome this challenge, we introduce OptiPrime, a protocol-hardware co-optimization framework for efficient private DNN inference. OptiPrime features a novel HE protocol for convolutions that substantially reduces the number of transmitted output ciphertexts and mitigates the network communication bottleneck. Meanwhile, as the new protocol introduces complex computation for fewer output ciphertext, we observe new memory access challenges due to a high volume of weight plaintexts and intermediate ciphertexts. Hence, we further propose a lightweight compression system for the weight plaintexts, reducing memory traffic by 10 times, as well as a specialized dataflow to maximize on-chip data reuse of intermediate ciphertexts. Extensive experiments show that our framework outperforms the Cheetah baseline by at most 5.7 times on CPUs and 4.2 times with an accelerator.