# SHE: A Fast and Accurate Deep Neural Network for Encrypted Data

arXiv: https://arxiv.org/abs/1906.00148 · HTML: https://mlsanity.com/paper/1906.00148/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2019.

## Built on by (most cited first, 30 of 36)

- [Privacy-Preserving Machine Learning with Fully Homomorphic Encryption for Deep Neural Network](https://mlsanity.com/paper/2106.07229/) (2021)
- [A2Q: Accumulator-Aware Quantization with Guaranteed Overflow Avoidance](https://mlsanity.com/paper/2308.13504/) (2023)
- [Deep Neural Networks for Encrypted Inference with TFHE](https://mlsanity.com/paper/2302.10906/) (2023)
- [CryptoGCN: Fast and Scalable Homomorphically Encrypted Graph Convolutional Network Inference](https://mlsanity.com/paper/2209.11904/) (2022)
- [CipherPrune: Efficient and Scalable Private Transformer Inference](https://mlsanity.com/paper/2502.16782/) (2025)
- [HyPHEN: A Hybrid Packing Method and Optimizations for Homomorphic Encryption-Based Neural Networks](https://mlsanity.com/paper/2302.02407/) (2023)
- [FedVLN: Privacy-preserving Federated Vision-and-Language Navigation](https://mlsanity.com/paper/2203.14936/) (2022)
- [Hiding Visual Information via Obfuscating Adversarial Perturbations](https://mlsanity.com/paper/2209.15304/) (2022)
- [Private and Reliable Neural Network Inference](https://mlsanity.com/paper/2210.15614/) (2022)
- [PrivCirNet: Efficient Private Inference via Block Circulant Transformation](https://mlsanity.com/paper/2405.14569/) (2024)
- [Accumulator-Aware Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2409.17092/) (2024)
- [A2Q+: Improving Accumulator-Aware Weight Quantization](https://mlsanity.com/paper/2401.10432/) (2024)
- [RL-DistPrivacy: Privacy-Aware Distributed Deep Inference for low latency IoT systems](https://mlsanity.com/paper/2208.13032/) (2022)
- [TT-TFHE: a Torus Fully Homomorphic Encryption-Friendly Neural Network Architecture](https://mlsanity.com/paper/2302.01584/) (2023)
- [Optimized Layerwise Approximation for Efficient Private Inference on Fully Homomorphic Encryption](https://mlsanity.com/paper/2310.10349/) (2023)
- [TFHE-Coder: Evaluating LLM-agentic Fully Homomorphic Encryption Code Generation](https://mlsanity.com/paper/2503.12217/) (2025)
- [SoK: Privacy-preserving Deep Learning with Homomorphic Encryption](https://mlsanity.com/paper/2112.12855/) (2021)
- [Quantized Neural Networks for Low-Precision Accumulation with Guaranteed Overflow Avoidance](https://mlsanity.com/paper/2301.13376/) (2023)
- [CryptoTrain: Fast Secure Training on Encrypted Dataset](https://mlsanity.com/paper/2409.16675/) (2024)
- [Inhibitor Transformers and Gated RNNs for Torus Efficient Fully Homomorphic Encryption](https://mlsanity.com/paper/2308.05629/) (2023)
- [From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks](https://mlsanity.com/paper/2405.06038/) (2024)
- [Securing Transformer-based AI Execution via Unified TEEs and Crypto-protected Accelerators](https://mlsanity.com/paper/2507.03278/) (2025)
- [Efficient CNN Building Blocks for Encrypted Data](https://mlsanity.com/paper/2102.00319/) (2021)
- [Bi-CryptoNets: Leveraging Different-Level Privacy for Encrypted Inference](https://mlsanity.com/paper/2402.01296/) (2024)
- [DCT-CryptoNets: Scaling Private Inference in the Frequency Domain](https://mlsanity.com/paper/2408.15231/) (2024)
- [SoK: Towards Security and Safety of Edge AI](https://mlsanity.com/paper/2410.05349/) (2024)
- [Privacy-Preserving Chest X-ray Classification in Latent Space with Homomorphically Encrypted Neural Inference](https://mlsanity.com/paper/2506.15258/) (2025)
- [DictPFL: Efficient and Private Federated Learning on Encrypted Gradients](https://mlsanity.com/paper/2510.21086/) (2025)
- [On the Evaluation of User Privacy in Deep Neural Networks using Timing Side Channel](https://mlsanity.com/paper/2208.01113/) (2022)
- [The Inhibitor: ReLU and Addition-Based Attention for Efficient Transformers under Fully Homomorphic Encryption on the Torus](https://mlsanity.com/paper/2310.02041/) (2023)

## Abstract (arXiv)

Homomorphic Encryption (HE) is one of the most promising security solutions to emerging Machine Learning as a Service (MLaaS). Leveled-HE (LHE)-enabled Convolutional Neural Networks (LHECNNs) are proposed to implement MLaaS to avoid large bootstrapping overhead. However, prior LHECNNs have to pay significant computing overhead but achieve only low inference accuracy, due to their polynomial approximation activations and poolings. Stacking many polynomial approximation activation layers in a network greatly reduces inference accuracy, since the polynomial approximation activation errors lead to a low distortion of the output distribution of the next batch normalization layer. So the polynomial approximation activations and poolings have become the obstacle to a fast and accurate LHECNN model. In this paper, we propose a Shift-accumulation-based LHE-enabled deep neural network (SHE) for fast and accurate inferences on encrypted data. We use the binary-operation-friendly Leveled Fast Homomorphic Encryption over Torus (LTFHE) encryption scheme to implement ReLU activations and max poolings. We also adopt the logarithmic quantization to accelerate inferences by replacing expensive LTFHE multiplications with cheap LTFHE shifts. We propose a mixed bitwidth accumulator to accelerate accumulations. Since the LTFHE ReLU activations, max poolings, shifts and accumulations have small multiplicative depth overhead, SHE can implement much deeper network architectures with more convolutional and activation layers. Our experimental results show SHE achieves the state-of-the-art inference accuracy and reduces the inference latency by 76.21% ~ 94.23% over prior LHECNNs on MNIST and CIFAR-10. The source code of SHE is available at https://github.com/qianlou/SHE.