# SoK: Privacy-preserving Deep Learning with Homomorphic Encryption

arXiv: https://arxiv.org/abs/2112.12855 · HTML: https://mlsanity.com/paper/2112.12855/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 47 references that are in the map.
- First posted on arXiv 23 Dec 2021.

## Builds on (most cited first, 13 of 13)

- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights](https://mlsanity.com/paper/1702.03044/) (2017)
- [Low Latency Privacy Preserving Inference](https://mlsanity.com/paper/1812.10659/) (2018)
- [CryptoDL: Deep Neural Networks over Encrypted Data](https://mlsanity.com/paper/1711.05189/) (2017)
- [Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference](https://mlsanity.com/paper/1811.09953/) (2018)
- [SHE: A Fast and Accurate Deep Neural Network for Encrypted Data](https://mlsanity.com/paper/1906.00148/) (2019)
- [nGraph-HE: A Graph Compiler for Deep Learning on Homomorphically Encrypted Data](https://mlsanity.com/paper/1810.10121/) (2018)
- [PrivFT: Private and Fast Text Classification with Homomorphic Encryption](https://mlsanity.com/paper/1908.06972/) (2019)
- [Towards the AlexNet Moment for Homomorphic Encryption: HCNN, theFirst Homomorphic CNN on Encrypted Data with GPUs](https://mlsanity.com/paper/1811.00778/) (2018)
- [SoK: Fully Homomorphic Encryption Compilers](https://mlsanity.com/paper/2101.07078/) (2021)
- [Encrypted Speech Recognition using Deep Polynomial Networks](https://mlsanity.com/paper/1905.05605/) (2019)
- [Neural Network Training With Homomorphic Encryption](https://mlsanity.com/paper/2012.13552/) (2020)
- [Reducing ReLU Count for Privacy-Preserving CNN Speedup](https://mlsanity.com/paper/2101.11835/) (2021)

## Built on by (most cited first, 3 of 3)

- [Training Large Scale Polynomial CNNs for E2E Inference over Homomorphic Encryption](https://mlsanity.com/paper/2304.14836/) (2023)
- [A Taxonomy of Attacks and Defenses in Split Learning](https://mlsanity.com/paper/2505.05872/) (2025)
- [A Graph-Based Analysis of Comparability, Scale, and Attribution in Private Transformer Inference](https://mlsanity.com/paper/2610.19607/) (2026)

## Abstract (arXiv)

Outsourced computation for neural networks allows users access to state of the art models without needing to invest in specialized hardware and know-how. The problem is that the users lose control over potentially privacy sensitive data. With homomorphic encryption (HE) computation can be performed on encrypted data without revealing its content. In this systematization of knowledge, we take an in-depth look at approaches that combine neural networks with HE for privacy preservation. We categorize the changes to neural network models and architectures to make them computable over HE and how these changes impact performance. We find numerous challenges to HE based privacy-preserving deep learning such as computational overhead, usability, and limitations posed by the encryption schemes.