# Enabling Homomorphically Encrypted Inference for Large DNN Models

arXiv: https://arxiv.org/abs/2103.16139 · HTML: https://mlsanity.com/paper/2103.16139/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Gazelle: A Low Latency Framework for Secure Neural Network Inference](https://mlsanity.com/paper/1801.05507/) (2018)
- [A generic framework for privacy preserving deep learning](https://mlsanity.com/paper/1811.04017/) (2018)
- [CrypTFlow: Secure TensorFlow Inference](https://mlsanity.com/paper/1909.07814/) (2019)
- [Low Latency Privacy Preserving Inference](https://mlsanity.com/paper/1812.10659/) (2018)
- [CryptoDL: Deep Neural Networks over Encrypted Data](https://mlsanity.com/paper/1711.05189/) (2017)
- [XONN: XNOR-based Oblivious Deep Neural Network Inference](https://mlsanity.com/paper/1902.07342/) (2019)
- [Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference](https://mlsanity.com/paper/1811.09953/) (2018)
- [nGraph-HE: A Graph Compiler for Deep Learning on Homomorphically Encrypted Data](https://mlsanity.com/paper/1810.10121/) (2018)
- [Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning](https://mlsanity.com/paper/1801.08058/) (2018)
- [nGraph-HE2: A High-Throughput Framework for Neural Network Inference on Encrypted Data](https://mlsanity.com/paper/1908.04172/) (2019)
- [ARIANN: Low-Interaction Privacy-Preserving Deep Learning via Function Secret Sharing](https://mlsanity.com/paper/2006.04593/) (2020)
- [Private Machine Learning in TensorFlow using Secure Computation](https://mlsanity.com/paper/1810.08130/) (2018)
- [SEALion: a Framework for Neural Network Inference on Encrypted Data](https://mlsanity.com/paper/1904.12840/) (2019)

## Built on by (most cited first, 6 of 6)

- [A methodology for training homomorphicencryption friendly neural networks](https://mlsanity.com/paper/2111.03362/) (2021)
- [Training Large Scale Polynomial CNNs for E2E Inference over Homomorphic Encryption](https://mlsanity.com/paper/2304.14836/) (2023)
- [Syfer: Neural Obfuscation for Private Data Release](https://mlsanity.com/paper/2201.12406/) (2022)
- [A Scalable Multi-GPU Framework for Encrypted Large-Model Inference](https://mlsanity.com/paper/2512.11269/) (2025)
- [A Multivocal Literature Review on Privacy and Fairness in Federated Learning](https://mlsanity.com/paper/2408.08666/) (2024)
- [Privatar: Scalable Privacy-preserving Multi-user VR via Secure Offloading](https://mlsanity.com/paper/2604.17476/) (2026)

## Abstract (arXiv)

The proliferation of machine learning services in the last few years has raised data privacy concerns. Homomorphic encryption (HE) enables inference using encrypted data but it incurs 100x-10,000x memory and runtime overheads. Secure deep neural network (DNN) inference using HE is currently limited by computing and memory resources, with frameworks requiring hundreds of gigabytes of DRAM to evaluate small models. To overcome these limitations, in this paper we explore the feasibility of leveraging hybrid memory systems comprised of DRAM and persistent memory. In particular, we explore the recently-released Intel Optane PMem technology and the Intel HE-Transformer nGraph to run large neural networks such as MobileNetV2 (in its largest variant) and ResNet-50 for the first time in the literature. We present an in-depth analysis of the efficiency of the executions with different hardware and software configurations. Our results conclude that DNN inference using HE incurs on friendly access patterns for this memory configuration, yielding efficient executions.